[TestVis] Impacted tests POC #6366

daniel-romano-DD · 2024-11-27T21:52:30Z

Summary of changes

Implementer Milestones 1.X of Impacted Tests Detection RFC

Reason for change

Detecting which tests have changed in a commit and mark them will enable future actions over them, like launching Early Flake Detection

Implementation details

Milestone 1.5 requires having access to Git CLI, and being in a PR. Retrieves changed files and lines by Git Diff from base commit and intersects them with the test definition file. If collision is detected, the test is marked as modified.

Test coverage

Added unit and integration tests.
Integration tests relies on two predefined commits with known changes to certain tests, which get checked for the is_modifed tag in a verify snapshot.

Other details

andrewlock · 2024-11-27T22:29:07Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (68ms)  : 65, 71
     .   : milestone, 68,
    master - mean (68ms)  : 65, 71
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (976ms)  : 959, 994
     .   : milestone, 976,
    master - mean (979ms)  : 954, 1003
     .   : milestone, 979,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (107ms)  : 105, 109
     .   : milestone, 107,
    master - mean (107ms)  : 105, 109
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (676ms)  : 661, 692
     .   : milestone, 676,
    master - mean (678ms)  : 663, 693
     .   : milestone, 678,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (630ms)  : 616, 644
     .   : milestone, 630,
    master - mean (633ms)  : 618, 647
     .   : milestone, 633,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (194ms)  : 189, 199
     .   : milestone, 194,
    master - mean (195ms)  : 190, 200
     .   : milestone, 195,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (1,096ms)  : 1069, 1123
     .   : milestone, 1096,
    master - mean (1,099ms)  : 1071, 1127
     .   : milestone, 1099,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (279ms)  : 274, 283
     .   : milestone, 279,
    master - mean (278ms)  : 273, 283
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (872ms)  : 835, 908
     .   : milestone, 872,
    master - mean (876ms)  : 839, 912
     .   : milestone, 876,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6366) - mean (267ms)  : 263, 271
     .   : milestone, 267,
    master - mean (268ms)  : 263, 272
     .   : milestone, 268,

    section CallTarget+Inlining+NGEN
    This PR (6366) - mean (845ms)  : 806, 884
     .   : milestone, 845,
    master - mean (853ms)  : 809, 896
     .   : milestone, 853,

datadog-ddstaging · 2024-11-27T22:41:31Z

Datadog Report

Branch report: dani/testvis/impacted_tests_poc
Commit report: be6dd91
Test service: dd-trace-dotnet

✅ 0 Failed, 453123 Passed, 2889 Skipped, 19h 32m 42.49s Total Time

andrewlock · 2024-11-27T22:48:30Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6366) (11.178M)   : 0, 11178280
    master (11.161M)   : 0, 11161110
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6366) (7.155M)   : 0, 7155342
    master (7.229M)   : 0, 7229017
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.435M)   : 0, 7435310

    section Manual
    master (11.146M)   : 0, 11146248

    section Manual + Automatic
    This PR (6366) (6.641M)   : 0, 6640918
    master (6.610M)   : 0, 6610117

    section DD_TRACE_ENABLED=0
    master (10.159M)   : 0, 10159204

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6366) (9.667M)   : 0, 9667022
    master (9.601M)   : 0, 9600540
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6366) (6.374M)   : 0, 6373833
    master (6.366M)   : 0, 6365756

    section Trace stats
    master (6.424M)   : 0, 6424147

    section Manual
    master (9.621M)   : 0, 9620773

    section Manual + Automatic
    This PR (6366) (5.961M)   : 0, 5960608
    master (5.938M)   : 0, 5937506

    section DD_TRACE_ENABLED=0
    master (8.890M)   : 0, 8889593

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6366) (9.140M)   : 0, 9140405
    master (10.145M)   : 0, 10145415
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6366) (5.975M)   : crit ,0, 5975345
    master (6.521M)   : 0, 6521495
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.245M)   : 0, 7245193

    section Manual
    master (10.135M)   : 0, 10135197

    section Manual + Automatic
    This PR (6366) (5.746M)   : crit ,0, 5745941
    master (6.058M)   : 0, 6057608

    section DD_TRACE_ENABLED=0
    master (9.431M)   : 0, 9430769

andrewlock · 2024-11-27T22:56:53Z

Benchmarks Report for tracer 🐌

Benchmarks for #6366 compared to master:

1 benchmarks are faster, with geometric mean 1.177
1 benchmarks are slower, with geometric mean 1.174
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.15μs	45.7ns	306ns	0.0198	0.00794	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.3μs	56.5ns	348ns	0.0253	0.0101	0	5.8 KB
master	`StartStopWithChild`	net472	16.3μs	40.8ns	158ns	1.03	0.298	0.0968	6.22 KB
#6366	`StartStopWithChild`	net6.0	8.02μs	45.8ns	366ns	0.0165	0.00413	0	5.62 KB
#6366	`StartStopWithChild`	netcoreapp3.1	10.3μs	53.6ns	251ns	0.021	0.0105	0	5.8 KB
#6366	`StartStopWithChild`	net472	16.3μs	54.9ns	212ns	1.04	0.291	0.089	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	480μs	411ns	1.59μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	659μs	190ns	712ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	855μs	573ns	2.14μs	0.428	3.3 KB
#6366	`WriteAndFlushEnrichedTraces`	net6.0	491μs	201ns	725ns	0	2.7 KB
#6366	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	653μs	469ns	1.82μs	0	2.7 KB
#6366	`WriteAndFlushEnrichedTraces`	net472	861μs	910ns	3.52μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	157μs	1.01μs	10.1μs	0.147	14.47 KB
master	`SendRequest`	netcoreapp3.1	183μs	1.15μs	11.2μs	0.192	17.27 KB
master	`SendRequest`	net472	0.00354ns	0.00107ns	0.00414ns	0	0 b
#6366	`SendRequest`	net6.0	154μs	1.04μs	10.3μs	0.164	14.47 KB
#6366	`SendRequest`	netcoreapp3.1	177μs	1μs	7μs	0.155	17.27 KB
#6366	`SendRequest`	net472	0.00229ns	0.000763ns	0.00295ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6366

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.96 KB	41.74 KB	-215 B	-0.51%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	562μs	2.8μs	15.1μs	0.556	0	0	41.67 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	700μs	3.79μs	26.8μs	0.44	0	0	41.96 KB
master	`WriteAndFlushEnrichedTraces`	net472	837μs	3.62μs	16.6μs	8.33	2.5	0.417	53.26 KB
#6366	`WriteAndFlushEnrichedTraces`	net6.0	601μs	3.16μs	16.7μs	0.628	0	0	41.74 KB
#6366	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	684μs	3.93μs	30.5μs	0.363	0	0	41.74 KB
#6366	`WriteAndFlushEnrichedTraces`	net472	838μs	4.04μs	17.1μs	8.33	2.5	0.417	53.31 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.34μs	0.918ns	3.31ns	0.014	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.82μs	1.97ns	7.62ns	0.0136	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.09μs	2.62ns	9.81ns	0.157	0.00104	987 B
#6366	`ExecuteNonQuery`	net6.0	1.32μs	1.61ns	6.24ns	0.0145	0	1.02 KB
#6366	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	1.53ns	5.74ns	0.0131	0	1.02 KB
#6366	`ExecuteNonQuery`	net472	2.08μs	1.15ns	4.47ns	0.157	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.28μs	0.51ns	1.91ns	0.0135	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.61μs	1.25ns	4.84ns	0.0129	976 B
master	`CallElasticsearch`	net472	2.54μs	2.54ns	9.83ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.3μs	0.834ns	3.12ns	0.013	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	1.93ns	7.48ns	0.0134	1.02 KB
master	`CallElasticsearchAsync`	net472	2.62μs	2.87ns	10.7ns	0.166	1.05 KB
#6366	`CallElasticsearch`	net6.0	1.25μs	0.427ns	1.66ns	0.0138	976 B
#6366	`CallElasticsearch`	netcoreapp3.1	1.56μs	0.839ns	3.25ns	0.0133	976 B
#6366	`CallElasticsearch`	net472	2.4μs	1.08ns	4.18ns	0.157	995 B
#6366	`CallElasticsearchAsync`	net6.0	1.39μs	0.736ns	2.85ns	0.013	952 B
#6366	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	0.613ns	2.29ns	0.0139	1.02 KB
#6366	`CallElasticsearchAsync`	net472	2.52μs	1.52ns	5.7ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.31μs	3.34ns	12.5ns	0.0131	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.56μs	1.35ns	4.86ns	0.0124	952 B
master	`ExecuteAsync`	net472	1.81μs	0.341ns	1.23ns	0.145	915 B
#6366	`ExecuteAsync`	net6.0	1.26μs	1.05ns	3.93ns	0.0132	952 B
#6366	`ExecuteAsync`	netcoreapp3.1	1.66μs	0.588ns	2.12ns	0.0126	952 B
#6366	`ExecuteAsync`	net472	1.78μs	0.543ns	2.1ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.46μs	1.75ns	6.77ns	0.031	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.25μs	2.15ns	8.33ns	0.0369	2.85 KB
master	`SendAsync`	net472	7.22μs	1.16ns	4.5ns	0.493	3.12 KB
#6366	`SendAsync`	net6.0	4.62μs	1.96ns	7.08ns	0.0323	2.31 KB
#6366	`SendAsync`	netcoreapp3.1	5.28μs	2.03ns	7.58ns	0.0369	2.85 KB
#6366	`SendAsync`	net472	7.43μs	1.79ns	6.7ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.44μs	0.662ns	2.48ns	0.0232	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.36μs	1.04ns	3.88ns	0.0212	1.64 KB
master	`EnrichedLog`	net472	2.8μs	1.18ns	4.57ns	0.248	1.57 KB
#6366	`EnrichedLog`	net6.0	1.46μs	0.521ns	2.02ns	0.0228	1.64 KB
#6366	`EnrichedLog`	netcoreapp3.1	2.14μs	1.08ns	4.17ns	0.0224	1.64 KB
#6366	`EnrichedLog`	net472	2.74μs	1.08ns	4.04ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	117μs	191ns	739ns	0.0586	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	122μs	210ns	757ns	0	0	4.28 KB
master	`EnrichedLog`	net472	153μs	77.9ns	281ns	0.692	0.231	4.46 KB
#6366	`EnrichedLog`	net6.0	120μs	204ns	790ns	0.0599	0	4.28 KB
#6366	`EnrichedLog`	netcoreapp3.1	121μs	341ns	1.32μs	0	0	4.28 KB
#6366	`EnrichedLog`	net472	154μs	145ns	562ns	0.687	0.229	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.92μs	0.994ns	3.85ns	0.0307	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.39μs	2.02ns	7.82ns	0.0306	2.2 KB
master	`EnrichedLog`	net472	4.9μs	0.713ns	2.57ns	0.319	2.02 KB
#6366	`EnrichedLog`	net6.0	3.04μs	0.827ns	3.2ns	0.0315	2.2 KB
#6366	`EnrichedLog`	netcoreapp3.1	4.22μs	2.46ns	8.52ns	0.0295	2.2 KB
#6366	`EnrichedLog`	net472	4.88μs	1.06ns	4.12ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.44μs	0.62ns	2.4ns	0.0159	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.83μs	1.79ns	6.93ns	0.0156	1.14 KB
master	`SendReceive`	net472	2.11μs	1.45ns	5.43ns	0.183	1.16 KB
#6366	`SendReceive`	net6.0	1.44μs	1.05ns	4.08ns	0.0158	1.14 KB
#6366	`SendReceive`	netcoreapp3.1	1.81μs	1.06ns	4.12ns	0.0154	1.14 KB
#6366	`SendReceive`	net472	2.11μs	1.16ns	4.49ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.81μs	0.832ns	3.22ns	0.0224	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.91μs	1.98ns	7.65ns	0.0215	1.65 KB
master	`EnrichedLog`	net472	4.4μs	3.23ns	12.1ns	0.322	2.04 KB
#6366	`EnrichedLog`	net6.0	2.72μs	1.01ns	3.92ns	0.0218	1.6 KB
#6366	`EnrichedLog`	netcoreapp3.1	4.03μs	1.64ns	6.36ns	0.0221	1.65 KB
#6366	`EnrichedLog`	net472	4.45μs	2.47ns	9.55ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6366

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.174	678.75	796.93

Faster 🎉 in #6366

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.177	742.85	631.07

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	431ns	0.455ns	1.76ns	0.00811	576 B
master	`StartFinishSpan`	netcoreapp3.1	593ns	0.48ns	1.79ns	0.00793	576 B
master	`StartFinishSpan`	net472	743ns	0.662ns	2.48ns	0.0915	578 B
master	`StartFinishScope`	net6.0	558ns	0.323ns	1.25ns	0.0097	696 B
master	`StartFinishScope`	netcoreapp3.1	679ns	0.428ns	1.6ns	0.00946	696 B
master	`StartFinishScope`	net472	872ns	1.11ns	4.31ns	0.104	658 B
#6366	`StartFinishSpan`	net6.0	398ns	0.261ns	1.01ns	0.00803	576 B
#6366	`StartFinishSpan`	netcoreapp3.1	571ns	0.449ns	1.74ns	0.00771	576 B
#6366	`StartFinishSpan`	net472	631ns	0.64ns	2.48ns	0.0916	578 B
#6366	`StartFinishScope`	net6.0	545ns	0.464ns	1.8ns	0.00975	696 B
#6366	`StartFinishScope`	netcoreapp3.1	797ns	0.556ns	2.15ns	0.00914	696 B
#6366	`StartFinishScope`	net472	812ns	0.624ns	2.42ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	582ns	0.426ns	1.65ns	0.00985	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	863ns	0.584ns	2.26ns	0.00926	696 B
master	`RunOnMethodBegin`	net472	1.1μs	1.16ns	4.48ns	0.104	658 B
#6366	`RunOnMethodBegin`	net6.0	604ns	0.469ns	1.81ns	0.00985	696 B
#6366	`RunOnMethodBegin`	netcoreapp3.1	929ns	0.726ns	2.72ns	0.00906	696 B
#6366	`RunOnMethodBegin`	net472	1.08μs	0.483ns	1.81ns	0.104	658 B

tracer/src/Datadog.Trace/Ci/Coverage/Util/FileBitmap.cs

tonyredondo · 2024-12-17T14:11:25Z

tracer/src/Datadog.Trace/Ci/Coverage/Util/FileBitmap.cs

+    /// <summary>
+    /// Gets if two bitmaps have at least one enabled bit in common.
+    /// </summary>
+    /// <returns>True if both bitmaps has at least 1 bit set to 1 in the same position; otherwise, false..</returns>
+    public bool IntersectsWith(ref FileBitmap other)
+    {
+        var size = Math.Min(_size, other._size);
+
+        // Iterate over each byte of the bitmap
+        for (var i = 0; i < size; i++)
+        {
+            if ((_bitmap[i] & other._bitmap[i]) > 0)
+            {
+                return true;
+            }
+        }
+
+        return false;
+    }


for later: improve this algorithm based on AND method in the same type.

tracer/src/Datadog.Trace/Ci/GitCommandHelper.cs

tracer/src/Datadog.Trace/Ci/ImpactedTestsModule.cs

tracer/src/Datadog.Trace/Ci/GitCommandHelper.cs

tracer/src/Datadog.Trace/Ci/ImpactedTestsModule.cs

tonyredondo · 2024-12-17T15:11:25Z

tracer/src/Datadog.Trace/Ci/IntelligentTestRunnerClient.cs

+                    {
+                        if (response.ContentLength is { } contentLength and > 0)
+                        {
+                            // TODO: Check for compressed responses - currently these are not handled and will throw when we attempt to deserialize


Compressed responses should be handled by the HttpClient AutomaticDecompression flag

tracer/src/Datadog.Trace/Ci/IntelligentTestRunnerClient.cs

WIP FileBitmap tests added Get modified lines from git diff Added metrics Record telemetry event Integration tests Fixed relative paths Fixed range Fix error

Git command helper Trimming file fix Fixed telemetry intake Added missing metric in json Restored test files Milestone 1.0 support Rebase fix Fixed tests FileBitmap suggestions PR comments Updated metrics Better traces Limit test to windows because 'git branch --get-current' returns empty due to a possible detached head status on linux Fixed tests in linux

tonyredondo · 2025-01-13T11:15:44Z

tracer/src/Datadog.Trace/Ci/ImpactedTestsModule.cs

+        if (CIVisibility.ImpactedTestsDetectionResponse is { } response && response.Files is { Length: > 0 } files)
+        {
+            var res = new List<FileCoverageInfo>();
+            foreach (var file in files)
+            {
+                res.Add(new FileCoverageInfo(file));
+            }
+
+            return res.ToArray();
+        }


We have the Length, we can create the FileCoverageInfo[] array and avoid creating the list, resizing the list and the calling ToArray()

daniel-romano-DD force-pushed the dani/testvis/impacted_tests_poc branch from b39ff49 to 9a6741f Compare December 17, 2024 13:40

tonyredondo reviewed Dec 17, 2024

View reviewed changes

daniel-romano-DD force-pushed the dani/testvis/impacted_tests_poc branch from 349dc7f to bb439b4 Compare December 18, 2024 09:23

daniel-romano-DD marked this pull request as ready for review December 18, 2024 11:58

daniel-romano-DD requested review from a team as code owners December 18, 2024 11:58

daniel-romano-DD force-pushed the dani/testvis/impacted_tests_poc branch 3 times, most recently from 7b64f58 to 20cf3d9 Compare December 23, 2024 16:26

daniel-romano-DD added 3 commits December 24, 2024 10:37

Initial impl

164aed5

WIP FileBitmap tests added Get modified lines from git diff Added metrics Record telemetry event Integration tests Fixed relative paths Fixed range Fix error

Modified test files for testing purpose

340fa40

Reverted modified test files

c7fd869

daniel-romano-DD force-pushed the dani/testvis/impacted_tests_poc branch from 1ca8a31 to ed3e8c1 Compare December 24, 2024 09:37

daniel-romano-DD added 6 commits December 24, 2024 10:37

Updated testing commit shas

6eb87de

Debug test in linux

e52c672

Improved git command traces

6b9880d

Added git availability checks

ace72bb

Fixed ammend command

be6dd91

tonyredondo reviewed Jan 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TestVis] Impacted tests POC #6366

[TestVis] Impacted tests POC #6366

daniel-romano-DD commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #6366

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6366

Faster 🎉 in #6366

Raw results

Raw results

tonyredondo Dec 17, 2024

tonyredondo Dec 17, 2024

tonyredondo Jan 13, 2025

[TestVis] Impacted tests POC #6366

Are you sure you want to change the base?

[TestVis] Impacted tests POC #6366

Conversation

daniel-romano-DD commented Nov 27, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock commented Nov 27, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Nov 27, 2024 • edited Loading

Datadog Report

andrewlock commented Nov 27, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Nov 27, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #6366

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6366

Faster 🎉 in #6366

Raw results

Raw results

tonyredondo Dec 17, 2024

Choose a reason for hiding this comment

tonyredondo Dec 17, 2024

Choose a reason for hiding this comment

tonyredondo Jan 13, 2025

Choose a reason for hiding this comment

daniel-romano-DD commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading

andrewlock commented Nov 27, 2024 •

edited

Loading