Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APMSP-1543 feat: setup boilerplate for data-pipeline integration #6314

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

ganeshnj
Copy link

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@ganeshnj ganeshnj changed the base branch from master to feature/data-pipeline November 19, 2024 14:01
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from 9fdb202 to b4f83a1 Compare November 19, 2024 14:03
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 19, 2024

Datadog Report

Branch report: ganeshnj/feature/datapipeline-bindings
Commit report: bf49f93
Test service: dd-trace-dotnet

❌ 4031 Failed (2 Known Flaky), 50712 Passed, 142 Skipped, 9h 8m 52.07s Total Time
⌛ 2 Performance Regressions

❌ Failed Tests (4031)

This report shows up to 5 failed tests.

  • StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

  • StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

  • StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

  • AzureFunctionsTests+InProcessRuntimeV4.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Details

    Expand for error
     Expected collection to contain at least 21 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
    
  • AzureFunctionsTests+IsolatedRuntimeV4.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Details

    Expand for error
     Expected collection to contain at least 21 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
    

⌛ Performance Regressions vs Default Branch (2)

  • Baseline - Samples.FakeDbCommand.windows.netcoreapp31.json.scenarios 110.63ms (+3.31ms, +3%) - Details
  • Baseline - Samples.FakeDbCommand.windows.net60.json.scenarios 94.25ms (+3.24ms, +4%) - Details

@ganeshnj ganeshnj changed the title feat: setup boilerplate for data-pipeline integration APMSP-1543 feat: setup boilerplate for data-pipeline ®integration Nov 25, 2024
@ganeshnj ganeshnj changed the title APMSP-1543 feat: setup boilerplate for data-pipeline ®integration APMSP-1543 feat: setup boilerplate for data-pipeline integration Nov 25, 2024
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from 968ca73 to 31c6659 Compare November 25, 2024 16:33
Copy link
Contributor

github-actions bot commented Nov 25, 2024

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

2 occurrences of :

-    Error: 1,
[...]
-      error.msg: The HTTP response has status code 500.,
[...]
-      http.status_code: 500,
+      http.status_code: 200,

6 occurrences of :

-    Resource: Uncompleted operation,
+    Resource: Query ErrorQuery,

6 occurrences of :

-      graphql.operation.type: Uncompleted,
+      graphql.operation.name: ErrorQuery,
+      graphql.operation.type: Query,
[...]
-  throwNotImplementedException {
-    name
-  }
[...]
+  throwException

2 occurrences of :

-      http.status_code: 400,
+      http.status_code: 200,

2 occurrences of :

-    },
-    MetaStruct: {
-      iast: 

@ganeshnj ganeshnj changed the base branch from feature/data-pipeline to master November 25, 2024 16:39
@ganeshnj ganeshnj changed the base branch from master to feature/data-pipeline November 25, 2024 16:39
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch 3 times, most recently from a5b2c7c to f3dcd3b Compare December 3, 2024 09:29
@andrewlock
Copy link
Member

andrewlock commented Dec 3, 2024

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6314) (11.139M)   : 0, 11138618
    master (11.080M)   : 0, 11079759
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6314) (10.962M)   : 0, 10962124
    master (7.154M)   : 0, 7154253
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.512M)   : 0, 7512246

    section Manual
    master (11.035M)   : 0, 11035311

    section Manual + Automatic
    This PR (6314) (10.864M)   : 0, 10863755
    master (6.611M)   : 0, 6611049

    section DD_TRACE_ENABLED=0
    master (10.155M)   : 0, 10155447

Loading
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6314) (9.562M)   : 0, 9562465
    master (9.567M)   : 0, 9566967
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6314) (9.318M)   : 0, 9318314
    master (6.361M)   : 0, 6360592

    section Trace stats
    master (6.540M)   : 0, 6540289

    section Manual
    master (9.643M)   : 0, 9642857

    section Manual + Automatic
    This PR (6314) (9.352M)   : 0, 9352441
    master (5.877M)   : 0, 5877320

    section DD_TRACE_ENABLED=0
    master (8.928M)   : 0, 8928120

Loading
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6314) (10.625M)   : 0, 10625387
    master (9.916M)   : 0, 9915888
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6314) (9.832M)   : 0, 9832043
    master (6.337M)   : 0, 6336520
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (6.963M)   : 0, 6963468

    section Manual
    master (9.842M)   : 0, 9842464

    section Manual + Automatic
    This PR (6314) (9.706M)   : 0, 9705884
    master (6.172M)   : 0, 6171910

    section DD_TRACE_ENABLED=0
    master (9.319M)   : 0, 9318507

Loading

@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch 2 times, most recently from b47644d to 356bf2b Compare December 3, 2024 13:58
@ganeshnj ganeshnj changed the base branch from feature/data-pipeline to master December 3, 2024 13:58
@ganeshnj ganeshnj changed the base branch from master to feature/data-pipeline December 3, 2024 13:58
@andrewlock
Copy link
Member

andrewlock commented Dec 3, 2024

Benchmarks Report for tracer 🐌

Benchmarks for #6314 compared to master:

  • 3 benchmarks are slower, with geometric mean 1.167
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.94μs 45.2ns 329ns 0.0161 0.00805 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.3μs 57.6ns 387ns 0.0203 0.0101 0 5.8 KB
master StartStopWithChild net472 16.6μs 49.1ns 190ns 1.02 0.275 0.0729 6.21 KB
#6314 StartStopWithChild net6.0 8.03μs 46ns 353ns 0.0119 0.00395 0 5.61 KB
#6314 StartStopWithChild netcoreapp3.1 10.1μs 47.1ns 226ns 0.0149 0.00498 0 5.8 KB
#6314 StartStopWithChild net472 16.7μs 51.2ns 198ns 1.05 0.314 0.0991 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 492μs 372ns 1.39μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 658μs 480ns 1.86μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 841μs 806ns 3.12μs 0.419 0 0 3.3 KB
#6314 WriteAndFlushEnrichedTraces net6.0 482μs 157ns 608ns 0 0 0 2.7 KB
#6314 WriteAndFlushEnrichedTraces netcoreapp3.1 636μs 731ns 2.74μs 0 0 0 2.7 KB
#6314 WriteAndFlushEnrichedTraces net472 844μs 368ns 1.38μs 0.425 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 158μs 1.22μs 11.9μs 0.132 0 0 14.47 KB
master SendRequest netcoreapp3.1 183μs 1.13μs 11.2μs 0.177 0 0 17.27 KB
master SendRequest net472 0.00649ns 0.00221ns 0.00855ns 0 0 0 0 b
#6314 SendRequest net6.0 171μs 1.59μs 15.8μs 0.181 0 0 14.47 KB
#6314 SendRequest netcoreapp3.1 197μs 1.62μs 15.9μs 0.184 0 0 17.27 KB
#6314 SendRequest net472 0.00284ns 0.000977ns 0.00338ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6314

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.77 KB 41.5 KB -261 B -0.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 591μs 3.39μs 29.8μs 0.607 0 0 41.77 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 659μs 3.04μs 11.8μs 0.331 0 0 41.81 KB
master WriteAndFlushEnrichedTraces net472 841μs 3.59μs 13.4μs 8.75 2.5 0.417 53.31 KB
#6314 WriteAndFlushEnrichedTraces net6.0 573μs 3.31μs 28.8μs 0.336 0 0 41.5 KB
#6314 WriteAndFlushEnrichedTraces netcoreapp3.1 711μs 4.11μs 35.8μs 0.372 0 0 41.78 KB
#6314 WriteAndFlushEnrichedTraces net472 841μs 1.28μs 4.61μs 8.39 2.52 0.419 53.31 KB
Benchmarks.Trace.DbCommandBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6314

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 1.156 1,194.13 1,380.86

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.2μs 1.58ns 5.9ns 0.0146 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.74μs 3.65ns 13.7ns 0.0131 0 0 1.02 KB
master ExecuteNonQuery net472 2.19μs 1.56ns 5.85ns 0.156 0.00109 0 987 B
#6314 ExecuteNonQuery net6.0 1.38μs 0.842ns 3.04ns 0.0145 0 0 1.02 KB
#6314 ExecuteNonQuery netcoreapp3.1 1.81μs 1.52ns 5.87ns 0.0138 0 0 1.02 KB
#6314 ExecuteNonQuery net472 2.05μs 1.84ns 7.14ns 0.156 0.00102 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.24μs 0.751ns 2.81ns 0.0137 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.63μs 1.46ns 5.46ns 0.0135 0 0 976 B
master CallElasticsearch net472 2.55μs 1.14ns 4.26ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.35μs 0.551ns 2.06ns 0.0129 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.67μs 0.96ns 3.59ns 0.0142 0 0 1.02 KB
master CallElasticsearchAsync net472 2.76μs 3.41ns 13.2ns 0.167 0 0 1.05 KB
#6314 CallElasticsearch net6.0 1.17μs 0.735ns 2.75ns 0.0135 0 0 976 B
#6314 CallElasticsearch netcoreapp3.1 1.57μs 0.533ns 1.99ns 0.0134 0 0 976 B
#6314 CallElasticsearch net472 2.57μs 1.28ns 4.79ns 0.157 0 0 995 B
#6314 CallElasticsearchAsync net6.0 1.27μs 0.657ns 2.46ns 0.0135 0 0 952 B
#6314 CallElasticsearchAsync netcoreapp3.1 1.71μs 0.952ns 3.69ns 0.0136 0 0 1.02 KB
#6314 CallElasticsearchAsync net472 2.69μs 1.67ns 6.47ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.17μs 0.484ns 1.81ns 0.013 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.66μs 0.981ns 3.67ns 0.0124 0 0 952 B
master ExecuteAsync net472 1.83μs 0.611ns 2.12ns 0.145 0 0 915 B
#6314 ExecuteAsync net6.0 1.25μs 0.907ns 3.27ns 0.0133 0 0 952 B
#6314 ExecuteAsync netcoreapp3.1 1.69μs 0.882ns 3.05ns 0.0126 0 0 952 B
#6314 ExecuteAsync net472 1.85μs 0.684ns 2.56ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.4μs 0.958ns 3.71ns 0.0309 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.3μs 1.78ns 6.65ns 0.0372 0 0 2.85 KB
master SendAsync net472 7.36μs 1.44ns 5.59ns 0.493 0 0 3.12 KB
#6314 SendAsync net6.0 4.39μs 1.66ns 6.41ns 0.0329 0 0 2.31 KB
#6314 SendAsync netcoreapp3.1 5.24μs 1.1ns 4.25ns 0.0366 0 0 2.85 KB
#6314 SendAsync net472 7.35μs 2.42ns 9.05ns 0.496 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.54μs 0.428ns 1.54ns 0.0238 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.1μs 1.29ns 4.84ns 0.0212 0 0 1.64 KB
master EnrichedLog net472 2.74μs 1.49ns 5.75ns 0.249 0 0 1.57 KB
#6314 EnrichedLog net6.0 1.53μs 0.671ns 2.51ns 0.0229 0 0 1.64 KB
#6314 EnrichedLog netcoreapp3.1 2.27μs 1.03ns 3.87ns 0.0226 0 0 1.64 KB
#6314 EnrichedLog net472 2.62μs 1.08ns 4.04ns 0.25 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 118μs 168ns 651ns 0.0593 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 122μs 166ns 644ns 0.0608 0 0 4.28 KB
master EnrichedLog net472 153μs 100ns 389ns 0.687 0.229 0 4.46 KB
#6314 EnrichedLog net6.0 116μs 64.3ns 241ns 0.058 0 0 4.28 KB
#6314 EnrichedLog netcoreapp3.1 122μs 176ns 681ns 0 0 0 4.28 KB
#6314 EnrichedLog net472 151μs 241ns 933ns 0.688 0.229 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.97μs 0.861ns 3.34ns 0.0311 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.13μs 2.47ns 9.59ns 0.0287 0 0 2.2 KB
master EnrichedLog net472 4.92μs 1.14ns 4.4ns 0.32 0 0 2.02 KB
#6314 EnrichedLog net6.0 3.04μs 1.09ns 4.23ns 0.0303 0 0 2.2 KB
#6314 EnrichedLog netcoreapp3.1 4.17μs 1.64ns 6.35ns 0.0291 0 0 2.2 KB
#6314 EnrichedLog net472 4.91μs 1.57ns 6.1ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.34μs 0.405ns 1.57ns 0.016 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.83μs 0.768ns 2.97ns 0.0155 0 0 1.14 KB
master SendReceive net472 2.11μs 1.62ns 6.29ns 0.183 0 0 1.16 KB
#6314 SendReceive net6.0 1.36μs 0.579ns 2.17ns 0.0163 0 0 1.14 KB
#6314 SendReceive netcoreapp3.1 1.76μs 0.672ns 2.51ns 0.015 0 0 1.14 KB
#6314 SendReceive net472 2.13μs 1.39ns 5.39ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.75μs 1.09ns 4.22ns 0.022 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.99μs 2.05ns 7.96ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.44μs 1.7ns 6.37ns 0.322 0 0 2.04 KB
#6314 EnrichedLog net6.0 2.65μs 0.886ns 3.31ns 0.0226 0 0 1.6 KB
#6314 EnrichedLog netcoreapp3.1 3.91μs 6.06ns 23.5ns 0.0214 0 0 1.65 KB
#6314 EnrichedLog net472 4.35μs 2.44ns 8.79ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6314

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.140 673.92 768.47

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 394ns 0.353ns 1.37ns 0.008 0 0 576 B
master StartFinishSpan netcoreapp3.1 550ns 0.348ns 1.3ns 0.00771 0 0 576 B
master StartFinishSpan net472 638ns 0.795ns 3.08ns 0.0917 0 0 578 B
master StartFinishScope net6.0 554ns 0.328ns 1.27ns 0.00975 0 0 696 B
master StartFinishScope netcoreapp3.1 674ns 0.368ns 1.43ns 0.00922 0 0 696 B
master StartFinishScope net472 895ns 2.3ns 8.9ns 0.104 0 0 658 B
#6314 StartFinishSpan net6.0 401ns 0.437ns 1.69ns 0.00809 0 0 576 B
#6314 StartFinishSpan netcoreapp3.1 586ns 0.646ns 2.42ns 0.00766 0 0 576 B
#6314 StartFinishSpan net472 648ns 0.619ns 2.4ns 0.0917 0 0 578 B
#6314 StartFinishScope net6.0 561ns 0.42ns 1.62ns 0.00975 0 0 696 B
#6314 StartFinishScope netcoreapp3.1 768ns 0.768ns 2.97ns 0.00924 0 0 696 B
#6314 StartFinishScope net472 856ns 0.984ns 3.81ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6314

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.205 584.44 704.29

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 589ns 2.18ns 8.43ns 0.00964 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 911ns 0.626ns 2.42ns 0.00916 0 0 696 B
master RunOnMethodBegin net472 1.09μs 0.511ns 1.91ns 0.104 0 0 658 B
#6314 RunOnMethodBegin net6.0 704ns 0.27ns 1.05ns 0.00957 0 0 696 B
#6314 RunOnMethodBegin netcoreapp3.1 967ns 2.33ns 9.02ns 0.00932 0 0 696 B
#6314 RunOnMethodBegin net472 1.14μs 0.588ns 2.28ns 0.104 0 0 658 B

@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from 555cc8c to 23abd8b Compare December 6, 2024 14:27
@ganeshnj ganeshnj changed the base branch from feature/data-pipeline to master December 9, 2024 16:19
@ganeshnj ganeshnj changed the base branch from master to feature/data-pipeline December 9, 2024 16:19
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from 23abd8b to cf6363a Compare December 16, 2024 16:20
@ganeshnj ganeshnj changed the base branch from feature/data-pipeline to master December 16, 2024 16:26
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from cf6363a to eafc440 Compare December 16, 2024 18:55
@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (68ms)  : 66, 71
     .   : milestone, 68,
    master - mean (68ms)  : 66, 71
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (1,000ms)  : 978, 1022
     .   : milestone, 1000,
    master - mean (978ms)  : 955, 1000
     .   : milestone, 978,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (107ms)  : 105, 110
     .   : milestone, 107,
    master - mean (107ms)  : 105, 109
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (702ms)  : 679, 726
     .   : milestone, 702,
    master - mean (679ms)  : 662, 696
     .   : milestone, 679,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (662ms)  : 634, 689
     .   : milestone, 662,
    master - mean (637ms)  : 617, 656
     .   : milestone, 637,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (193ms)  : 189, 198
     .   : milestone, 193,
    master - mean (194ms)  : 189, 199
     .   : milestone, 194,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (1,124ms)  : 1095, 1153
     .   : milestone, 1124,
    master - mean (1,099ms)  : 1063, 1134
     .   : milestone, 1099,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (278ms)  : 274, 282
     .   : milestone, 278,
    master - mean (277ms)  : 271, 284
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (894ms)  : 865, 923
     .   : milestone, 894,
    master - mean (873ms)  : 848, 898
     .   : milestone, 873,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (266ms)  : 261, 270
     .   : milestone, 266,
    master - mean (271ms)  : 265, 276
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (875ms)  : 850, 899
     .   : milestone, 875,
    master - mean (856ms)  : 826, 886
     .   : milestone, 856,

Loading

@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch 3 times, most recently from ad47914 to 5cc85f9 Compare December 18, 2024 09:43
@ganeshnj ganeshnj force-pushed the ganeshnj/feature/datapipeline-bindings branch from 22b9f30 to 067753f Compare December 19, 2024 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants