OpenTelemetryLoggerOptions.IncludeScopes does not add scopes to logs

[mtaron]

Package

OpenTelemetry

Package Version

❯ dotnet list package --include-transitive
Project 'asp-logging' has the following package references
   [net8.0]: 
   Top-level Package                             Requested   Resolved
   > Azure.Monitor.OpenTelemetry.AspNetCore      1.1.1       1.1.1   
   > OpenTelemetry.Exporter.Console              1.8.1       1.8.1   

   Transitive Package                                           Resolved
   > Azure.Core                                                 1.37.0  
   > Azure.Monitor.OpenTelemetry.Exporter                       1.2.0   
   > Microsoft.Bcl.AsyncInterfaces                              1.1.1   
   > Microsoft.Extensions.Configuration                         8.0.0   
   > Microsoft.Extensions.Configuration.Abstractions            8.0.0   
   > Microsoft.Extensions.Configuration.Binder                  8.0.0   
   > Microsoft.Extensions.DependencyInjection                   8.0.0   
   > Microsoft.Extensions.DependencyInjection.Abstractions      8.0.0   
   > Microsoft.Extensions.Diagnostics.Abstractions              8.0.0   
   > Microsoft.Extensions.FileProviders.Abstractions            8.0.0   
   > Microsoft.Extensions.Hosting.Abstractions                  8.0.0   
   > Microsoft.Extensions.Logging                               8.0.0   
   > Microsoft.Extensions.Logging.Abstractions                  8.0.0   
   > Microsoft.Extensions.Logging.Configuration                 8.0.0   
   > Microsoft.Extensions.Options                               8.0.0   
   > Microsoft.Extensions.Options.ConfigurationExtensions       8.0.0   
   > Microsoft.Extensions.Primitives                            8.0.0   
   > OpenTelemetry                                              1.8.1   
   > OpenTelemetry.Api                                          1.8.1   
   > OpenTelemetry.Api.ProviderBuilderExtensions                1.8.1   
   > OpenTelemetry.Extensions.Hosting                           1.8.1   
   > OpenTelemetry.Instrumentation.AspNetCore                   1.8.1   
   > OpenTelemetry.Instrumentation.Http                         1.8.1   
   > OpenTelemetry.PersistentStorage.Abstractions               1.0.0   
   > OpenTelemetry.PersistentStorage.FileSystem                 1.0.0   
   > System.Diagnostics.DiagnosticSource                        8.0.0   
   > System.Memory.Data                                         1.0.2   
   > System.Numerics.Vectors                                    4.5.0   
   > System.Text.Encodings.Web                                  4.7.2   
   > System.Text.Json                                           4.7.2   
   > System.Threading.Tasks.Extensions                          4.5.4  

Runtime Version

net8.0

Description

I expect when OpenTelemetryLoggerOptions.IncludeScopes = true that scopes would be included on logs. However, the scopes do not appear in either the console exporter, or Azure Monitor.

Steps to Reproduce

1. dotnet new web
2. dotnet add package Azure.Monitor.OpenTelemetry.AspNetCore && dotnet add package OpenTelemetry.Exporter.Console
3. Add the following code to Program.cs and run (add a valid connection string for app insights)

using Azure.Monitor.OpenTelemetry.AspNetCore;
using OpenTelemetry.Logs;

const string APPLICATIONINSIGHTS_CONNECTION_STRING = "...";

var builder = WebApplication.CreateBuilder(args);

builder.Logging.AddOpenTelemetry(options =>
{
    options.IncludeScopes = true;
    options.AddConsoleExporter();
});

builder.Services.AddOpenTelemetry().UseAzureMonitor(configure => configure.ConnectionString = APPLICATIONINSIGHTS_CONNECTION_STRING);

builder.Services.AddSingleton<TestService>();

var app = builder.Build();

app.MapGet("/", async (TestService service) =>
    await service.GetStuff());

app.Run();

class TestService(ILogger<TestService> logger)
{
    private readonly ILogger<TestService> _logger = logger;

    public Task<string> GetStuff()
    {
        List<KeyValuePair<string, object>> context =
        [
            new("test", "abc")
        ];

        using var scope = _logger.BeginScope(context);
        _logger.LogInformation("Getting stuff");
        return Task.FromResult("Hello");
    }
}

Expected Result

I expect to see "test": "abc" as a customDimensions in app insights and to show up in the console output as well.

Actual Result

The scope values do not show up in Application Insights or the console logs.

info: TestService[0]
      Getting stuff
TestService: Information: Getting stuff
LogRecord.Timestamp:               2024-06-07T16:59:44.0769620Z
LogRecord.TraceId:                 c26c2cc5a382307b61d8a5848173fbb0
LogRecord.SpanId:                  6ed463b97ffa35eb
LogRecord.TraceFlags:              Recorded
LogRecord.CategoryName:            TestService
LogRecord.Severity:                Info
LogRecord.SeverityText:            Information
LogRecord.FormattedMessage:        Getting stuff
LogRecord.Body:                    Getting stuff
LogRecord.Attributes (Key:Value):
    OriginalFormat (a.k.a Body): Getting stuff

Resource associated with LogRecord:
telemetry.distro.name: Azure.Monitor.OpenTelemetry.AspNetCore
telemetry.sdk.name: opentelemetry
telemetry.sdk.language: dotnet
telemetry.sdk.version: 1.8.1
service.name: unknown_service:asp-logging

Additional Context

No response

[cijothomas]

Could you remove AzureMonitor part and see if it reproes with OTel SDK and Console Exporter?

[mtaron]

If I remove Azure Monitor, I do see scopes in the console output.

info: TestService[0]
      Getting stuff
TestService: Information: Getting stuff
LogRecord.Timestamp:               2024-06-07T23:26:11.5309790Z
LogRecord.TraceId:                 c4949647becdb3ff363b3bbc18cadae0
LogRecord.SpanId:                  8c438e4b3ffe7f4b
LogRecord.TraceFlags:              None
LogRecord.CategoryName:            TestService
LogRecord.Severity:                Info
LogRecord.SeverityText:            Information
LogRecord.FormattedMessage:        Getting stuff
LogRecord.Body:                    Getting stuff
LogRecord.Attributes (Key:Value):
    OriginalFormat (a.k.a Body): Getting stuff
++LogRecord.ScopeValues (Key:Value):
++[Scope.0]:SpanId: 8c438e4b3ffe7f4b
++[Scope.0]:TraceId: c4949647becdb3ff363b3bbc18cadae0
++[Scope.0]:ParentId: 0000000000000000
++[Scope.1]:ConnectionId: 0HN4794RP0BEC
++[Scope.2]:RequestId: 0HN4794RP0BEC:00000001
++[Scope.2]:RequestPath: /
++[Scope.3]:test: abc

Resource associated with LogRecord:
telemetry.sdk.name: opentelemetry
telemetry.sdk.language: dotnet
telemetry.sdk.version: 1.8.1
service.name: unknown_service:asp-logging

[cijothomas]

@mtaron Thanks for confirming. I'd recommend to report this issue to AzureMonitor repo, as adding AzureMonitor seems to affect other exporters 😞

[mtaron]

I searched GitHub for "azure monitor" and didn't find a repo. It it an internal Microsoft repo? It looks like you work on both OpenTelemetry and Azure Monitor -- could you transfer this issue to the correct repository?

[cijothomas]

Sorry I missed to share the actual repo! You can consult : https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-enable?tabs=aspnetcore#support as well.

https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/monitor - This is the place where AzureMonitor exporter/distro is hosted, and you can report the issue in that repo.

I cannot "transfer" this issue, as Github transfer only works for repos under same organization, and AzureMonitor exporter is not under OpenTelemetry repo! So you have to create a new issue in the above repo.

[mtaron]

Thanks for the links! Filed Azure/azure-sdk-for-net#44480