使用 Application Insights 采样时，是否可以保留相关条目以及排除的项目

Question

我一直在尝试遥测采样，以避免达到 App Insights 每日日志上限。理想情况下，我想对所有内容应用采样，但排除异常并保留异常的相关跟踪（相同的操作 ID）。

我已经创建了一个示例控制台应用程序来测试事物，到目前为止我可以成功地采样并保留异常。但相关痕迹也会被采样。

我考虑过实现自定义

ITelemetryProcessor

，但它一次处理一个条目。所以我不确定是否可以使用定制处理器。也许有一些东西可以帮助实现所需的行为。

Program.cs

代码如下

using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

const string appInsightConnString = "<connection string>";
const double samplingPercentage = 50;

var services = new ServiceCollection();

// add context
var context = new Context();
services.AddSingleton(context);

// configure application insights
services.AddApplicationInsightsTelemetryWorkerService(
    (options) =>
    {
        // disable adaptive sampling
        options.EnableAdaptiveSampling = false;
        options.ConnectionString = appInsightConnString;
    });

// configure logging
services.AddLogging(loggingBuilder =>
{
    loggingBuilder.ClearProviders();
    loggingBuilder.Services.AddSingleton<ILoggerProvider, ContextApplicationInsightsLoggerProvider>();
    loggingBuilder.AddConsole();
});

var serviceProvider = services.BuildServiceProvider();

// setup sampling
var telemetryConfiguration = serviceProvider.GetRequiredService<TelemetryConfiguration>();
var telemetryBuilder = telemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
telemetryBuilder.UseSampling(samplingPercentage, excludedTypes: "Exception");
telemetryBuilder.Build();

// get logger
var logger = serviceProvider.GetRequiredService<ILogger<Program>>();

// do something important
DoWork(context, logger);

// explicitly call Flush() followed by sleep is required in console apps.
// this is to ensure that even if application terminates, telemetry is sent to the back-end.
var telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();
telemetryClient.Flush();
Task.Delay(10000).Wait();

Console.WriteLine("Flushed. Press any key to exit");
Console.ReadKey();


static void DoWork(Context context, ILogger logger)
{
    const int iterations = 50;
    const int errors = 15;

    // session Id to filter logs
    var sessionId = Guid.NewGuid().ToString();
    Console.WriteLine($"Session Id: {sessionId}");

    // randomize errors
    var random = new Random();
    var errorsHash = new HashSet<int>();
    while (errorsHash.Count < errors)
    {
        errorsHash.Add(random.Next(0, iterations));
    }

    // log
    for (var i = 0; i < iterations; i++)
    {
        context.CorrelationId = Guid.NewGuid().ToString();
        logger.LogInformation($"Begin operation: {context.CorrelationId}. Session Id: {sessionId}");
        if (errorsHash.Contains(i))
            logger.LogError(new Exception("test ex"), $"Error operation: {context.CorrelationId}. Session Id: {sessionId}");
        logger.LogInformation($"End operation: {context.CorrelationId}. Session Id: {sessionId}");
    }
}

Answer 1

下面的代码使用 Microsoft Application Insights 在控制台应用程序中进行监视和日志记录，并利用 Application Insights 中的

ITelemetryProcessor

界面。

RelatedTelemetryProcessor

类实现

ITelemetryProcessor

并在使用 Application Insights 时保留相关条目以及排除的项目

我参考此 doc 在 Application Insights SDK Azure Monitor 中进行过滤和预处理。

• 使用自定义遥测处理器（

  RelatedTelemetryProcessor

  ），它根据操作 ID 保留与异常相关的所有遥测项目。在使用默认遥测处理器链排除例外情况 (

  excludedTypes: "Exception"

  ) 的同时，还采用了采样。

   private const string AppInsightConnectionString = "<connection string>";
    private const double SamplingPercentage = 50;

    public static async Task Main(string[] args)
    {
        // Create a ServiceCollection
        var services = new ServiceCollection();

        // Configure Application Insights telemetry
        services.AddApplicationInsightsTelemetryWorkerService(options =>
        {
            options.EnableAdaptiveSampling = false;
            options.ConnectionString = AppInsightConnectionString;
        });

        // Configure logging
        services.AddLogging(loggingBuilder =>
        {
            loggingBuilder.AddConsole();
            loggingBuilder.AddApplicationInsights(AppInsightConnectionString);
        });

        // Build the service provider
        var serviceProvider = services.BuildServiceProvider();

        // Configure telemetry pipeline with custom processor and sampling
        var telemetryConfiguration = serviceProvider.GetRequiredService<TelemetryConfiguration>();
        var telemetryProcessorChainBuilder = telemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;

        // Use custom processor and sampling
        telemetryProcessorChainBuilder.Use((next) => new CustomTelemetryProcessor(next));
        telemetryProcessorChainBuilder.UseSampling(SamplingPercentage, excludedTypes: "Exception");
        telemetryProcessorChainBuilder.Build();

        // Get logger
        var logger = serviceProvider.GetRequiredService<ILogger<Program>>();

        // Perform work
        DoWork(logger);

        // Flush telemetry
        var telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();
        telemetryClient.Flush();
        await Task.Delay(5000); // Wait for telemetry to be sent

        Console.WriteLine("Done. Press any key to exit.");
        Console.ReadKey();
    }

    static void DoWork(ILogger logger)
    {
        const int iterations = 50;
        const int errors = 10;

        var random = new Random();
        var errorsSet = new HashSet<int>();
        while (errorsSet.Count < errors)
        {
            errorsSet.Add(random.Next(0, iterations));
        }

        // Perform operations with logging
        for (int i = 0; i < iterations; i++)
        {
            string operationId = Guid.NewGuid().ToString();
            logger.LogInformation($"Begin operation: {operationId}");

            if (errorsSet.Contains(i))
            {
                logger.LogError(new Exception("Sample exception"), $"Error in operation: {operationId}");
            }

            logger.LogInformation($"End operation: {operationId}");
        }
    }
}

public class CustomTelemetryProcessor : ITelemetryProcessor
{
    private readonly ITelemetryProcessor _next;
    private readonly HashSet<string> _preservedOperationIds = new HashSet<string>();

    public CustomTelemetryProcessor(ITelemetryProcessor next)
    {
        _next = next;
    }

    public void Process(ITelemetry item)
    {
        // Get the operation ID of the telemetry item
        string operationId = item.Context.Operation.Id;

        // Check if the telemetry item is an exception
        if (item is ExceptionTelemetry exceptionTelemetry)
        {
            // Add the operation ID to the set of preserved operation IDs
            _preservedOperationIds.Add(operationId);
        }

        // Check if the operation ID is in the set of preserved operation IDs
        if (_preservedOperationIds.Contains(operationId))
        {
            // Pass the item through without sampling
            _next.Process(item);
        }
        else
        {
            // Apply your desired sampling here
            // If you decide to keep the item, pass it to the next processor
            _next.Process(item);
        }
    }

为了确保在对其他遥测项目应用采样时出现异常时保留相关跟踪，您可以调整自定义遥测处理器以正确处理这两种情况。

public class CustomTelemetryProcessor : ITelemetryProcessor
{
    private ITelemetryProcessor _next;
    private HashSet<string> _preservedOperationIds = new HashSet<string>();

    public CustomTelemetryProcessor(ITelemetryProcessor next)
    {
        _next = next;
    }

    public void Process(ITelemetry item)
    {
        if (item is ExceptionTelemetry exceptionTelemetry)
        {
            // Add the operation ID to the set of preserved IDs
            _preservedOperationIds.Add(exceptionTelemetry.Context.Operation.Id);
        }

        // Check if the operation ID is in the preserved set
        if (_preservedOperationIds.Contains(item.Context.Operation.Id))
        {
            // Pass the item through without sampling
            _next.Process(item);
        }
        else
        {
            // Apply sampling or other processing as usual
            _next.Process(item);
        }
    }
}

在您的

Main

方法中：

// Configure telemetry pipeline with custom processor and sampling
var telemetryConfiguration = serviceProvider.GetRequiredService<TelemetryConfiguration>();
var telemetryProcessorChainBuilder = telemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;

// Use custom processor and sampling
telemetryProcessorChainBuilder.Use((next) => new CustomTelemetryProcessor(next));
telemetryProcessorChainBuilder.UseSampling(SamplingPercentage, excludedTypes: "Exception");
telemetryProcessorChainBuilder.Build();

自定义

ITelemetryProcessor

一次处理一个遥测项目，这使得确保在发生异常时保留相关跟踪变得具有挑战性。