Sanitize HTTP Client Logging

🌟 Intro

This blog post focusses on the security aspect from logging and describes my new NuGet package designed to safeguard sensitive information such as API keys, tokens or PII (Personally Identifiable Information) that might appear in the RequestUri’s query values.
It achieves this through configurable regular expression matching, offering a solution to replace these sensitive data elements.

🔒 Security

🔑 API Keys

An API key is be used as a secret token for authenticating and authorizing access to an (external) API. Logging this API key introduces security risks. Here are some reasons why it is not desired to include an API key with the value in logging:

• Exposure to Unauthorized Access: Logs are often used for diagnostic or issue track purposes, and they can be accessed by multiple individuals or systems. If an API key is included in the logging, there is a risk that unauthorized persons or systems could get access to the value. Once this API key is exposed, malicious users can access the API, leading to potential data breaches or service misuse.
• Risk of Accidental Leakage: Logs may be stored in multiple locations or transferred over networks. These logs may be included in error reports, debugging outputs, or used by logging tools. The more places a log is stored or accessed, the higher the chance that the API key could be accidentally leaked. Such leaks could occur through mishandling of logs, insecure storage practices, or inadvertent exposure in publicly accessible places (like GitHub repositories or shared logs).
• Compliance and Audit Failures: Many regulatory frameworks and security standards require that sensitive information such as credentials, which include API keys, must be handled and stored securely. Logging API keys can lead to non-compliance with these standards and result in failed security audits. Non-compliance could involve legal and financial penalties or reputational damage.

📒 PII (Personally Identifiable Information)

In addition to the reasons described for exposing API Keys, additional security and compliance issues arise when unintentionally exposing Personally Identifiable Information (PII) via logging:

• Identity Theft: If logs containing PII are accessed by malicious actors, it can lead to identity theft, financial fraud, and other security issues related to personal data.
• Compliance Violations: Many regulations and laws, such as the General Data Protection Regulation (GDPR) in Europe, the Health Insurance Portability and Accountability Act (HIPAA) in the United States, define clear and strict guidelines on how PII should be handled and protected. Storing PII in logs can lead to non-compliance with these regulations, resulting in fines and legal penalties.
• Loss of Trust: If a company is found to be negligent in protecting customer data by allowing PII to be exposed in logs, it can severely damage its relationship with customers and partners. And reputational damage for the organization and loss of trust can lead to decreased business, as customers might choose to move to competitors who they perceive as taking better care of their personal data.

🏆 Challenge

The default Informational logging which is used by the HttpClient generates 4 logging text lines like:

→ Start processing HTTP request GET https://my.api.com/v1/status?apikey=my-secret-key
→ Sending HTTP request GET https://my.api.com/v1/status?apikey=my-secret-key
→ Received HTTP response headers after 188.6041ms - 200
→ End processing HTTP request after 188.8026ms - 200

And as you can see, the secret apikey in the RequestUri with the value is also logged, which is not desired.

🪄 Possible solutions

There are several ways to approach this problem, the best solution would be not to provide sensitive information in the RequestUri, but when possible, via the Http Header or via the Body.
In case this is not possible for that specific API, the next sub-chapters describe other solutions on how to fix this.

1️⃣ Change the default log-level from the HttpClient

Changing the log-level from the HttpClient to Warning or higher makes sure that no Informational logging is written which could include sensitive data.

Using configuration (via appsettings.json)

{
  "Logging": {
    "Debug": {
      "LogLevel": {
        "Default": "Information",
        "System.Net.Http.HttpClient": "Warning"
      }
    }
  }
}

Using C# code

Configure the logging levels programmatically in the ConfigureServices method (or an equivalent setup method in your application) is also an option. You can specifically target System.Net.Http.HttpClient to set its log level to Warning, like this:

public void ConfigureServices(IServiceCollection services)
{
    services.AddLogging(builder =>
    {
        builder.AddFilter("System.Net.Http.HttpClient", LogLevel.Warning);
        // Other logging configuration can go here
    });

    // Other service configurations can go here
}

This means however that in both options all Informational logging from the HttpClient is suppressed.

2️⃣ Implementing the new IHttpClientLogger interface

This new interface is available in .NET 8 and is an abstraction for custom HTTP request logging for a named HttpClient instances returned by IHttpClientFactory.

It contains the following methods:
– LogRequestStart(HttpRequestMessage) : Logs before sending an HTTP request.
– LogRequestStop(Object, HttpRequestMessage, HttpResponseMessage, TimeSpan) : Logs after receiving an HTTP response.
– LogRequestFailed(Object, HttpRequestMessage, HttpResponseMessage, Exception, TimeSpan) : Logs the exception that occurred while sending an HTTP request.

A simple implementation for this custom HttpClientLogger could be:

public class CustomHttpClientLogger : IHttpClientLogger
{
    private readonly ILogger<CustomHttpClientLogger> _logger;
 
    public CustomHttpClientLogger(ILogger<CustomHttpClientLogger> logger)
    {
        _logger = logger;
    }
 
    public object? LogRequestStart(HttpRequestMessage request)
    {
        // Call _logger.LogInformation to log the request start
        // Make sure to sanitize the RequestUri
        return null;
    }
 
    public void LogRequestStop(object? context, HttpRequestMessage request, HttpResponseMessage response, TimeSpan elapsed)
    {
        // Call _logger.LogInformation to log the request stop
        // Make sure to sanitize the RequestUri
    }
 
    public void LogRequestFailed(object? context, HttpRequestMessage request, HttpResponseMessage? response, Exception exception, TimeSpan elapsed)
    {
        // Call _logger.Error to log the error
        // Make sure to sanitize the RequestUri
    }
}

In addition to this custom class, this new HttpClientLogger should be correctly registered. This can done in three steps:

1. Remove all existing loggers from the HttpClient
   Make sure that for your application, all loggers can be removed. If this is not the case, only specific loggers should be removed.
2. Add our custom logger to the HttpClient
3. Register our custom logger in DI

The code looks like this:

services
    .AddHttpClient("my-http-client", client =>
    {
        client.BaseAddress = new Uri("https://my.api.com/v1");
    })
    .RemoveAllLoggers() // 1.
    .AddLogger<CustomHttpClientLogger>(wrapHandlersPipeline: true); // 2

services
    .TryAddScoped<CustomHttpClientLogger>(); // 3.

📝Note that for step 2, the wrapHandlersPipeline argument is set to true, this makes sure that the custom logging handler will be added to the top or to the bottom of the additional handlers chains.

In addition to the CustomHttpClientLogger class, some logic needs to be added which can replace sensitive data in the RequestUri using regular expression matching, this will be explained in chapter 4.

🔶This solution works fine for .NET 8, however in case you are still using an older .NET version, this solution cannot be used because the IHttpClientLogger interface is only defined in .NET 8.
To solve this issue, I’ll explain a generic solution in the next chapter which can be used for older .NET Frameworks.

3️⃣ Extending the DelegatingHandler

It’s not possible to configure the log pattern of the Microsoft.Extensions.Http based HttpClient loggers. The solution is is to create a custom class which extends the abstract DelegatingHandler and use this logging class instead of the default loggers. This DelegatingHandler is present in older .NET Full Frameworks, .NET Standard 2.x and .NET 6, which makes it broader usable.

The DelegatingHandler contains two methods which can be overridden:

1. HttpResponseMessage Send(HttpRequestMessage request, CancellationToken cancellationToken)
2. Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)

In the next example code, I’ll show the implementation for SendAsync (the Send implementation is almost the same):

protected override async Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    Guard.NotNull(request);

    var sanitizedRequestUri = SanitizeRequestUri(request);
    var stopwatch = ValueStopwatch.StartNew();

    try
    {
        var response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
        return LogInfo(request, sanitizedRequestUri, response, stopwatch);
    }
    catch (Exception exception)
    {
        LogWarning(exception, request, sanitizedRequestUri, stopwatch);
        throw;
    }
}

An interesting piece of code is is the var stopwatch = ValueStopwatch.StartNew();.

This ValueStopwatch is a new class which is added to .NET 7 which is essentially a bare minimum version from StopWatch class (without any stop or restart, just measuring the elapsed time) and without any memory allocation, so no pressure on the Garbage Collector.
More information can be found here and here.

4️⃣ SanitizeRequestUri

Sanitizing the RequestUri is done by using this following simple class:

internal class RequestUriReplacer : IRequestUriReplacer
{
  private readonly ConcurrentDictionary<Lazy<Regex>, string> _requestUriReplacements = new();
 
  public RequestUriReplacer(IOptions<SanitizedHttpLoggerOptions> options)
  {
    var requestUriReplacements = Guard.NotNull(options.Value).RequestUriReplacements;
    foreach (var replacement in requestUriReplacements)
    {
      _requestUriReplacements
        .TryAdd(new Lazy<Regex>(() => new Regex(replacement.Key, RegexOptions.Compiled, TimeSpan.FromMilliseconds(100))), replacement.Value);
    }
  }
 
  public string? Replace(string? uri)
  {
    return uri == null ?
      null :
      _requestUriReplacements.Aggregate(uri, (current, item) => item.Key.Value.Replace(current, item.Value));
  }
}

The SanitizedHttpLoggerOptions contains a Dictionary<string,string> which contains the regular expression and the replacement value. In this case this regular expression is matched in the RequestUri.

And example could be:

• Regex: "(?i)apikey=[^&]*"
  This regex pattern will match any part of a string that starts with “apikey=” (in a case-insensitive manner) followed by any number of characters that are not an ampersand.
• Value: "apikey=xxx"
  In case the regex pattern matches, the value is replaced by “apikey=xxx”.

💻 Example

The above described functionality is built into two NuGet packages: (SanitizedHttpLogger and SanitizedHttpClientLogger) which can easily be integrated and used.

I did create a project which accesses the Freecurrency API using a RestEase defined API to retrieve currency conversion rates via the free service: freecurrencyapi.com.
This API requires an api-key as a query parameter in the RequestUri to successfully authenticate to the endpoint. When the Informational logging is enabled for a project which uses this API, the secret api-key is logged. To avoid this, this new package can be used. See the next chapter how to configure this.

Usage

.NET 8

When using the NuGet in a .NET 8 project, you need to use the following code in your DI-setup:

string RegexMatch = "(?i)apikey=[^&]*";
string RegexReplacement = "apikey=***";

services
    .AddHttpClient(options.HttpClientName!, httpClient =>
    {
        httpClient.BaseAddress = options.BaseAddress;
        httpClient.Timeout = TimeSpan.FromSeconds(options.TimeoutInSeconds);
    })
    .UseWithRestEaseClient(new UseWithRestEaseClientOptions<IFreecurrencyApiInternal>
    {
        InstanceConfigurer = freecurrencyAPI =>
        {
            freecurrencyAPI.ApiKey = options.ApiKey;
        }
    })

    // ⬇️ Add this
    .ConfigureSanitizedLogging(o => o.RequestUriReplacements.Add(RegexMatch, RegexReplacement));

.NET 7 and lower

When using the NuGet in a .NET 7 or lower project, you need to use the following code in your DI-setup:

string RegexMatch = "(?i)apikey=[^&]*";
string RegexReplacement = "apikey=***";

services
    .AddHttpClient(options.HttpClientName!, httpClient =>
    {
        httpClient.BaseAddress = options.BaseAddress;
        httpClient.Timeout = TimeSpan.FromSeconds(options.TimeoutInSeconds);
    })
    .UseWithRestEaseClient(new UseWithRestEaseClientOptions<IFreecurrencyApiInternal>
    {
        InstanceConfigurer = freecurrencyAPI =>
        {
            freecurrencyAPI.ApiKey = options.ApiKey;
        }
    });

    // ⬇️ Add this
services
    .UseSanitizedHttpLogger(o => o.RequestUriReplacements.Add(RegexMatch, RegexReplacement));

Result

When the SanitizedHttpLogger functionality is used, the Informational logging does not show the secret value from the apikey anymore when calling the freecurrencyapi (the secret value is replaced by ***):

[INF] Sending HTTP request GET https://api.freecurrencyapi.com/v1/status?apikey=***
[INF] Received HTTP response GET https://api.freecurrencyapi.com/v1/status?apikey=*** - 200 in 294,8ms

❔ Feedback or Questions

Do you have any feedback or questions regarding this or one of my other blogs? Feel free to contact me: LinkedIn or Twitter.

📚 Resources

📦 GitHub Projects

• My GitHub project: SanitizedHttpLogger
• My GitHub project: FreecurrencyAPI which uses the SanitizedHttpLogger

✍️ BlogPosts

• BlogPost by John Kors about MinimalHttpLogger
• Blogpost by Josef Ottosson about HttpClient logging

🔗 Security Links

• Analysis of storm-0558 techniques for unauthorized email access
• Information exposure through query strings in url

📝 Notes

Some content in this blog is created with the help of an AI. I did review and revise the content were needed.