In this blog entry, I'm going to cover using System.Net.Http.HttpClient so that sockets are effectively consumed. There are several blogs and articles that cover this already but I'm going to consolidate the various options here and throw my own in as well.

The Problem

It is pretty well documented that while HttpClient implements IDisposable, it is intended to be reused which means not wrapping it in a using statement.

As noted in this ASP.Net article, "HttpClient is intended to be instantiated once and re-used throughout the life of an application. Especially in server applications, creating a new HttpClient instance for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors.".

It is also noted in the HttpClient's documentation page that "HttpClient is intended to be instantiated once and re-used throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors.".

The Microsoft Patterns and Practices team even has a Performance Optimization repo that mentions this issue as well.

The problem is also mentioned in the following blog posts:

Long story short, this issue has been documented quite a bit and there are well-known solutions as well.

Managing sockets isn't limited to just HttpClient, if you're using Azure, you have the same sort of issues with TopicClient, QueueClient and MessageSender among others. Let's stay on topic and stick to HttpClient though.

Solution Basics

Part 1 - Cache HttpClient

This is by far the easiest solution and all it really means is making the HttpClient a static field. Keep in mind that not all members are thread safe but those that represent HTTP methods are (i.e. GET, POST, PUT, DELETE). The sample below is taken from the HttpClient's documentation page.

public class GoodController : ApiController {  
    // OK
    private static readonly HttpClient HttpClient;

    static GoodController() {
        HttpClient = new HttpClient();
    }
}

There are a couple of downsides to this solution.

  1. As mentioned earlier, not all members are thread-safe. One example is the DefaultRequestHeaders. If you want to take advantage of these, one option is to create one shared intance of the HttpClient per endpoint you will be connecting to. This can get messy as the number of these endpoints increases.
  2. As mentioned in Blog #1, if a DNS entry changes, the client won't see these changes until the next time a TCP socket is opened. The default connection idle timeout is 100 seconds. In a busy website, you may not reach this timeout for quite some time. Which leads us to the next part of the solution.

Part 2 - Set ServicePoint.ConnectionLeaseTimeout

Set ServicePoint.ConnectionLeaseTimeout for each URL you will be connecting to at app startup. Only the host, schema and port are taken into account when looking up the Service Point - everything else is ignored. Meaning that if the Uri you use to find the Service Point includes segments and query parameters, they are ignored and you will get the same Service Point for "http://localhost/my/segmented/url?with=params" as you would for "http://localhost". This is taken from Singleton HttpClient? Beware of this serious behaviour and how to fix it.

var sp = ServicePointManager.FindServicePoint(new Uri("http://foo.bar/baz/123?a=ab"));  
sp.ConnectionLeaseTimeout = 60*1000; // 1 minute  

While this addresses DNS issue, it does not address the messiness that comes with having to cache and now configure the ServicePoint for all of the endpoints that an application may use. So let's look at some implementation options.

Implementing Solution

Using Static Field

If you are using what I consider the typical implementation of the Repository Design pattern (e.g. Martin Folwer Microsoft Repository Pattern Article), you typically end up with a Repository class per Root Aggregate. In this case, using a static field is usually good enough.

public sealed class CustomerRepository : IRepository<Customer> {  
  private static readonly HttpClient SharedHttpClient;

  static CustomerRepository() {
    var uri = new Uri("http://foo.bar");
    SharedHttpClient = new HttpClient() { 
      BaseAddress = uri,
      /* Other setup */
    };
    var sp = ServicePointManager.FindServicePoint(uri);
    sp.ConnectionLeaseTimeout = 60*1000; // 1 minute
  }

  public Customer GetById(int id) {
    var response = await SharedHttpClient.GetAsync(...);
    ...
  }

As mentioned earlier, only the host, schema and port are taken into account when looking up the Service Point (i.e. calling ServicePointManager.FindServicePoint). What this means is that if the same host contains several services and we have a repository for each service, you will be maintaining several instances of HttpClient, all pointing to the same host each managing their own sockets. This is probably good enough in a lot of situations.

If you are using something like Command Handlers [1] [2] and Query Handlers [1] [2], the number of HttpClient could grow substantially since you would have one QueryHandler per query not per repository. For example:

public sealed class GetCustomerByIdQueryHandler : IQueryHandler<GetCustomerByIdQuery> {  
  private static readonly HttpClient SharedHttpClient;

  static GetCustomerByIdQueryHandler() {
    var uri = new Uri("http://foo.bar");
    SharedHttpClient = new HttpClient() { 
      BaseAddress = uri,
      /* Other setup */
    };
    var sp = ServicePointManager.FindServicePoint(uri);
    sp.ConnectionLeaseTimeout = 60*1000; // 1 minute
  }

  public Customer Execute(GetCustomerByIdQuery query) {
    var response = await SharedHttpClient.GetAsync(...);
    ...
  }

The implementation isn't much different. Since the number of classes increase from one per repository to one per query, the number of HttpClient instances increases along with it. Just as before, this may also be okay but let's look at some other options.

Sharing via Inheritance

This is one of the easiest way to share the HttpClient Instance. The idea here is to simply to create a base for each web service host that you use. In the example, since http://foo.bar is the host for our service, we create a FooBarServiceRepository. If we were using another service host, we would create another subclass.

using System.Net.Http;

public abstract class FooBarServiceRepository {  
  protected static readonly HttpClient SharedHttpClient;

  static FooBarServiceRepository() {
    var uri = new Uri("http://foo.bar");
    SharedHttpClient = new HttpClient() { 
      BaseAddress = uri,
      /* Other setup */
    };

    var sp = ServicePointManager.FindServicePoint(uri);
    sp.ConnectionLeaseTimeout = 60*1000; // 1 minute
  }
}
public sealed class CustomerRepository : FooBarServiceRepository, IRepository<Customer> {  
  public Customer GetById(int id) {
    var response = await SharedHttpClient.GetAsync(...);
    ...
  }
}
public sealed class AddressRepository : FooBarServiceRepository, IRepository<Address> {  
  public Address GetById(int id) {
    var response = await SharedHttpClient.GetAsync(...);
    ...
  }
}

One thing to note is that the base class is not an open generic type since creating a static member in an open generic type will actually result in multiple instances (one for every closed generic type) as mentioned in this ReSharper article.

Using a Factory Class

Using a factory class is slightly more complicated but allows you to get away from using inheritance strictly to achieve reusing the HttpClient instance.

The following sample uses a Factory class that keeps a cached list of HttpClient instances for specific hosts. I took that approach since that's the ServicePointManager's level of granularity. While ConcurrentDictionary's GetOrAdd(TKey, Func<TKey, TValue>) member is not atomic, in our case, this is alright since the act of instantiating an HttpClient does not open up a socket.

using System.Net.Http;  
using System.Collections.Concurrent;

public interface IHttpClientFactory {  
  HttpClient GetForHost(Uri uri);
}

public sealed class HttpClientFactory : IHttpClientFactory {  
  protected static readonly ConcurrentDictionary<string, HttpClient> HttpClientCache = new ConcurrentDictionary<string, HttpClient>();

  public HttpClient GetForHost(Uri uri) {
    var key = $"{uri.Scheme}://{uri.DnsSafeHost}:{uri.Port}";

    return HttpClientCache.GetOrAdd(key, k => {
      var client = new HttpClient() { 
        /* Other setup */
      };

      var sp = ServicePointManager.FindServicePoint(uri);
      sp.ConnectionLeaseTimeout = 60*1000; // 1 minute

      return client;
    });
  }
}

Once you've got the HttpClientFactory defined, up next is registering the factory with your IoC container of choice. In my case that is Autofac.

// During app startup...
var builder = new ContainerBuilder();

builder.Register<HttpClientFactory>()  
       .As<IHttpClientFactory>()
       .SingleInstance();

Then declare your dependency using constructor injection and use the HttpClientFactory.

public sealed class CustomerRepository : IRepository<Customer> {  
  private readonly IHttpClientFactory _httpClientFactory;

  public CustomerRepository(IHttpClientFactory httpClientFactory) {
    _httpClientFactory = httpClientFactory;
  }

  public Customer GetById(int id) {
    var httpClient = _httpClientFactory.GetForHost(new Uri("http://foo.bar/customers"));
    var response = await httpClient.GetAsync(...);
    ...
  }
}

Now all that caching is done outside of your repository/query classes and only one factory class is needed. This avoids having to create any wrappers or new abstractions around the HttpClient.

Using AutoRest

If you have not heard of AutoRest yet, it is an open source Open API (i.e. Swagger) client code generator. It supports C#, Java, Node.js and TypeScript among others. Here's an Azure Friday video that gives you an overview. We are going to take the generated code and see how to cache and configure the ServicePointManager.ConnectionLeaseTimeout.

In a way, this is by far the easiest option. Because the lifetime of the HttpClient is tied to the lifetime of the Microsoft.Rest.ServiceClient<T> subclass generated by AutoRest (MyClient in the example below), all you have to do is share that instance. AutoRest generates both the class and the interface for this subclass so not only is it easy to register with an IoC container, you can also easy inject a mock instance when unit testing. In this example, I'm going to use the sample generated client code that the AutoRest GitHub project README.md file walks you through creating.

Here's a condensed version of the generated interface and client code.

public partial interface IMyClient : System.IDisposable {  
  Task<HttpOperationResponse<string>> GetGreetingWithHttpMessagesAsync(...);
}

public partial class MyClient : Microsoft.Rest.ServiceClient<MyClient>, IMyClient {  
  public async Task<HttpOperationResponse<string>> GetGreetingWithHttpMessagesAsync(...) { ... }
  /* Rest of Auto generated code ommitted */
}

Once you have the code generated, all you have to do is register MyClient with the IoC container. That will take care of caching the HttpClient since ServiceClient creates that for you during its initialization phase and caches that HttpClient instance. Now all you have left is configuring the ConnectionLeaseTimeout. An easy way to do this is to do that as part of app startup.

// During app startup...
var builder = new ContainerBuilder();

// While MyClient handles caching HttpClient, we still
// need to configure the ConnectionLeaseTimeout
var sp = ServicePointManager.FindServicePoint(new Uri("http://foo.bar"));  
sp.ConnectionLeaseTimeout = 60*1000; // 1 minute

builder.Register<MyClient>()  
       .As<IMyClient>()
       .SingleInstance();

If you wanted to get bonus points, because MyClient is a partial class, you can put the ConnectionLeaseTimeout configuration in a separate partial class file to encapsulate all the logic in one place. There's a handy partial method CustomInitialize that the generated code has that is ideal for this purpose.

//In MyClient.partial.cs File that you create
public partial class MyClient {  
  partial void CustomInitialize() {
    var sp = ServicePointManager.FindServicePoint(new Uri("http://foo.bar"));
    sp.ConnectionLeaseTimeout = 60*1000; // 1 minute
  }
}

// And Now Registration Is Even Easier During app startup...
var builder = new ContainerBuilder();

builder.Register<MyClient>()  
       .As<IMyClient>()
       .SingleInstance();

// If you're using Autofac's ability to scan assemblies and register types
// as in the example below you may not even need the code above! 
builder.RegisterAssemblyTypes(myAssembly)  
       .AsImplementedInterfaces();

Using that separate partial class file prevents your CustomInitialize implementation from being overwritten if you regenerate the client code.

Now not only are you minimizing the sockets opened and setting a max connection lifetime so that DNS changes are honored in a timely manner but you're also getting a strongly typed set of classes that let you call your API.

The issue of effectively using components that manage socket connections is not isolated to HttpClient. The Microsoft Patterns and Practices team even has a Performance Optimization repo that mentions this is an issue for connecting to the Service Bus and DocumentDB as well. I highly recommend reading that article as it provides links to some great resources.