Testing QUIC and mTLS Before a Load Balancer Cutover

When preparing to move an application onto new infrastructure, it is easy to take a successful connection and a healthy status on the load balancer as evidence that things are working, particularly when the project has a fixed delivery date and there are still several other teams waiting to complete their part of the work. The difficulty is that those checks may only cover a small part of what the application actually needs, leaving problems with authentication or recovery undiscovered until users start connecting through the new environment.

An AWS walkthrough of mutual TLS over QUIC using Network Load Balancer provides a useful example of how this can happen. QUIC is the transport protocol carrying the application’s connections, while mutual TLS, usually shortened to mTLS, allows both the client and server to authenticate using certificates. In the architecture described by AWS, the application handles that certificate verification, while a separate TCP listener responds to the load balancer’s health checks.

This means that the load balancer can show a healthy target even though the connection the client needs is failing, because the health check is testing something different from the application itself. If that distinction is not understood by the people reviewing the results, a project can reach its cutover decision with considerably less evidence than everyone believes they have.

Given that our clients tend to operate in well-regulated finance and insurance environments, this is the type of detail that deserves attention before a change goes into production, where an authentication failure or interrupted application process can quickly become a business issue involving people well beyond the infrastructure team.

Testing the connection that will actually be used

The test plan should reflect how the service will be used, including the client software, the authentication requirements and the application response that confirms the requested work has succeeded. A client should be able to verify the server, present an acceptable certificate and complete a representative exchange, with the results recorded separately from the load balancer’s own health status so that a disagreement between the two remains visible.

The AWS example demonstrates why the intended security configuration needs to be included in those tests. It first checks server-only TLS before enabling mutual TLS, and describes a connection-ID formatting issue that can remain hidden during the simpler connection but appear when the client certificate chain changes how the handshake is packaged. That particular issue belongs to the implementation being described, although the reason for repeating the test with the production security settings is relevant to other delivery projects.

There also needs to be evidence that clients are refused when their credentials are missing or untrusted, since a successful connection with a valid certificate only tells us about the case that was expected to work. The team should be able to show what the client experienced and that application access was denied, without assuming that every implementation will return the same error message.

Allowing for the changes that come after launch

Certificates will expire and need to be renewed, and some environments will also need to change the authority that issues them or the certificates that their systems trust. These are additional checks we would recommend including in an acceptance plan, rather than outcomes established by the AWS walkthrough, because a service that works with today’s certificates still needs a workable process for replacing them.

That process should cover a new connection using an expired certificate, reconnection after renewal and any planned transition between trusted issuers, while treating existing sessions separately from new connections. Where certificate revocation is required, the team also needs to identify which component enforces it and demonstrate that it works, rather than assuming that the use of a private certificate authority takes care of it automatically.

A similar distinction applies to connection continuity. AWS describes routing that allows a client to change its IP address while keeping its connection to the backend, but also explains that replacing a backend pod removes its old server identity and breaks connections using it. In the documented setup, deregistering the target does not drain those QUIC connections, so successful testing of a client network change would leave backend replacement behavior untested.

For a planned replacement, the teams involved should agree how the connection will be closed, how the client will reconnect and how much interruption the service can tolerate, including what happens to application work that was underway at the time. If the client retries a request after reconnecting, the application needs to handle that retry as intended, since restoring the connection alone does not establish whether the original work completed or might be repeated.

Giving the cutover decision enough substance

The results do not need to become an unnecessarily large document, but they should make it possible to understand what was tested, which configuration and software versions were used, what happened and who owns anything that remains unresolved. Client and backend evidence should support the application result without collecting private keys or unnecessary certificate details, and affected tests should be repeated after configuration changes or rollback rehearsals.

At Ahead, our goal is to deliver high-quality work efficiently, which includes making sure that the people approving a change have a clear understanding of what the testing has established and where there are still gaps. Taking the time to connect the infrastructure checks to the application’s actual requirements gives those teams a more reliable basis for proceeding, while reducing the likelihood that an assumption made during delivery becomes a problem the business has to deal with afterward.