Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support using an ALB between VA and RVA #7386

Closed
2 tasks
mcpherrinm opened this issue Mar 15, 2024 · 3 comments
Closed
2 tasks

Support using an ALB between VA and RVA #7386

mcpherrinm opened this issue Mar 15, 2024 · 3 comments

Comments

@mcpherrinm
Copy link
Contributor

We have a bit of fragility between the VA and RVA because of our use of DNS to locate RVA servers. As well, our private PKI needs to be stretched out from our secure DCs to the cloud environment, which causes excess complexity.

We'd like to fix both of these problems at once, by using an AWS Application Load Balancer (ALB) between the VAs and RVAs.
ALBs can load balance grpc, and can validate the incoming mTLS connection. The ALB will have a Web PKI certificate issued by AWS.

We need two changes from Boulder to make this happen:

  • In the RVA, support exposing the RPCs without requiring client certificate authentication
  • In the VA, allow specifying a different CA trust store for outbound connections (or potentially to use the system trust store)

The first is more important than the 2nd, as we can manually install a private PKI certificate in the ALB, but we'd rather not have to write automation for that.

Having a general option to run the RVAs unauthenticated is somewhat high-risk for other services, so we may want that to block on #5294 so that the setting can be self-contained only in the RVAs.

@aarongable
Copy link
Contributor

We previously removed our use of AWS Network Load Balancer (NLB) between our RVAs and Unbounds. Have we ever used NLB between the VAs and the RVAs? If we did, would that let us dynamically route traffic to all of our RVAs without having to let the load-balancer terminate TLS?

@mcpherrinm
Copy link
Contributor Author

mcpherrinm commented Mar 19, 2024

Using an NLB as a layer-4 proxy is an option, but it won't work as well as an ALB:

  • we can't use it for autoscaling based on request volume, which would be a big improvement for our autoscaling policies
  • it load balances connections rather than RPCs, so we can't gracefully remove instances without terminating in-flight RPCs
  • we can't use the ALB's grpc health check features to ensure the instance is properly up and running

@aarongable aarongable added this to the Sprint 2024-04-09 milestone Apr 9, 2024
@aarongable aarongable removed this from the Sprint 2024-04-09 milestone Apr 9, 2024
pgporada added a commit that referenced this issue May 13, 2024
* Adds a `VerifyGRPCClientCertIfGiven` boolean to the `remoteva` config
that cause the RVA server to use the less strict
`tls.VerifyClientCertIfGiven` for use with an Amazon Web Services
Application Load Balancer (ALB) between the `boulder-va` and `remoteva`.
See #7386.

Part of #5294

---------

Co-authored-by: Samantha <[email protected]>
@aarongable
Copy link
Contributor

  • In the RVA, support exposing the RPCs without requiring client certificate authentication
  • In the VA, allow specifying a different CA trust store for outbound connections (or potentially to use the system trust store)

Both done in #7473; the latter by virtue of the fact that the caCert pem file loaded by the VA's TLSConfig can contain multiple roots.

vbaranovskiy-plesk pushed a commit to plesk/boulder that referenced this issue May 30, 2024
…ncrypt#7473)

* Adds a `VerifyGRPCClientCertIfGiven` boolean to the `remoteva` config
that cause the RVA server to use the less strict
`tls.VerifyClientCertIfGiven` for use with an Amazon Web Services
Application Load Balancer (ALB) between the `boulder-va` and `remoteva`.
See letsencrypt#7386.

Part of letsencrypt#5294

---------

Co-authored-by: Samantha <[email protected]>
AlinaADmi pushed a commit to plesk/boulder that referenced this issue Jul 29, 2024
…ncrypt#7473)

* Adds a `VerifyGRPCClientCertIfGiven` boolean to the `remoteva` config
that cause the RVA server to use the less strict
`tls.VerifyClientCertIfGiven` for use with an Amazon Web Services
Application Load Balancer (ALB) between the `boulder-va` and `remoteva`.
See letsencrypt#7386.

Part of letsencrypt#5294

---------

Co-authored-by: Samantha <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants