Introduction
In today’s security-conscious world, many companies avoid Public Endpoints as a necessity. Private subnets and internal-facing capabilities are the de-facto standard when building an application that doesn’t require access from the Internet.
In a recent project, I was asked to extend the capabilities of an SNS module to allow HTTPS subscriptions. After much trial and error, it became apparent that it was not possible to subscribe SNS to a private HTTPS endpoint. This fact is not readily discernable when browsing service Amazon FAQs; however, others posting in online communities were facing the same issues, and I was able to confirm that it was a limitation from a helpful AWS product team.
Why use a Private HTTPS Endpoint?
Well, the alternative is to use a public Endpoint, which means exposing your Application to the internet. Whilst you can guard this endpoint (for example using IP restrictions, WAFs, etc.), this:
- Increases the complexity of your application, and requires the ongoing maintenance of the security rules;
- Increases your attack surface area;
- Allows the misconfiguration of a Public Endpoint to unwittingly expose your application to attack by bad actors (not the “Battlefield Earth” type, the hacky type);
- Can fall foul of corporate standards requiring all intra-component communication to be private – and earn you glowering looks and/or eye-rolls from your security and compliance teams.
Bridging the Gap
So, you have decided (or someone has decided for you!) that you need to connect SNS to a private HTTPS endpoint.
An alternative approach is to use another service which is acceptable to SNS as a proxy – in this case, Lambda:
As you can see, access to a Public HTTPS endpoint works as expected, however, access to a Private Endpoint is restricted.
The Proxy approach works as follows:
- Instead of registering HTTP as an endpoint, you register the Lambda (let’s call this the “Lambda Proxy”). One Lambda is created per Private Endpoint and the Lambda will contain code that points directly to that Endpoint. This avoids a scenario where a user may subvert the Lambda to call unintended services and allows very tight coupling between the Lambda Proxy and the Endpoint Security Group (for example);
- The Lambda Proxy receives a Payload with a Message (and pre-requisite headers, and potentially additional paths) to be sent to the HTTPS endpoint;
- The Lambda Sends the Payload to the Endpoint and receives a Response
- The Lambda relays the response back to SNS.
There are some things to consider when using an approach such as this :
- Depending on the frequency of SNS Messages (i.e. in a high throughput environment with a “peaky workload”), the latency involved in spinning up a Lambda may be unacceptable from an application perspective (however, there are ways to pre-warm Lambdas to avoid this);
- There will be an increased cost since quick Network I/O to an HTTPS endpoint will be a fraction of the cost of the execution of a Lambda.
However, for most use cases, these considerations should not cause an issue
Conclusion
It is possible to proxy requests to Private HTTPS endpoints using existing AWS Services.
Whilst this may not be an ideal solution for certain application types (i.e. HFT applications), it should fit most use cases.
Hopefully, in the future, this gap will be closed by AWS – although it is understandable that the SNS service was first envisaged as a service that facilitated external integration only / messaging only.
But the cloud moves fast!
Code
https://github.com/mcmastercloud/sns-to-private-https
Note: This article is an opinion and does not constitute professional advice – any actions taken by a reader based on this article are at the discretion of the reader, who is solely responsible for the outcome of those actions.
