Localstack S3 and Go

July 26, 2020

I spent too much time Saturday getting the Go S3 SDK to work with LocalStack.. It turns out that if you are using LocalStack, you need to explicitly configure the following properties:

	sess, err := session.NewSession(aws.NewConfig().
		WithEndpoint(endpoint).
		WithRegion(region).
		WithS3ForcePathStyle(true))

The requirements for Endpoint and Region are obvious. If S3ForcePathStyle is not specified, then LocalStack will fail.

	data, err := svc.GetObject(&s3.GetObjectInput{
		Bucket: &bucket,
		Key:    &cfgKey,
	})

What is path-style? In May 2019, Amazon deprecated path-based access model for S3 objects. This means one should no longer use URLs of the form:

https://s3.amazonaws.com/jbarr-public/images/ritchie_and_thompson_pdp11.jpeg

But rather use

https://jbarr-public.s3.amazonaws.com/images/ritchie_and_thompson_pdp11.jpeg

While the recommended workaround works, it is a deprecated API. One could only enable path-based addressing locally, but then you will be using different code paths locally vs in-AWS.

To be fair, I never expected LocalStack to be a faithful representation of S3. The fact that it works at all is something to be grateful about. But it is annoying to write LocalStack-specific code. I suppose that is the nature of leaky abstractions.