AWS Step Functions

I briefly worked on a workflow system at $past_job, implemented using AWS Step Functions. My experience was pretty terrible. I wasn’t sure which technical requirements led the team to this system. Some people said we needed a “system that is configuration driven and not code driven” and some people said “we needed something that scales.” Whatever the reason was, making improvements to this system was a pain in the ass, with AWS Step Functions itself being somewhat responsible.

September 8, 2020

I want to love DynamoDB

I want to love DynamoDB. I love that it just scales (disk usage and processing power). I love that it is tightly integrated with AWS’s IAM model, so I don’t have to deal with user/role/permissions management. But DynamoDB does some weird things by design. For example, only the primary key can be unique. If you want a table with multiple unique attributes, for example, a Users table where both the user_name and email are unique, you’ll have to do weird things like this.

July 26, 2020

Localstack S3 and Go

I spent too much time Saturday getting the Go S3 SDK to work with LocalStack.. It turns out that if you are using LocalStack, you need to explicitly configure the following properties: sess, err := session.NewSession(aws.NewConfig(). WithEndpoint(endpoint). WithRegion(region). WithS3ForcePathStyle(true)) The requirements for Endpoint and Region are obvious. If S3ForcePathStyle is not specified, then LocalStack will fail. data, err := svc.GetObject(&s3.GetObjectInput{ Bucket: &bucket, Key: &cfgKey, }) What is path-style? In May 2019, Amazon deprecated path-based access model for S3 objects.

August 19, 2019

Own Your Data

I previously wrote about owning my own data. An important part of data ownership is backing up your data. I use S3 as my long term data store. It is pretty easy to set this up using Terraform.

S3

Provisioning a S3 bucket is simply a single Terraform resource:

resource "aws_s3_bucket" "repo_archive_log" {
  acl = "log-delivery-write"
  bucket = "example-bucket"
  tags = {
    Name = "example"
    TTL = "persistent"
    ManagedBy = "Terraform"
  }
}