Pull DynamoDB locally

Being able to architect, develop and test your application without deploying to AWS when using AWS DynamoDB? Lets explore the self-contained version by doing some hands-on Docker and C#.

I've found myself using AWS DynamoDB way too many times already. The service is great due to the astonishing performance and the durability it comes with. Overall one could say AWS DynamoDB is perfect in its simplicity, but complex in its subtlety.

One of the recent projects I worked on at my company was a simple API (backed by AWS API Gateway and running on AWS Lambda). I was looking for a data store and quickly decided to go with DynamoDB (due to the throughput and data we planned on storing).

I dislike to conflict with a peer on accident in a staging environment. It's always a good practice to stay away from shared resources when testing or developing. I was unable to debug my DynamoDB integration without having to deploy it to the staging environment everytime. After some googling I quickly came across:

AWS DynamoDB locally

A downloadable version of AWS DynamoDB available as a Docker image.
Yes you read that right, and it works like charm. In this article I'll demonstrate how to work with the local AWS DynamoDB service and provide you with some C# example code to get you started quickly. I wont explain you how AWS DynamoDB works and I expect you to have Docker installed.

Lets get started by creating a docker compose file. I tend to prefer 'docker-compose' over simply using 'docker run' as I quickly forget what I defined for a specific container running on my system. We defined port mapping (8000), a volume for persistence and a command that runs when the container is created.

The parameter I'm passing to the jar file '-sharedDb' indicates that DynamoDB will create a single database file named 'shared-local-instance.db'. Every application that connects to DynamoDB accesses this file. If you don't pass '-sharedDb' the database file is named 'myaccesskeyid_region.db' (seperate files are created depending on configured credentials). The '-dbPath' represents the directory where AWS DynamoDB writes its database file. If you don't specify this option, the file is written to the current directory. There's a few other parameters that we can pass, but that's not relevant for now.

Lets spin up our AWS DynamoDB container by running the following:

To verify if our container is up and running we can pass a filter down to the 'docker ps' command.

The container is up and everything looks ok, time to get started.

Pointing the AWS cli to our container

We interact with the container by using the AWS cli (if you don't have this installed yet, you can run the following formulae: 'brew install awscli'). When running any 'aws dynamodb' command by default the AWS cli uses an endpoint based of your configured profile: 'http://dynamodb.{Region}.amazonaws.com' to interact with the web service. As we are not interested in talking to the web service we can pass an additional parameter to the 'aws dynamodb' command in order to point to our container. The parameter I'm passing is '--endpoint-url' with the value 'http://localhost:8000'. This tells the AWS cli to interact with my AWS DynamoDB running locally.

Running the preceding command now yields an empty array of table names as we haven't created any yet, but we did manage to talk to our container.

Lets create a table

Creating a table through the AWS cli is fairly simple. Be aware that when passing provisioned throughput options to the 'create-table' command it will be ignored in the downloadable AWS DynamoDB even though it's required to pass a '--billing-mode' parameter to the 'create-table' command.

After running the 'create-table' command you should get the same output as above. If you now feel like running the 'list-tables' command again (which was earlier returning us an empty array), you'll see the 'our-demo-person-table' in the response.

The container is running, the table is created and we know how to interact with it by using the AWS cli. Great, we've setup all of the prerequisites in order to debug our AWS DynamoDB integration locally.

Time to hook up our C# application

Doing so, simply requires us to set the optional 'ServiceURL' property. We do this when we new up a 'AmazonDynamoDBConfig' object that we inject in the 'AmazonDynamoDBClient', which we then in inject in the 'DynamoDBContext'. This might sound a bit abstract, therefore I've shared a piece of code with you.

By only adjusting the table and ServiceURL we managed to make use of our local running AWS DynamoDB. Now you no longer need to deploy over and over to test your AWS DynamoDB integration. Great, problem solved.

Differences between the web service

It's good to note that there's a few differences due to the downloadable version of AWS DynamoDB being intended for development and testing purpose only compared to the web service. One difference that I want to point out, which is very logical but often forgotten is that the read/write ops on table data is limited by the speed of your machine. Please visit the 'Differences Between Downloadable DynamoDB and the DynamoDB Web Service' section for a more detailed list of differences.


Footnotes


Did you enjoy this read? Feel free to buy me a coffee! :)

Contact me? You can do that through blog@bschaatsbergen.com or LinkedIn.

If you're looking for other articles I recommend you to look in the library.