5 Things We Learned About AWS From Q2 Earnings
11 Aug 2016How To Reduce EBS Costs On AWS
23 Sep 2016The Serverless framework is awesome and we love working with it! We even used it to build a Serverless HipChat Bot. Side note: you can check out our Serverless HipChat Bot boilerplate repository. However, we quickly found that the feedback loop in development was really slow. Every time we wanted to test a change, we had to deploy to a stage, which at times would take a while (thanks to our average Australian Internet)!
Deploying frequently to a development stage gave us great dev/prod parity (factor 10 of 12) but unfortunately, it also meant that things were moving a little too slowly. I realized that what I needed was a local environment that I could run on my computer that could emulate components of a serverless deployment in AWS.
A HipChat plugin is just like most other web applications and can be expressed as a few different components:
- Endpoints, that expose configuration, actions and data (sometimes via glances)
- Static content, that is rendered in the plugin store, in the sidebar and dialogs
- Persistent state, about each HipChat installation and OAuth details
In our serverless HipChat plugin, we used:
- API Gateway and Lambda for endpoints
- S3 for static content
- DynamoDB for persistent state
But how could these be replaced locally?
API Gateway
Luckily, this problem had already been addressed for node development at least, by David Hérault in the serverless-offline npm module.
The good:
- This plugin handles cache invalidation (loads each time you save your files)
- Handles some configuration from your
s-function.json, such as timeouts, response parameters, etc
The bad:
- No 40x error codes when errors thrown, still receive a 20x
- Differences in some response formats between AWS and serverless-offline
All in all, this is a pretty good emulation of API Gateway in most cases and will continue to get better as it matures.
To get started: $ npm install --save-dev serverless-offline
Add `“plugins”: [“serverless-offline”]` to your `s-project.json` file $ sls offline start
S3
S3 is great for serving static content. Having to upload to S3 to test your static content is bothersome, even with great commands like the AWS CLI’s `aws s3 sync` command.
This was a simple solution. Run the SimpleHTTPServer module $ python -m SimpleHTTPServer 8010 This serves your static content out of port 8010.
Check out our Gruntfile.js for a more sophisticated example, where we also substitute stage/environment specific configuration into our static assets.
DynamoDB
DynamoDB is AWS’ fully managed NoSQL database. It isn’t the only choice for storing state for serverless applications, but a good choice where you may be looking to have more structured data, or don’t need the state in memory.
DynamoDB Local is a downloadable java version of DynamoDB that you can download and run locally.
To get started:
- Download DynamoDB Local
$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar
This exposes a local version of DynamoDB that by default runs on port 8000. Now, you just need to specify a local endpoint when interfacing with DynamoDB via the CLI or in the persistence layer in your code.
$ aws dynamodb list-tables --endpoint-url http://localhost:8000 --region __your-region__
The last thing you will need to do is to create tables in your local DynamoDB to match those in whichever stage you are trying to emulate locally. We specifically wrote Our serverless HipChat Connect script to simplify this process, in cases where we save our DynamoDB table names as environment variables from our CloudFormation Template outputs. We also made it easier to invoke this in our Gruntfile.js.
Tunnel to localhost
The last bit of magic that made things even easier was ngrok. Using ngrok, we could expose our local versions of API Gateway and S3 to public HTTP and HTTPS endpoints. Using ngrok, I could install my locally hosted HipChat plugin to a live HipChat room. This meant that my local changes to the codebase were immediately exposed for testing from my HipChat room’s actions & webhooks.
If you want to see a development environment in action, check out our screencast
This was our first serverless project and I’m sure this is just one of many approaches to solving this problem. How are you doing local development with serverless projects? Let us know!
