3

I recently put together a small service to help produce a table seating plan, which was also a good chance to try some genetic programming. Before continuing, I should point out that there are already plenty of excellent software packages designed for this which is much more mature and fully featured, such as Perfect Table plan however, that’s never really the point.

Initially, the Clojure backend was a really simple library containing the table planning evolution code along with a single API endpoint wrapped in the Liberator library. The frontend was a really single page app written in Clojurescript and Om app (Om is a Clojurescript interface to React.js) and didn’t really do anything except that to save writing persistence backend functionality, it allowed the user to configure tables, guests and save them to the browsers local storage.

This was all great as the front-end was a single page app and was really lightweight, except that the backend was providing a single API POST endpoint. Even though my host for personal projects, Digital Ocean, is very reasonable from a cost perspective, it seems ludicrous to be maintaining (startup scripts, deployment automation, continuous delivery pipeline etc) and running a JVM process for a single endpoint that is so close requiring not a huge amount to run.

Luckily, Amazon recently announced their new AWS service, API Gateway, which promised server-less APIs so I thought I’d see if I could provide the API more easily and cheaply.

Although you can run both Clojure and Clojurescript on AWS Lambda (AWS Lambda Clojure guide), I had recently seen this excellent post about using Clojurescript on AWS Lambda, which linked to a cljs-lambda plugin and template for lein.

I had expected the Clojurescript version to start up more quickly (based on the node runtime) than the Clojure JVM equivalent and as AWS Lambda is charged based on the number of requests, time consumed, and memory used this seemed like an optimal choice. Without a JVM to start up, a no-op Clojurescript lambda seemed to run consistently at about 20ms.

As a minor word of warning for anyone using cljs-lambda within a VMWare host shared folder, the Lambda zip creation fails consistently, so better to use the VM storage.

The cljs-lambda plugin comes with some handy tooling to deploy a version of your lambda with a single command:

lein cljs-lambda deploy

The lambda exposed method is disarmingly simple, mine looks like this:

(ns tplambda.core
  (:require [cljs-lambda.util :refer [async-lambda-fn]]
                [tplambda.table-planner :as tp])
  (:require-macros [cljs.core.async.macros :refer [go]]))

(def ^:expose evolve
  (async-lambda-fn
   (fn [{:keys [popsize generations tables people]} context]
     (go
       (tp/evolve popsize generations tables people)))))

My Clojure table planning namespace is probably a detailed for another day, but crucially worked almost entirely correctly when compiled as Clojurescript once I’d removed the ratio literals. Once deployed, you can see your newly created lambda method from the AWS Console:

AWS Lambda

You can then test it via the console, logs also will be output to AWS Cloudwatch and will tell you the duration of the invocation and the memory used.

At this point, if you’re happy with the way it is running, you can create an API endpoint, simply by clicking on the API endpoints tab:

AddAPIEndpointFromLambda

And then adding your API Endpoint:

AddAPIEndpoint

APIGatewayoverview

If you created your AWS API Gateway endpoint as anything but open, you will likely need to create an API key:

CreateAPIKey

You can test the endpoint via the AWS Console, or if you prefer to check it yourself, you can make curl requests to test the endpoint and attach your API Key as a header using x-api-key.

curl -H "Content-Type: application/json" -H "x-api-key: APIKEY" -X POST -d "{\"json body\": \"goes here\"}" https://my-api-id.execute-api.region-id.amazonaws.com/test/mydemoresource

You might also need to enable CORS for my endpoint so that the Om front-end could make requests with cljs-ajax directly to the endpoint.

Unfortunately, the Clojurescript version of my table planning function seems to run approximately 70 times slower than the JVM hosted variant. I haven’t tried profiling it yet, but with an overhead of approximately 1400ms per invocation, there’s enough overhead to trial a JVM Clojure version instead.

So, was it cheaper?
Assuming 250,000 invocations in a given month and some moderate data transfer, this currently equates to roughly $3.62 per month. Considering that you can get a 1 vCPU, 512Mb Digital Ocean instance for $5 per month, it’s not cheap. It does however save you a lot of the admin overhead, but as with all AWS Services, you will really want to ensure that you properly estimate your usage fees before committing to any publicly accessible use.

Futher links.

Tags: , ,

3 Comments

  1. Moe Aboulkheir on the 31st July 2015 remarked #

    Jamie,

    I just deployed 0.2.1-SNAPSHOT of the cljs-lambda plugin, which consists of a fix for broken temp file renaming (in response to an Arch-related GH issue). I’m unclear of what the underlying problem is/was, but I’m curious about whether this fixes anything for the VMWare case you were mentioning. If you get a moment, can you bump your plugin version and file a Github issue if it doesn’t successfully deploy?

  2. Moe Aboulkheir on the 31st July 2015 remarked #

    Eh, 0.2.2-SNAPSHOT. Apologies for the spam.

  3. Jamie on the 1st August 2015 remarked #

    @Moe: Very sharp spot. This did indeed resolve the issue with the plugin being unable to create the zip file when the project resides on a VMware shared folder. Thank-you so much for letting me know.

Leave a Comment