Creating modules

So you want to build your own module...

Great. Well we’ve already built some. We recommend looking at the group mailer as a an example.

Your module can be anything, from a teeny tiny lambda to a web app. An application can both produce and consume events. It’s completely up to you.

Regardless, the steps depend upon whether your app will be producing events, consuming events, or both.

Will your app be producing events?

At some point in its code it will need to connect to to the Kinesis stream to send events. Stream details should be provided through configuration rather than hard-coded in the app. Especially since ideally your module will be available for others to use.

We provide a stream client for javascript which helps to abstract away some of the logic for connecting to kinesis.

You’ll also want to work out what your payload will look like, structuring it in a way that is friendly to consumers.

And you may also want to think about validation logic. Your events are going to be asynchronous, so if you break something downstream you won’t easily know about it.

Will your app be consuming events?

It will need to pick up events from the stream. However what with javascript being single-threaded we can’t keep a thread regularly polling the kineses stream. So we implement our javascript consumers a little differently from our producers.

We have created an event forwarder which polls the stream on behalf of a javascript client. Whenever it picks up an event it is interested in it will forward it over https to it's registered client.

In more practical terms, this means the following when creating a javascript consumer app:

1. Create an RESTful endpoint for your app to pick up POST requests for ‘/events/‘. You can then create a handler for any events sent to this endpoint.

But you don't need to do a lot of the heavy-lifting. We provide a stream client for javascript which abstracts away some of the pain of processing events. Using the client you can register types of events to handlers and register the client itself against the events endpoint.

2. Create an event forwarder lambda to forward any relevant events to your app.

Please note: If you write a consumer in a multi-threaded language you can just poll the stream directly.