hook.io

Nubstep: Overview and Part I - knob.io

Nubstep: Overview and Part I - knob.io

Nubstep is the project @pedrogte, @telmofcosta and me presented at Codebits 2011.

It is a sound synthethizer that collects inputs from an arduino, and generates sound through Core Audio. It also includes a web console for looking at readings (not shown at the competition presentation due to presenter lameness). All the data flowing from the arduino to the synthetizer is channeled through hook.io.

Overview:

  1. Arduino: Collect data from analog inputs (Detailed in this post)
  2. knob.io: Get readings from arduino and send them to hook.io (Also detailed in this post)
  3. knob-view.io: Show arduino readings on a browser
  4. nubstep: Synthetize sound waves and feed them to roar
  5. roar: Node library to output sound through Core Audio

Keep reading

CQRS and Hook.io

Hook.io is popular library for node.js, which allows you to build up distributed cluster of applications (hooks) which communicate by sending an events to each other. Command and query responsibility segregation (cqrs) is the way to design your server, where write and read side are completely separated. In last few weeks I’ve found those two perfectly match each other, allowing you to build up decoupled, scalable and fault tolerant applications. I expect reader to be familiar with both hook.io and cqrs. If you’re not, links above would be a great start.

Look into architecture

To build up your application, you would split it into multiple pieces - hooks. Each hook is a process with a single responsibility (the pattern is used in OOP as single responsibility principle, although its well suitable on the processes level as well). Each hooks does its job and emit event if something happens. Another hook, can listen for an event and act on that. Its basically observer pattern and thats the way hooks talk to each other (inter process communication is implemented in hook.io and is based on dnode).

In case of CQRS system, the application was distributed into 3 main hooks:

Fig.1: Green boxes (webserver / domain / denormalizer) represent separate processes (hooks). Each of them works separately and send events (green arrows) to other components. As you can see, events and read view data are stored separately.

Domain

Domain is responsible to encapsulate the business logic. Its a write side of the application stack.

Fig.2: Domain architecture. The domain has single command bus and multiple handlers (one per each command). Each handler can work with multiple aggregates and orchestrate processing of the command.

Domain listen for a specific set of hook events, where each trigger one command. Command is sent to a command bus, where appropriate handler is chosen. Handler executes the command by calling public API of aggregates (business unit in CQRS system) and each time command result in change of the system state, aggregate emits an event (This pattern is called event sourcing), which is stored in the event storage and emited from domain hook for interested parties.

Denormalizer

Data in the event storage would be hard to use to build up the views. Transforming the events to “read-ready” data (the data displayed in UI - reports, forms, …) is the job for denormalizer. Each time new event arrives, it takes the last snapshot, replay the new event and store the new snapshot (there can be even multiple snapshots with different version being built in parallel).

Fig.3: The denormalization process can be done either synchronously (directly on new event arrival) or asynchronously (sitting behind the queue, triggered once per X events or once in a given time) if you are ok with the data being stale for a reasonable amount of time. Both approaches can be used for different views.

Webserver

The webserver responsibility is to server the requests. It doesn’t matter which framework, protocol or data type is used. All what is done by webserver is processing the incoming request, build up an event with command to domain (in general when some data need to be saved) or query the read model (if some data need to be loaded from server). Notice the data were already denormalized for a read side, thus loading the data by webserver is trivial and very fast. It mostly does simple query by an ID to the read data storage.

Why cqrs and hook.io?

Scalability

In given configuration, you are able to scale each system component separately. You can choose a different strategy for each or use a different database for your events and for your read data models as well. With a hook.io you can easily distribute the components across your server infrastructure.

Fault tolerance

With a recent changes there is no master hook, thus the system keep running if any hook fail down. On top of that with an autoheal feature, system can try to heal itself by spawning new process after failure. Because hooks are very light-weight processes, they can be spawned in a very fast way, ensuring your system keeps up and running with a minimal down time.

Decoupling

You have well decoupled architecture where your delivery mechanism (webserver) and data persistancy (denormalizer and read model) are decoupled from your business logic (domain). The biggest benefit in my eyes is, that you are allowed to focus on a single thing in a given system component.

Easy webserver switch

Because webserver only responsibility is to handle requests, you would have no problem to change it for another one. It would make sense as well to have multiple webservers side by side. One can serve the HTML pages while other might focus on JSON API (if your HTTP server goes down, your API users wouldn’t worry at all).

Ability to change read data

It happens quite often these days business requirements change significantly in time. Data your users wanted to see might not be valid anymore thus you will have to adapt as well. Event sourcing allows you to drop the read data completely and reconfigure your denormalizer to build up another read data (as long as necessary information is contained in your events). This mean you can react instantly and project changes even to the past.

Conclusion

I hope I’ve listed reasonable set of arguments, why it might be worth to use cqrs with hook.io. I am not saying its silver bullet for everything but might be well suitable for many systems.

Project node cqrs (v0.5.1) is still quite a lot under development and would probably be considered as naive by C# or Java enterprise guys, where CQRS is used more often, although its light weight and supports the described architecture already. I am working on another set of features and if someone is interested in more details about implementation with hook.io let me know.