Scalable applications need a way to process many incoming request per second. Each request may take some time to process because data needs to be retrieved from a data source or because some calculations need to be performed. Since waiting for each request to complete before handling the next request would lead to slow response time there are at least two techniques that can be implemented: multithreading or asynchronous processing. Multithreading is provided by many programming languages such as Java and C++. It provides scalability by running incoming requests as parallel threads. Asynchronous processing with JavaScript or TypeScript is used in Node.js and Deno. These environments use non-blocking event model where the main thread never stops to handle IO related tasks such a database connections, reading files or http requests. The main advantage of this model is that it requires less resources because it does not need to keep track of threads and their context. To explain the conceptual difference let’s look at an analogy where a computer server is a restaurant and clients are coming in to have a meal.
Multithreading
In the multithreading ‘Java’ version of the restaurant a waiter is assigned to every guest. The waiter takes the request from the guest and goes to the kitchen. The cook reads the order and prepares the food. The waiter remains in the kitchen until the food is prepared and then delivers the order to the guest. So if you have five quests you need to have five waiters. Even if these waiters are idle while waiting for the order to be prepared you still need to pay them. You also need a kitchen that is big enough to accommodate them while they are waiting and maybe provide some chairs. You also need to take precautions to prevent clashes when waiters try to enter and exit through the same door. And of course you need to avoid race conditions where two waiters try to grap the same order that it just prepared.
Asynchronous
In the asynchronous ‘Node.js’ version of the restaurant there is only one waiter. The waiter takes the order and walks to the kitchen to hand over the order. However, the waiter does not wait for the order to be prepared but instead immediately goes back to the restaurant to takes orders from another tables. If the kitchen has completed the order the waiter is informed and picks up the meal and brings it to the table. It is obvious that with one waiter the restaurant needs less resources and the situation of idle waiters that take up kitchen space is avoided. There is also no danger of clashes with the kitchen door or multiple waiters grabbing the same order.
However, there is a downside to this model. Since there is only one waiter everything will come to a halt if this waiter is busy. For example, if a group of guests wants to split the bill based on what each of them ordered the waiter needs to make calculations which takes time. Or maybe the waiter is hold up by one of the guests telling a story or a joke. In a real world Node.js scenario this means that the application stops responding to user input when the server is iterating over a loop or executing code. Luckily a solution comes to the rescue here.
Worker threads
To solve the issue of blocking the main event loop while doing difficult work, the Node.js platform has introduced worker threads. Worker threads can process custom logic in parallel to the main event loop. Information between the threads can be passed via messages or shared memory. Going back to the analogy of the restaurant, this means that the waiter temporarily clones itself so the clone can handle the difficult calculations while the original waiter can continue to serve other guests. Of course there can be many clones active at the same time and they magically disappear when they are done, not taking up any resources.
A practical application of worker threads is running real time machine learning or data analysis in the background while the main event loop continues to serve user requests and handling browser clicks. When the server has completed the work the result can be send back as a standard response to the frame that triggered the request, for example in a dashboard panel. For single page applications with only one frame you can use web sockets to push the result back to the client application, which can then update the screen to show the result.
Conclusion
With worker threads Node.js provides a unique framework for developing compute-intensive application with a single JavaScript/TypeScript environment for both front-end and back-end. The combination of asynchronous processing and multi threading is not unique to Node.js, but is also supported by Java 8, Go or Rust. However, these languages are not designed for front-end development so you have a split code base and skill set. An alternative to using worker threads is implementing compute-intensive logic as separate Node.js services that can be run in parallel as docker containers or as pods in a Kubernetes environment. The choice between asynchronous, multi threading or microservices depends on many factors including use case, load and skillset of software engineers and DevOps. Many frameworks can often be applied to achieve the same result.
