I love talking about asynchronicity quite a lot.
The concept of asynchronicity, where things are executed flawlessly, is quite fascinating to me. We can be more efficient and get more things done this way. Async is a way of life!
Coming back to technology.
Have you heard of the C10k problem? It’s a pretty interesting one.
In the early 2000s, when RAMs were really expensive, there was this theory that servers can only serve up to 10,000 simultaneous requests. This is not 10k requests per second, mind that. But 10,000 requests were processed simultaneously.
There are multiple factors to it.
One was that for every request, a thread(a very small process) is allocated and each of them consumes 1MB of memory. Given the size of RAMs those days and considering the overhead of context switching, 10k was the magic number that came up.
Ryan Dahl, the game-changer 🔥
Enter 2009, Ryan Dahl, an amazing amazing programmer, asked a very serious question that changed our ways of running web servers.
He asked, “What are the threads doing while they are serving database queries and making external API calls?”. The answer is “they are sitting idle”. He argued that this idle time can be utilized to serve other requests.
Wow! Amazing!
He then went on to create NodeJS, which is a single-threaded server, following which other frameworks also started shifting to single-threaded architectures.
What is Asynchronous execution in a server?
The concept of asynchronous programming is general to programming but here we will talk only about servers and APIs.
The core idea behind asynchronous programming is that a single thread is responsible for executing multiple requests or functions.
This is counterintuitive. How can one single thread be more efficient than multiple threads?
For this, let’s blame our poor understanding of how a request works.
When a user makes a request, the underlying API code doesn’t take a lot of time to execute assuming we are using standard libraries which are written in C.
The program communicates with the database and asks for a query. Now, it’s the database that does the rest of the processing i.e. there’s a separate process or thread created at the DB’s end which queries the tables and returns the result.
So while I say that the thread executing the code on the server is idle, this is what I mean!
Okay, so what’s async?
When the thread on our server is waiting for the DB process to return the result, it can do other things like serving other requests! (Basically get the most bang for your buck from those AWS guys 😎)
Why are threads not the best option here? As mentioned in the section above, there are limits to the number of threads you can spin up because of their memory consumption. If you only have one thread executing all requests then that’s pretty cool, right!
Where does async fail?
Let’s take another scenario. You are exposing an API, which serves a deep learning model. Your benchmarking shows that it takes more than a minute to complete the request because it’s a hell of lot of calculation that your CPU has to do to achieve the result.
Can async help here? No!
Async only helps in situations where CPU consumption is minimal and where your API is waiting on external calls like DB queries and third party API requests.
In the above scenario, threads or multiprocessing will be a better alternative.
How does async work in Django?
Async is only possible if your framework as well as your HTTP server program (like gunicorn, uvicorn) should implement this feature explicitly.
Let’s take a case where you are serving a Django application on a Gunicorn server.
In Gunicorn, the process which handles your requests is termed as a worker. Gunicorn has multiple worker types which you can choose from. The choice depends on how you want to serve your requests. It can be a threaded implementation. Or it can be asynchronous implementation. Gunicorn along with Uvicorn gives you async support. So that’s on the HTTP server end!
Now comes the framework. Earlier if you had Django, every request is executed using a thread. But with the release of Django 3.1, you can change this behavior.
Django 3.1 introduced async views. Now, if you want, you can write your views such that your service can take advantage of the asynchronicity mentioned above.
You wouldn’t really notice any difference between sync and async in your application until you reach a certain scale. Once you cross a certain number of simultaneous users, your application starts to slow down and requests will start taking more time. That’s when async’s advantages start to kick in.
If you are building for scale, adding async is gonna impact your application on a whole another level. Also async allows you to use slow streaming, long-polling, and other response types.
To learn more and get code samples, you can visit Django’s official documentation .
References:
c10k problem: https://en.wikipedia.org/wiki/C10k_problem
Django async views, explanation with code:
I write about Software Engineering and how to scale your applications on my weekly newsletter. To get such stories directly in your inbox, subscribe to it! 😃
