Polling

Polling refers to the process of checking the status(getting new data) of a request made by the client to the server at regular intervals. This process of checking the status can occur thousands of times a second.

Types of polling

1. Short polling

2. Long polling

Some applications require real-time updates and developers often decide between using short or long polling for backend communication. Short polling is quite easy to implement but it wastes backend resources because it has to make requests several times. Long polling, on the other hand, reduces the number of requests sent but it’s more complicated to implement.

In this article, we'll explore both short polling and long polling, and discuss their pros and cons, using JavaScript as the programming language.

Short Polling

In short polling, the client sends a request to the server, and the server responds with a unique ID while the request will keep processing in the server. The client will send another request using the ID, if there are no updates, the client will send another request, and the process repeats until the server responds with a success status to the client. This technique is useful when a request takes a long time to process, such as uploading a video to YouTube.

Let's go through a code sample, In the example, I will be using JavaScript and the express framework to spin up an HTTP server

// short-poll.js
const app = require('express')();
const { v4: uuidv4 } = require('uuid');

const jobs = {
  '1b9d6bcd-bbfd-4b2d-9b5d-ab8dfbbd4bed': 'pending',
  'u65675654654-bbfd-4b2d-9b5d-ab8dfbbd4bed': 'done',
};

app.post('/submit', (req, res) => {
  const jobId = `job:${uuidv4()}`;
  jobs[jobId] = 'pending';
  setTimeout(() => updateJob(jobId), 10000);
  res.json({ jobId, status: jobs[jobId]  });
});

app.get('/status', (req, res) => {
  console.log(jobs[req.query.jobId]);
  res.end('\n\nJobStatus: ' + jobs[req.query.jobId] + '%\n\n');
});

app.listen(9001, () => console.log('listening on 9001'));

function updateJob(jobId) {
  jobs[jobId] = 'done';
}

The code above is a simple express server, that uses an object to store data.

app.post('/submit', (req, res) => {
  const jobId = `job:${uuidv4()}`;
  jobs[jobId] = 'pending';
  setTimeout(() => updateJob(jobId), 10000);
  res.json({ jobId, status: jobs[jobId]  });
});

The route above is a POST route, that creates a job and set the current state of the job to pending then update it after 10s. The update will happen in the background but it immediately responds with the jobId which the client will use to poll.

app.get('/status', (req, res) => {
  console.log(jobs[req.query.jobId]);
  res.end('\n\nJobStatus: ' + jobs[req.query.jobId] + '%\n\n');
});

This route is where the client will check the status of the job, In our example, we have two different states (pending and done). The client will keep sending requests to this route till the server responds with the done status.

To start the server, run node short-poll.js

Now let's test out the code. I will be using postman as the client.

1. Send a POST request to this URL localhost:9001/submit

2. It will respond with the jobId. This jobId is what we will use to poll.

1. Send a GET request to localhost:9001/status?jobId=jobId

1. It will return the current state of the job whether it's pending or done. keep doing step 3 till the server responds with done.

In this short example, we have seen how easy to implement and how inefficient and resource-wasteful short polling is, as it generates a large number of requests, even when there are no updates.

Long Polling

With Long polling, the client sends a request to the server, if the response required is not ready, the server will keep the request open till there is a suitable response, when the server has the response required, the server will send the response back. This reduces the number of requests sent by the client, making it more efficient than short polling.

Let's go through a simple example of how long polling work with JavaScript

// long-polling.js

const app = require('express')();
const { v4: uuidv4 } = require('uuid');

const jobs = {};

app.post('/submit', (req, res) => {
  const jobId = `job:${uuidv4()}`;
  jobs[jobId] = 'pending';
  setTimeout(() => updateJob(jobId), 20000);
  res.json({ jobId, status: jobs[jobId] });
});

app.get('/status', async (req, res) => {
  const jobId = req.query.jobId;
  //long polling, don't respond until done
  while ((await checkJobComplete(req.query.jobId)) == false);
  res.json({ jobId, status: jobs[jobId] });
});

app.listen(9002, () => console.log('listening on 9002'));

async function checkJobComplete(jobId) {
  return new Promise((resolve, reject) => {
    if (jobs[jobId] !== 'done') this.setTimeout(() => resolve(false), 1000);
    else resolve(true);
  });
}

function updateJob(jobId) {
  jobs[jobId] = 'done';
}

The code above is similar to the short polling code, the major difference is how we check for the job status.

app.post('/submit', (req, res) => {
  const jobId = `job:${uuidv4()}`;
  jobs[jobId] = 'pending';
  setTimeout(() => updateJob(jobId), 20000);
  res.json({ jobId, status: jobs[jobId] });
});

This route submits the job to the server, it returns the jobId and status almost immediately while it updates the job status in the background after the 20s(20000ms).

app.get('/status', async (req, res) => {
  const jobId = req.query.jobId;
  //long polling, don't respond until done
  while ((await checkJobComplete(req.query.jobId)) == false);
  res.json({ jobId, status: jobs[jobId] });
});

This is where the long polling happens, this route checks for the status of the job, When a client sends a GET request here, it will keep the connection open with the while loop till it gets a status of done, then it returns to the client then closes the connection.

async function checkJobComplete(jobId) {
  return new Promise((resolve, reject) => {
    if (jobs[jobId] !== 'done') this.setTimeout(() => resolve(false), 1000);
    else resolve(true);
  });
}

This helper function takes a jobId as an argument which is used to check whether a specific job is complete or not. The function returns a Promise that resolves to a Boolean value indicating whether the job is complete or not.

Now let's test, I will use postman as the client.

1. Run the server node long-polling.js

2. Send a POST request to this URL localhost:9002/submit

   

3. It will respond with the jobId. This jobId is what we will use to poll.

4. Then the client will send another GET request to localhost:9002/status?jobId=<jobId>

   

5. This request will not return until it gets a good response from the backend

From the above example, Long polling reduced the number of requests made to the server by the client, making it more efficient than short polling. The downside of this approach is that it can be more complex to implement because the connection needs to be open on the server side till there is new data to be sent to the client.

Conclusion

When building applications that are real-time, polling can be an option to use and developers will decide on whether to use short polling or long polling to communicate between the client and server side. Short polling can be easy to implement, but it can become inefficient and wasteful of backend resources when too many requests start going in. Long polling, on the other hand, is quite efficient than short polling, because of the reduced number of requests, but it can be more complex to implement.

Code source:

https://github.com/rammyblog/polling-example

References

https://www.udemy.com/course/fundamentals-of-backend-communications-and-protocols/

Image

With the admin interface, You can rapidly perform CRUD operations with any database model. The admin comes with hooks for easy customization.

In this tutorial, I will guide you on how to create multiple independent admin sites. The common method to make the admin interface is to place all models in a single admin. Anyway, it is possible to have different admin pages in a single Django application.

Prerequisites

For this tutorial, we are going to create a simple Django application with two models. Before you begin this tutorial you'll need the

• Virtual Environment
• A web browser
• A code editor e.g VSCode

Let’s get started…

Creating the Django Application

Step 1: Create and activate a virtual environment, I will name the virtual environment venv.

virtualenv venv
source venv/bin/activate

You’ll know that your virtual environment has been activated, because your console prompt in the terminal will change. It should look something like this:

(venv) $

Step 2: Install the latest version of Django using the command below

pip install Django

Step 3: Create the Django application, for the purpose of this tutorial we would name the application core

django-admin startproject core .

The folder structure should look like this.

Run the application using the command below

python manage.py runserver

Your server should be running on http://127.0.0.1:8000/

Ignore the unapplied migrations message

Setting up the Django Application

In the previous section, we created our Django application, now it is time to set it up. Step 1: Migrate the default tables.

 python manage.py migrate

Step 2: Create a superuser, this superuser account will be used to access the admin pages.

python manage.py createsuperuser

You’ll then be asked to enter a username followed by your email address and password. Once you’ve entered the required details, you’ll be notified that the superuser has been created.

Username: admin
Email address: admin@example.com
Password:
Password (again):

Creating the app and models

Step 1: Create an app called blog.

python manage.py startapp blog

Step 2: Open the settings.py file located the core folder(core/settings.py), then edit the INSTALLED_APPS to this

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'blog'
]

Step 3: we need to create two simple models in the blog app called todo and post. In blog/models.py add the following code

from django.db import models

class Todo (models.Model):
    title = models.CharField(max_length=100)    

    def __str__(self):
        return self.title


class Post(models.Model):
    title = models.CharField(max_length=500)
    image_url = models.URLField(max_length=500)

    def __str__(self):
        return self.title

After adding the code, we need to makemigrations and migrate, so that the Post and Todo tables can be created in the database

python manage.py makemigrations
python manage.py migrate

Step 4: Add the two models to the admin interface so we can perform crud operations on both models, In blog/admin.py add the following code

from django.contrib import admin
from .models import Post, Todo

admin.site.register(Post)
admin.site.register(Todo)

After adding the models to the blog/admin.py file you can then start the server(python manage.py runserver), then navigate to http://127.0.0.1:8000/admin/, it will ask you to login, input the superuser details you created earlier. The page should look like this.

Currently, our todo and post models are in the same place. Now we want to split the admins.

We will not tamper with the default admin but rather create an independent admin for post which will be a new subclass of admin.AdminSite

In our blog/admin.py we do:

from django.contrib import admin
from .models import Post, Todo

class PostAdminSite(admin.AdminSite):
    site_header = "Blog Post admin"
    site_title = "Blog Post Admin Portal"
    index_title = "Welcome to Rammyblog"

post_admin_site = PostAdminSite(name='post_admin')


post_admin_site.register(Post)
admin.site.register(Todo)

I created a new class called PostAdminSite and inherited from admin.AdminSite, then I declared the following varaibles site_header is the HTML tag of the Post Admin Page site_title is the HTML tag of the Post Admin Page index_title will replace the Django Adminstration title on the admin page.

Then I created an Instance of the PostAdminSite class called post_admin_site, then I registered the Post model there.

Routing the independent admin pages

We already have a separate admin class for the Post model, we just need to find a way to access it on the web interface. Step 1: Open core/urls.py change the urls.py to

from django.contrib import admin
from django.urls import path
from blog.admin import post_admin_site

urlpatterns = [
    path('admin/', admin.site.urls),
    path('post-admin/', post_admin_site.urls)
]

This separates the Post admin from the Todo admin. Both admins are available at their respective URLs, http://127.0.0.1:8000/admin/ and http://127.0.0.1:8000/post-admin/.

The Post Admin should look like this

while the common admin page should look like this

You can create more models and add them to the admin class you want, you can have a model registered in different admin class as well.