Skip to content

Background Jobs

Definition

Background jobs run without user interaction.

Many types of applications require running background tasks independently of the UI. Tasks such as batch jobs, intensive processing tasks, and long-running processes.

The application will start the job and still continues to process the users requests. This approach minimizes load on the application UI, which increases availability.

Example

If an app needs to generate thumbnails that users upload, it can run this work in a background job and save the thumbnail to a storage. The user doesn't need to wait and can instead do other stuff on the app.

If a task can run without user interaction and without requiring the UI to wait for completion then it should be a background job.

Types of Background Jobs

Types of Background Jobs

  1. CPU/GPU-intensive jobs like math calculations or structural model analysis
  2. I/O intensive jobs like storage transactions or file indexing
  3. Batch jobs like nightly data updates or scheduled processing
  4. Long-running workflows like order fulfillment
  5. Sensitive-data processing where you send the task to a more secure location for processing

Triggers

A background job can be initiated in several ways. These triggers fall into two categories:

  1. Event-driven triggers: An event, usually a user action or a step in a workflow will start the task
  2. Schedule-driven triggers: A timer invokes the task on a recurring schedule or at a specific time

Event-Driven Triggers

Uses a trigger to start a background task.

Example

The UI or another job places a message in a queue. The message contains data about an action that occurred, like the user placing an order. The background task listens on this queue for new messages and will use the data as input for the background job.

This pattern is known as asynchronous message-based communication. If the request volume is bursty, its best practice to use the Queue-Based Load Levelling pattern so the queue buffers' demand and the background task processes work at a controlled rate.

Example

The UI or another job makes a request to an endpoint, like an HTTPS uniform resource identifier (URI), or an API exposed as a web service. It passes the data required by the background job and upon calling the endpoint, the background job gets invoked.