Skip to content

Latency vs Throughput

Definition

Latency: the time it takes for a single request to complete

Throughput: the amount of work done in an amount of time (ie. Requests per Second, Transactions per Second)

Anatomy of Latency

Latency is the total sum of the latency for each component in a system:

  • Network latency: The physical time it takes to travel across the internet
  • Processing latency: How quickly CPU's can execute logic
  • Queueing latency: The time a request sits in a buffer
  • Database querying latency: The time it takes to query data
  • etc...

{ width="500" }

Throughput and Capacity

Throughput is a measure of the system's capacity. If a system can handle 1,000 requests per second before error rate or latency spikes, then its effective throughput is 1,000 RPS.

Throughput is limited by the bottleneck - the slowest component in a system. If an application server can process 5,000 RPS but the database component can only handle 500 RPS, then the system's effective throughput is only 500 RPS.