Low Latency

In the old world of dial-up, we'd happily wait several seconds for a single web page to load. It didn't matter then—there was no live streaming, no chat, and no video calls. These days, we expect every page we visit to load in milliseconds and to have real-time communication with people on the other side of the planet. Low latency connections and applications are core to the modern web, particularly mobile applications.

What Is Low Latency?

Latency is the delay or lag time between input and response in a system. In technical terms, it denotes the time it takes for a packet of data to travel from the source to the destination. Latency is often measured in milliseconds (ms). Low latency means this delay is minimized. The qualification for "low latency" varies widely and depends on the specific application, user expectations, and industry standards. In audio or video streaming services, low latency is defined in terms of seconds, from 1-5 seconds. In a real-time video call, this amount of latency would completely ruin the experience, so latencies need to be less than < 150 milliseconds. In some contexts, like high-frequency trading, even microseconds matter, with low latency meaning 1-20 microseconds.

Applications Where Latency Matters

The below shows some applications where low latencies are important. But low latency is important in several applications on the web:

1. Audio and Video Streaming:

• Low Latency: 1-5 seconds
• In audio and video livestreaming, having a low latency of 1-5 seconds is vital to maintain sync between audio and video tracks and to allow real-time interaction between content creators and viewers. High latency causes delays in content delivery, leading to poor user experience. This delay is especially aggravating during live events, where the audience expects real-time interaction and engagement.

2. Activity Feeds and Notifications:

• Low Latency: < 1 second to a few seconds
• Activity feeds play a critical role in delivering timely updates to users. A notification at work, a new post on Discord, or a breaking news alert, users expect to receive these updates in near-real-time.

3. Real-Time Communication:

• Low Latency: < 150 ms (One-way)
• For chat solutions and video calls, maintaining one-way latency below 150 ms is crucial for chat and video calls to facilitate smooth and natural conversations. High latency can lead to awkward pauses and crosstalk, severely hampering the quality of communication. This low latency level is crucial for maintaining the flow of conversation and ensuring clear and coherent communication, especially in professional settings where clarity and immediacy are paramount.

4. Telemedicine:

• Low Latency: < 150 ms
• Telemedicine requires low latency to ensure timely and effective patient care. A minimal delay is essential, whether it's a simple consultation, remote monitoring, or even a surgery assisted by robotic devices. High latency in a telemedicine app can lead to miscommunications, inadequate medical assessments, or medical errors.

5. Online Gaming:

• Low Latency: < 100 ms
• In online gaming, achieving latency below 100 ms is critical to ensure fair and responsive gameplay. High latency can result in input lag and discrepancies between player actions and game responses, giving rise to unfair advantages and frustrating gaming experiences. This level of responsiveness is particularly crucial in competitive gaming, where even minor delays can significantly impact player performance and game outcomes.

6. Web Browsing:

• Low Latency: < 100 ms
• A latency below 100 ms for web browsing is important to ensure fast page loads and responsive user interaction. High latency can lead to slow website rendering and delayed response to user inputs, causing frustration and potentially leading users to abandon the website. Quick and responsive web pages are critical for user satisfaction, particularly in e-commerce sites where user experience directly impacts sales and conversions.

7. Augmented Reality (AR) and Virtual Reality (VR):

• Low Latency: < 20 ms
• For AR and VR experiences to feel immersive and real, a latency below 20 ms is critical. High latency in these applications can cause motion sickness or discomfort due to the discrepancy between the user's movements and the visual feedback they receive.

8. Financial Trading (e.g. High-Frequency Trading):

• Low Latency: 1-10 microseconds (μs)
• In financial trading, especially high-frequency trading, ultra-low latency is crucial because microsecond delays can result in significant financial losses due to price fluctuations. Traders rely on receiving and processing market data and executing trades faster than competitors to gain an advantage. In such environments, even minimal reductions in latency can lead to substantial improvements in trading performance and profitability.

In general, lower latency leads to better user experience and system performance, but achieving ultra-low latency might come with increased complexity and costs.

The Factors Affecting Latency

Several technical factors affect low latency, shaping the responsiveness and efficiency of systems and networks in handling data and executing actions. These can be broken down into two subcategories: physical factors and software factors.

Physical Factors

One primary factor is the network's bandwidth. A higher bandwidth allows more data to be transmitted over the network in a given time frame. However, a high bandwidth does not necessarily guarantee low latency, as the actual travel time of the data also depends on other factors. The propagation delay, determined by the distance between the communicating devices and the speed of the signal, is another critical factor affecting latency. The longer the distance and the slower the speed of light (or electrical signal) through the transmission medium, the greater the propagation delay and the higher the latency. The transmission medium itself, whether it's fiber, copper, or wireless, also plays a crucial role, as each medium has its inherent propagation speeds and susceptibility to interference, which can impact the signal's travel time. For example, fiber optic cables typically offer lower latency than copper cables due to the higher speed of light in glass or plastic. Router and switch processing times also contribute to latency, as each device a signal passes through needs time to process and forward that signal. The number of hops, or intermediate devices, between the source and destination, as well as the processing speed of each device, will impact the overall latency. Queueing delays at networking devices can also increase latency, especially in congested networks where multiple data packets compete for limited transmission resources. Effective network management and traffic prioritization can help in mitigating such delays.

Software Factors

In software and application domains, the efficiency of the code, the choice of algorithms, and the speed of the processing hardware also affect latency. Optimized, well-written code and fast, efficient processors can process inputs and generate outputs quickly, reducing the time delay experienced by the users.

Server processing time, which is the time the server takes to process a request and generate a response, is another factor that influences latency. Optimizing low-latency server performance, load balancing, and ensuring adequate resources can help reduce server processing times and lower latency. Lastly, the client-side rendering and processing also impact the perceived latency, as delays in rendering the received data or processing user inputs can make applications feel sluggish, even if the network and server-side components are optimized for low latency.

Planning For Low Latency

Implementing low latency can be challenging and critical for product managers or developers building low latency applications, as it directly influences user satisfaction and product performance.

First, product managers and developers must consider the application's performance needs and how latency affects user experience. Understanding the acceptable latency levels is crucial to meet user expectations, whether it's a gaming, livestreaming, or financial application. The design and architecture of the application play a vital role in achieving low latency. Asynchronous programming, efficient data structures, and algorithms are pivotal to optimize the software for speed and responsiveness.

Choosing the right network protocol is crucial. For example, utilizing protocols like HTTP/2 or WebSocket can reduce latency by enabling full-duplex communication over a single, long-lived connection, reducing the overhead of opening and closing multiple connections. From there, efficient data processing and storage solutions are essential. Opting for in-memory databases like Redis can decrease data retrieval times, and employing optimized query strategies can reduce data processing delays.

Developers then need to think about:

• 1. Code Optimization: Optimized and well-written code is fundamental. Developers need to employ profiling tools to identify and rectify bottlenecks in the code, ensuring the software components operate efficiently.
• 2. Content Delivery & Edge Computing: Deploying content closer to the user via CDNs or using edge computing solutions significantly reduces the time taken to deliver content to the user, improving responsiveness and user experience.
• 3. Scalability: The system must maintain low latency as it scales. Load balancing, efficient use of resources, and scalable architectures like microservices are crucial to handling increased loads without compromising latency.
• 4. Security Measures: Implementing security measures without affecting performance is challenging. Critical considerations include efficient encryption methods, lightweight authentication protocols, and secure yet fast transport layer security.

The Costs Of Low Latency

There is also a cost/benefit tradeoff with low latency.

Achieving low latency often means substantial investment in high-performance hardware, networking equipment, and potentially more expensive, low-latency data storage solutions. Additionally, developing optimized, low-latency software typically requires more time and specialized knowledge, which can increase development costs. Developers may need to learn new technologies, protocols, or programming techniques to optimize for low latency.

Managing and maintaining low-latency systems increases operational complexity and overhead because it requires more skilled personnel and sophisticated management solutions. Ensuring the system maintains low latency while scaling might require further expensive investments in scalable architectures, additional resources, and load-balancing solutions.

Implementing low latency is a significant challenge involving planning various technical aspects and a clear understanding of the associated costs, all while balancing performance, security, and user experience.

Frequently Asked Questions