Advanced Message Queuing Protocol (AMQP)

Your bank app lets you view your balance and transaction history, transfer funds, make investments, and manage your accounts and loans. It might also alert you when funds are low or a payment is due.

How do developers enable communication between disparate services for millions of users without dropping transactions or other errors? They're likely using AMQP.

Read more to learn about AMQP, how it works, popular brokers and clients, use cases, and best practices.

What Is Advanced Message Queuing Protocol?

Advanced Message Queuing Protocol (AMQP) 1.0 is an open standard and wire-level protocol for connecting message-oriented middleware systems. In 2014, it was defined internationally as ISO/IEC 19464:2014.

AMQP provides a secure way for systems to communicate with characteristics like:

• Broker and peer-to-peer architecture
• Interoperability across a variety of languages, applications, and platforms
• Flow control at multiple levels for resource management
• Secure messaging with transport-layer and application-layer security with Transport Layer Security (TLS), Simple Authentication and Security Layer (SASL), Kerberos, and more
• Three Quality of Service (QoS) levels for reliable message delivery

For example, the stock management department of a bank uses a publish/subscribe implementation of AMQP 1.0 with a broker. The broker receives all relevant market and transaction data and routes it to the correct endpoint. Users can place orders, see real-time market updates, and access other critical data on different applications and platforms. 

QoS levels and transaction messaging atomicity guarantee that all transactions will be successfully completed, robust security keeps threat actors out, and the broker handles message delivery at scale.

AMQP has its origins in the financial industry, but it also has widespread use in others like:

• Ecommerce and retail
• Healthcare
• Manufacturing
• Telecommunications
• Defense

It's used by major financial institutions, including Bank of America, JPMorgan, and Deutsche Börse, as well as tech giants like Microsoft, Cisco, Google, and VMware.

Before continuing, we should note that it may be more accurate to refer to AMQP as two separate protocols with the same name. There is AMQP 0.9.1 and AMQP 1.0. We will dig into their differences later, but AMQP 1.0 will be our primary focus.

How Does AMQP Work?

Let's examine how AMQP 1.0 enables messaging from connection initiation to termination. 

AMQP sets rules that applications must follow for effective communication. These rules are often flexible to allow for increased interoperability and customization, which means implementations can look different from one to the next.

At its core, AMQP lays out the following:

1. Transport layer: How nodes connect to exchange messages
2. Messaging layer: How messages are structured, managed, and delivered between nodes

AMQP Connections

AMQP connects over a transport-layer protocol. This is most often TCP with or without TLS, although WebSockets over TCP are also common.

After the TCP connection is established, an AMQP connection forms between two containers. Containers are the applications involved in the messaging process, like brokers and client applications. 

Connections can vary in structure and may comprise a client and a broker or two clients. 

Connections are full-duplex. They're also ephemeral, which means they won't retain their state when terminated.

AMQP and (if in use) SASL frames are sent over a connection in their specified order. 

A container starts a connection with another by sending connection packets, which determine the version of AMQP. Next, the containers set the frame size and channel number limits.

A connection consists of channels, which are unidirectional and ephemeral.

AMQP Sessions and Links

AMQP sessions tie two unidirectional channels together for bidirectional communication. Sessions control the number of frames that can be exchanged.

On top of the session, AMQP links attach at the termini of two nodes to send messages. One terminus is known as the source and plays the role of the sender; the other terminus is referred to as the target and acts as the receiver.  Links are unidirectional, recoverable, and named. Bidirectional exchange can be facilitated by attaching multiple links between the same nodes. Nodes are endpoints inside a container; multiple nodes can exist in the same container. In action, this process could look like a stock tracking program creating TCP and AMQP connections, establishing a session, and attaching links to a broker to transfer AMQP frames with message payloads containing data about a particular stock. The broker acts as the routing hub in a publish/subscribe format.

A bank app performs the same setup to receive the stock data. As the stock tracker updates, it sends messages to the broker that get routed to customers' investment screens to help them track real-time changes.

AMQP Frame Anatomy

Frames are divided into a required frame header, an optional extended header, and an optional frame body.

Frame headers are eight bytes. They're made up of three to four segments, which are:

• SIZE: This segment lists the total frame size.
• DOFF: A data offset that shows where the header ends and the body starts.
• TYPE: A type code that indicates whether the frame is AMQP or SASL.
• CHANNEL (AMQP type): This identifies the number of the channel to which the frame belongs.

The extended header's contents will vary by frame type, but it's often empty for AMQP frames and always ignored for SASL frames.

The frame body has a performative and a payload.

There are nine performatives in AMQP 1.0. They are:

Connection-level:

• Open: This opens the connection and is sent by both parties. These frames list the maximum frame size and the sender's capabilities.
• Close: This closes connections and is sent by both parties. These frames contain codes that explain the reason for termination.

Session-level:

• Begin: This is like the open performative but for establishing a session. The recipient assigns the session to an unused channel and informs the sender in its reply begin frame.
• End: When sessions are ended (excluding connection closure), one party sends an end frame with any relevant error information to the other party, which sends an end frame back.

Link-level:

• Attach: This attaches a link between two termini. This frame will contain information about the link's name, its handle (a unique ID for tracking state), role (sender or receiver), source node, and target node. It may contain optional information about an unsettled delivery state if it's reconnecting.
• Detach: This detaches a link from both nodes and is sent by both parties.
• Flow: This performative allocates credits for message-level flow control and manages the status updates of a link.
• Transfer: This is the performative that transfers messages from sender to receiver. It contains metadata like state and a delivery identifier for tracking.
• Disposition: The disposition performative is first sent by the message receiver to the sender to update the delivery status, and then the sender sends it to the receiver to confirm the final delivery state.

The payload contains the actual message data, application data, header, delivery annotations, and related information.

Message Structure

In AMQP 1.0, the message structure is composed of an immutable bare message and an annotated message that contains a bare message with mutable annotations. The OASIS specification differentiates them as the bare message being what's "supplied by the sender" and the annotated one being what's "seen at the receiver."

The bare message consists of three segments, which are referred to as the body. They are:

• Properties: An optional section containing information like message ID, sender ID, recipient node, and message content type.
• Application-properties: An optional and customizable section for adding application-relevant information, such as transaction and other business data, Internet of Things (IoT) device data, and other relevant metadata.
• Application-data: Mandatory section that contains the actual message. It can be binary data, such as images or files; a sequence of ordered elements, like arrays of lists; or a value, which can range from primitive data types (integers, floats, strings, booleans, and more) to structured values and nested structures like maps and arrays.

The sender is the only one who can set the bare message sections. Any intermediary nodes cannot alter them.

In addition to the bare message sections, annotated messages may also contain a header with details about the delivery, delivery annotations with delivery-related metadata, message annotations with message-related metadata, and immutable footers that might contain data that can only be properly calculated after the rest of the message is received.

Flow Control

Flow control is vital for AMQP to perform at scale.

There are three stages of flow control in AMQP 1.0:

• Connection-level: Open frames determine the maximum frame size. 
• Session-level: Sessions set the limit for the number of frames that can be exchanged.
• Link-level: Links control the number of messages that can be sent by allocating credits.

AMQP 0.9.1 vs AMQP 1.0

There are many differences between 0.9.1 and 1.0, but they boil down to 0.9.1 being more restricted yet simpler to implement and 1.0 being more flexible and complex.

Some of the ways that 0.9.1 differs include it:

• Only supporting broker-based architecture
• Being limited to TCP connections
• Having a more rigid message format
• Having fewer options for flow control

Despite 1.0 being the official standard per OASIS and ISO/IEC, 0.9.1 still had wider usage and support for many years. This gap has lessened, if not reversed, as even RabbitMQ, the largest 0.9.1 broker, added native support for 1.0 in 2024.

Popular AMQP Brokers and Clients

There are several brokers and clients on the market that can serve your AMQP needs. If you're not sure where to start, consider these three well-known options:

• RabbitMQ is an immensely popular broker for AMQP 0.9.1 and 1.0, the Message Queuing Telemetry Transport (MQTT) protocol, and more. It supports Python, Java, Spring Boot, Ruby, JavaScript, Elixir, Objective-C, .NET, and Swift.

• Apache Qpid is a collection of AMQP-centric tools, including Java and C++ brokers that are compatible with different versions of AMQP. Qpid JMS is a messaging API that combines the Jakarta Messaging Service (JMS) API with AMQP 1.0. Qpid Proton is an AMQP 1.0 messaging library with support for Java, C, C++, .NET, and Python.

• Red Hat AMQ Broker provides brokers and clients for AMQP 1.0 and other messaging protocols. The AMQ Broker is based on ActiveMQ Artemis, and Red Hat offers its own builds of various clients, including Rhea, Qpid JMS, and the Python and .NET versions of Qpid Proton. 

When Should You Use AMQP?

AMQP isn't right for every situation. Its complexity can make it difficult to learn and implement, and it has a larger overhead compared to some similar protocols. However, it's widely used in finance and tech due to its flexibility, reliability, and interoperability. You should use AMQP if you're looking for the following:

Flexible Routing and Messaging

One of AMQP's biggest strengths lies in its granular customization and flexibility. With it, you can do things like: 

• Set your own rules for how messages should be routed from one node to the next 
• Dedicate links to specific message types like high-value business transactions or critical market updates
• Filter out unimportant messages based on configurable criteria 
• Control the flow of data for better performance at scale

There are also practically limitless ways for nodes to be structured. You can set up peer-to-peer connections between two containers with one node each for simple messaging, publish/subscribe with brokers directing messages to thousands of endpoints, or more complex arrangements.

Reliable Message Delivery for Complex Messages

AMQP started in the financial sector, so it's built with reliable delivery in mind. Like MQTT, there are three message guarantee levels: fire-and-forget (also known as at most once), at least once, and exactly once. There is also strong atomic transactional message support. With AMQP, your transactions, no matter how complex, will get to where they need to be without the risk of only partially completing them.

Wide Interoperability

If you need to connect wildly different systems, you can do so with AMQP. It is language and platform agnostic, standardized, extensible, and deployable on-premises, in the cloud, or on edge networks. If your business needs cutting-edge hardware to communicate with legacy systems, AMQP makes it possible. You can even bridge AMQP with other protocols like MQTT and HTTP.

Best Practices for Using AMQP

Consider following these best practices as a roadmap for smoother navigation:

Secure Your Messages

Whether it's financial, medical, or internal company data, your AMQP messages will have valuable information that hackers will have their sights on. Employ TLS for encryption in transit and SASL for authentication.  You can also strengthen your defenses further with other security mechanisms and techniques at the application level, like end-to-end encryption and role-based access control.

Optimize Performance by Experimentation and Testing

AMQP's customizability comes at the expense of increased complexity. It may take some experimentation to optimize your implementation's performance. Run benchmarks and test different messaging patterns, node setups, message configurations, payload sizes, and other settings to determine what works best for your needs. For instance, you can receive messages significantly faster with persistence settings turned off (if not needed).

Read Broker and Client Documentation

AMQP 1.0 leaves a lot of the implementation specifics up to the user, which means there will be a lot of variation in how each broker and client adheres to the protocol. Consult product-specific documentation for recommendations and requirements for things like setup, configuration, and message routing. For example, Azure Service Bus, a Microsoft messaging broker, requires that TCP connections use TLS for AMQP, whereas TLS is only recommended for RabbitMQ.

Frequently Asked Questions