Message Delivery Guarantees: At-Most-Once vs At-Least-Once vs Exactly-Once

Message delivery guarantees describe what can happen when producers, brokers, consumers, networks, or databases fail. A message may be lost, delivered multiple times, or processed only once under specific conditions. These behaviours are commonly described as at-most-once, at-least-once, and exactly-once delivery.

The names sound simple, but they are often misunderstood. A broker delivering a message once does not guarantee that the business operation executes once. A consumer may update a database and crash before acknowledging the message, causing redelivery. Reliable systems therefore consider the complete path from publication through business-state persistence, not only the broker configuration.

Table of Contents

• What Is a Delivery Guarantee?
• Delivery vs Processing Semantics
• At-Most-Once Delivery
  • How At-Most-Once Works
  • At-Most-Once Example
• At-Least-Once Delivery
  • How At-Least-Once Works
  • At-Least-Once Example
• Exactly-Once Delivery
  • Why Exactly-Once Is Difficult
  • Exactly-Once Within a Defined Boundary
• Acknowledgement Timing
• Consumer Crash Scenarios
• Idempotent Consumers
• Deduplication Strategies
• Database Transactions and Message Processing
• External API Side Effects
• Producer Delivery Guarantees
• Ordering, Retries, and Duplicates
• Choosing a Delivery Guarantee
• Real-World Payment Processing Example
• Common Mistakes
• Production Checklist
• Conclusion
• Key Takeaway
• More Articles to Read

What Is a Delivery Guarantee?

A delivery guarantee defines how a messaging system behaves when a message cannot move cleanly from producer to consumer. Failures may occur while publishing, storing, transmitting, processing, or acknowledging the message.

The three common guarantees are:

• At-most-once: a message is delivered zero or one time.
• At-least-once: a message is delivered one or more times.
• Exactly-once: the intended effect occurs one time within a clearly defined processing boundary.

The correct guarantee depends on business impact. Losing one telemetry sample may be acceptable. Charging a payment twice is not. Sending the same promotional email twice may be inconvenient, while issuing the same refund twice can create a direct financial loss.

Delivery vs Processing Semantics

Delivery and processing are different operations. A broker can deliver one message several times while the consumer applies its business effect only once. Conversely, a broker may deliver a message once, but the consumer can accidentally repeat an external side effect inside its own handler.

Consider this consumer:

def handle_payment_message(message: dict) -> None:
    payment_gateway.charge(
        customer_id=message["customer_id"],
        amount=message["amount"]
    )

    broker.acknowledge(message)

If the charge succeeds and the worker crashes before acknowledgement, the broker may deliver the message again. The broker provides at-least-once delivery, but the customer may be charged twice.

A safer consumer separates the delivery event from the business effect and protects the effect using a stable identifier.

def handle_payment_message(message: dict) -> None:
    payment_gateway.charge(
        customer_id=message["customer_id"],
        amount=message["amount"],
        idempotency_key=message["payment_id"]
    )

    broker.acknowledge(message)

The delivery guarantee describes broker behaviour. Processing semantics describe whether the final business state can be applied more than once.

At-Most-Once Delivery

At-most-once delivery prioritises avoiding duplicates. A message is processed no more than once, but it may be lost if a failure occurs.

How At-Most-Once Works

A common implementation acknowledges or removes the message before business processing begins.

1. The consumer receives the message.
2. The message is acknowledged immediately.
3. The consumer performs the business operation.

def consume_at_most_once(message: dict) -> None:
    # Remove the message before processing.
    broker.acknowledge(message)

    # A crash here loses the work permanently.
    process_message(message)

If the worker crashes after acknowledgement, the broker does not redeliver the message. Duplicate processing is avoided, but the operation may never finish.

Advantages

• Simple consumer implementation.
• No broker-level duplicate redelivery after acknowledgement.
• Low storage and retry overhead.
• Useful when fresh data is more valuable than complete historical data.

Disadvantages

• Messages can be lost during consumer failures.
• Temporary outages can create permanent gaps.
• Unsuitable for critical financial or state-changing operations.

When to Use

Use at-most-once delivery when occasional loss is acceptable and duplicate processing would be more harmful or unnecessary.

Real-World Use Cases

• High-frequency application metrics.
• Live location updates where a newer update quickly replaces an older one.
• Non-critical logging.
• Real-time interface refresh signals.

At-Most-Once Example

A vehicle may publish its location every few seconds. Processing one old location twice provides little value, and losing one update is usually acceptable because another update will arrive shortly.

def handle_vehicle_location(message: dict) -> None:
    broker.acknowledge(message)

    location_cache.set(
        vehicle_id=message["vehicle_id"],
        latitude=message["latitude"],
        longitude=message["longitude"],
        recorded_at=message["recorded_at"]
    )

If processing fails, the location may remain slightly outdated until the next message arrives. Retrying every historical update could increase backlog and cause stale locations to overwrite newer data.

At-Least-Once Delivery

At-least-once delivery prioritises avoiding message loss. The broker keeps a message until the consumer confirms successful processing. If acknowledgement does not arrive, the broker makes the message available again.

How At-Least-Once Works

The normal sequence is:

1. The consumer receives the message.
2. The broker temporarily hides or assigns the message.
3. The consumer performs the business operation.
4. The consumer acknowledges successful processing.
5. The broker removes the message or advances the consumer position.

def consume_at_least_once(message: dict) -> None:
    try:
        process_message(message)

        # Acknowledge only after processing succeeds.
        broker.acknowledge(message)
    except TemporaryError:
        # The message becomes available for another attempt.
        broker.retry(message)

If the worker crashes before acknowledgement, the broker cannot know whether processing finished. It therefore redelivers the message.

Advantages

• Protects against message loss after successful broker acceptance.
• Supports recovery from temporary consumer failures.
• Fits most background jobs and integration workflows.
• Works well when consumers are idempotent.

Disadvantages

• Duplicate delivery is expected.
• Consumers need idempotency or deduplication.
• Retries can change processing order.
• Poison messages require bounded retries and dead-letter handling.

When to Use

Use at-least-once delivery when losing a message is unacceptable and duplicate processing can be safely prevented or tolerated.

Real-World Use Cases

• Order processing.
• Inventory updates.
• Invoice generation.
• Search indexing.
• Email delivery.
• Data synchronisation between services.

At-Least-Once Example

A consumer creates an invoice and records the processed message ID in the same database transaction.

def handle_invoice_message(message: dict) -> None:
    message_id = message["message_id"]
    invoice_id = message["invoice_id"]

    with database.transaction():
        if database.processed_messages.exists(message_id):
            # A previous attempt completed successfully.
            return

        database.invoices.generate(invoice_id)

        database.processed_messages.insert({
            "message_id": message_id,
            "processed_at": current_utc_time()
        })

    broker.acknowledge(message)

If the consumer crashes after the database transaction commits but before acknowledgement, the message is delivered again. The second attempt sees the stored message ID and skips duplicate invoice generation.

Exactly-Once Delivery

Exactly-once is often interpreted as a guarantee that every message crosses the network one time and is processed one time under all failures. Distributed systems cannot generally provide that unlimited guarantee across independent brokers, databases, APIs, and external services.

A practical definition is narrower:

The business effect occurs exactly once within a specified transactional or idempotent boundary, even when message delivery is repeated.

Why Exactly-Once Is Difficult

After processing a message, a consumer must communicate success to the broker. A failure can occur between those two operations.

1. The consumer receives a message.
2. The consumer updates the database.
3. The consumer sends an acknowledgement.

Consider a failure after step two:

def process_order(message: dict) -> None:
    database.mark_order_paid(message["order_id"])

    # The process crashes here.
    broker.acknowledge(message)

The database contains the completed update, but the broker did not receive the acknowledgement. Redelivery is necessary to avoid possible loss. The consumer must therefore recognise that the business operation already completed.

The opposite order is also unsafe:

def process_order(message: dict) -> None:
    broker.acknowledge(message)

    # The process crashes here and the update is lost.
    database.mark_order_paid(message["order_id"])

Without a transaction spanning both the broker and database, no acknowledgement order eliminates both loss and duplication.

Exactly-Once Within a Defined Boundary

Exactly-once semantics can be implemented inside a controlled boundary using mechanisms such as:

• A broker transaction that includes consuming and producing records.
• A database transaction containing the business update and deduplication record.
• An atomic conditional update based on a business identifier.
• An external API supporting idempotency keys.
• A transactional inbox or outbox pattern.

Advantages

• Prevents repeated business effects within the protected boundary.
• Supports critical financial and state-changing workflows.
• Simplifies downstream reasoning when implemented correctly.

Disadvantages

• Requires additional storage, transactions, or broker-specific features.
• May reduce throughput and increase latency.
• Does not automatically cover unrelated external systems.
• Can create a false sense of safety when the boundary is not documented.

When to Use

Use exactly-once effects when duplicate business operations would cause serious damage and the processing boundary can be protected transactionally or through idempotency.

Real-World Use Cases

• Payment capture.
• Refund creation.
• Ledger entry insertion.
• Inventory decrement.
• Issuing loyalty rewards.

Example

BEGIN;

-- Insert succeeds only once for this operation.
INSERT INTO payment_operations (
    operation_id,
    payment_id,
    operation_type,
    status
)
VALUES (
    'op-742',
    'pay-901',
    'capture',
    'completed'
)
ON CONFLICT (operation_id) DO NOTHING;

-- Apply the business update only when the operation was inserted.
UPDATE payments
SET status = 'captured'
WHERE payment_id = 'pay-901'
  AND status = 'authorised'
  AND EXISTS (
      SELECT 1
      FROM payment_operations
      WHERE operation_id = 'op-742'
  );

COMMIT;

The broker may still deliver the message several times. The database constraint ensures that operation op-742 is recorded only once.

Acknowledgement Timing

Acknowledgement timing determines whether a consumer prefers loss prevention or duplicate prevention.

For most important workloads, acknowledgement should happen only after durable business state has been saved.

def handle_message(message: dict) -> None:
    try:
        save_business_result(message)
    except TemporaryError:
        # Do not acknowledge incomplete work.
        return

    broker.acknowledge(message)

Acknowledgements should not be delayed indefinitely. If a task takes longer than the broker visibility timeout or lease, the broker may redeliver the message while the first worker is still processing it.

def process_large_export(message: dict) -> None:
    lease = broker.start_lease(
        message=message,
        timeout_seconds=60
    )

    try:
        for batch in export_batches(message["export_id"]):
            write_export_batch(batch)

            # Extend ownership while useful work continues.
            lease.extend(timeout_seconds=60)

        broker.acknowledge(message)
    finally:
        lease.close()

Consumer Crash Scenarios

Understanding crash timing is essential when designing message handlers.

Partial processing is especially dangerous. A handler may update several systems before failing.

def unsafe_order_handler(message: dict) -> None:
    inventory.reserve(message["order_id"])
    payment.capture(message["payment_id"])

    # Failure here leaves inventory and payment completed
    # while the order status remains unchanged.
    database.mark_order_confirmed(message["order_id"])

    broker.acknowledge(message)

Reliable workflows minimise non-transactional side effects, make each step idempotent, or use a state machine that records progress explicitly.

Idempotent Consumers

An idempotent operation can be executed multiple times with the same input without changing the final result after the first successful execution.

Setting an order status to confirmed is naturally more idempotent than incrementing a counter.

-- Naturally idempotent state assignment.
UPDATE orders
SET status = 'confirmed'
WHERE order_id = 'ORD-10482';

-- Not naturally idempotent.
UPDATE accounts
SET reward_points = reward_points + 100
WHERE account_id = 'ACC-702';

The second update adds points every time the message is processed. It requires an operation identifier or uniqueness constraint.

BEGIN;

INSERT INTO reward_transactions (
    transaction_id,
    account_id,
    points
)
VALUES (
    'reward-ORD-10482',
    'ACC-702',
    100
)
ON CONFLICT (transaction_id) DO NOTHING;

UPDATE accounts
SET reward_points = reward_points + 100
WHERE account_id = 'ACC-702'
  AND EXISTS (
      SELECT 1
      FROM reward_transactions
      WHERE transaction_id = 'reward-ORD-10482'
  );

COMMIT;

Idempotency should be based on a stable business operation, not a random identifier created by each retry. All attempts for the same logical payment capture must use the same idempotency key.

Deduplication Strategies

Deduplication detects that a message or business operation has already been processed. Several strategies are available.

Processed Message Table

The consumer stores every completed message ID.

def process_with_inbox(message: dict) -> None:
    with database.transaction():
        inserted = database.execute(
            """
            INSERT INTO processed_messages (
                consumer_name,
                message_id,
                processed_at
            )
            VALUES (%s, %s, NOW())
            ON CONFLICT DO NOTHING
            """,
            ["billing-consumer", message["message_id"]]
        )

        if inserted.row_count == 0:
            return

        apply_business_update(message)

Advantages

• Simple and reliable when stored with the business transaction.
• Supports tracing and audit investigation.
• Works with any broker.

Disadvantages

• The table grows continuously.
• Retention and cleanup policies are required.
• Every message adds a database write.

Business Key Constraint

A unique business operation ID prevents duplicate effects.

CREATE UNIQUE INDEX unique_refund_operation
ON refunds (refund_operation_id);

A repeated insert fails or becomes a no-op.

Advantages

• Protects the actual business invariant.
• Does not require permanent storage of every broker message ID.
• Remains effective when the same operation arrives through different messages.

Disadvantages

• Requires a stable operation identifier.
• May need custom logic for every business action.

Aggregate Version Check

Consumers reject events older than the latest processed entity version.

def apply_customer_event(event: dict) -> None:
    customer_id = event["data"]["customer_id"]
    version = event["data"]["customer_version"]

    customer = local_view.get(customer_id)

    if customer and version <= customer.version:
        return

    local_view.update(
        customer_id=customer_id,
        version=version,
        data=event["data"]
    )

Advantages

• Handles both duplicates and stale events.
• Useful for event-carried state transfer.

Disadvantages

• Requires producer-managed sequence numbers.
• Missing versions may block later processing if strict sequencing is required.

Time-Limited Deduplication Cache

The consumer stores recently processed message IDs in a cache with an expiration time.

def already_processed(message_id: str) -> bool:
    inserted = redis.set(
        name=f"processed:{message_id}",
        value="1",
        nx=True,
        ex=86400
    )

    return not inserted

Advantages

• Fast and suitable for high-throughput workloads.
• Automatically removes old records.

Disadvantages

• Duplicates arriving after expiration are processed again.
• Cache loss may remove deduplication history.
• Cache insertion and business processing are usually not atomic.

Database Transactions and Message Processing

When all business effects occur in one database, a transaction can combine the deduplication record and state update.

def handle_inventory_message(message: dict) -> None:
    with database.transaction():
        inserted = database.processed_messages.insert_once(
            consumer="inventory-consumer",
            message_id=message["message_id"]
        )

        if not inserted:
            return

        database.inventory.decrement(
            product_id=message["product_id"],
            quantity=message["quantity"]
        )

        database.orders.mark_inventory_reserved(
            order_id=message["order_id"]
        )

    broker.acknowledge(message)

If the transaction fails, neither the message record nor the inventory update is committed. If it succeeds but acknowledgement fails, redelivery sees the message record and skips the operation.

This protects operations inside one database. It does not automatically make an external API call transactional with the database.

def unsafe_mixed_transaction(message: dict) -> None:
    with database.transaction():
        database.mark_payment_pending(message["payment_id"])

        # This network call cannot usually participate
        # in the local database transaction.
        payment_gateway.capture(message["payment_id"])

        database.mark_payment_captured(message["payment_id"])

If the external capture succeeds but the database transaction rolls back, the database and payment provider disagree. The workflow needs an idempotency key, reconciliation process, or explicit state machine.

External API Side Effects

External operations such as charging cards, sending refunds, creating shipments, or sending emails cannot usually participate in a local transaction.

The preferred protection is a stable idempotency key supported by the external provider.

def capture_payment(message: dict) -> None:
    operation_id = f"capture:{message['payment_id']}"

    result = payment_gateway.capture(
        payment_id=message["payment_id"],
        amount=message["amount"],
        idempotency_key=operation_id
    )

    database.record_capture(
        operation_id=operation_id,
        provider_reference=result.reference
    )

Every retry sends the same key. The provider returns the original result instead of applying the charge again.

When the external system does not support idempotency, alternatives include:

• Querying the external system before repeating the operation.
• Using a consumer-side operation state machine.
• Serialising operations for one business entity.
• Running periodic reconciliation against provider records.
• Routing uncertain results to manual review.

Sending an email is often not perfectly deduplicatable. The email provider may accept the request while the network response is lost. A retry can send a second email. The system must decide whether occasional duplicates are acceptable or whether an intermediate email-delivery record should limit repeated sends.

Producer Delivery Guarantees

Delivery guarantees also apply when a producer publishes a message. The producer may not know whether the broker accepted the message if the connection fails while waiting for confirmation.

def publish_order_event(event: dict) -> None:
    try:
        broker.publish(
            topic="order-events",
            event=event,
            require_confirmation=True
        )
    except BrokerTimeout:
        # The broker may or may not have accepted the event.
        retry_publication(event)

Retrying publication can create duplicate messages. Stable event IDs allow consumers or the broker to recognise repeated publication attempts.

Another failure occurs when an application saves business data and publishes an event as two separate operations.

def create_order(order_data: dict) -> None:
    order = database.insert_order(order_data)

    # A crash here leaves an order without an event.
    broker.publish(
        topic="order-events",
        event={
            "event_type": "order.created",
            "order_id": order.id
        }
    )

The transactional outbox pattern solves this problem by storing the business record and outgoing event in the same database transaction. A separate publisher later transfers the event to the broker.

def create_order(order_data: dict) -> None:
    with database.transaction():
        order = database.insert_order(order_data)

        database.outbox.insert({
            "event_id": generate_event_id(),
            "event_type": "order.created",
            "aggregate_id": order.id,
            "payload": {
                "order_id": order.id
            }
        })

The outbox publisher may publish an event more than once if it crashes before marking the record as sent. Consumers must still handle duplicates.

Ordering, Retries, and Duplicates

Retries can change processing order. An older event may fail, while a newer event succeeds immediately.

Consider these events:

1. shipment.status_changed to in_transit, version 4.
2. shipment.status_changed to delivered, version 5.

If version 4 fails temporarily and version 5 succeeds, retrying version 4 later could incorrectly move the shipment back to in_transit.

def apply_shipment_status(event: dict) -> None:
    shipment = database.get_shipment(
        event["data"]["shipment_id"]
    )

    incoming_version = event["data"]["version"]

    if incoming_version <= shipment.last_event_version:
        return

    database.update_shipment(
        shipment_id=shipment.id,
        status=event["data"]["status"],
        last_event_version=incoming_version
    )

Partitioning related events by entity ID can preserve broker order, but consumer version checks remain useful because replay, retry queues, and manual recovery may still reintroduce older events.

Choosing a Delivery Guarantee

The choice should be based on the cost of loss, the cost of duplication, and the complexity required to prevent each outcome.

At-least-once delivery with idempotent consumers is the most common practical choice. It avoids silent message loss while keeping the broker and consumer model manageable.

Real-World Payment Processing Example

Consider a worker that captures an authorised payment after receiving a payment.capture_requested command.

The unsafe implementation calls the provider and acknowledges afterward.

def unsafe_capture(message: dict) -> None:
    payment_gateway.capture(
        payment_id=message["payment_id"],
        amount=message["amount"]
    )

    broker.acknowledge(message)

If the capture succeeds and acknowledgement fails, redelivery may charge the customer again.

A safer workflow uses a stable operation ID and explicit processing states.

def capture_payment(message: dict) -> None:
    operation_id = message["operation_id"]

    operation = database.payment_operations.get(
        operation_id
    )

    if operation and operation.status == "completed":
        broker.acknowledge(message)
        return

    database.payment_operations.create_if_missing({
        "operation_id": operation_id,
        "payment_id": message["payment_id"],
        "status": "processing"
    })

    result = payment_gateway.capture(
        payment_id=message["payment_id"],
        amount=message["amount"],
        idempotency_key=operation_id
    )

    with database.transaction():
        database.payments.mark_captured(
            payment_id=message["payment_id"],
            provider_reference=result.reference
        )

        database.payment_operations.mark_completed(
            operation_id=operation_id
        )

        database.outbox.insert({
            "event_id": generate_event_id(),
            "event_type": "payment.captured",
            "aggregate_id": message["payment_id"],
            "payload": {
                "payment_id": message["payment_id"],
                "operation_id": operation_id,
                "provider_reference": result.reference
            }
        })

    broker.acknowledge(message)

This workflow protects against several failures:

• A repeated message uses the same provider idempotency key.
• A completed operation is not executed again.
• The payment update and outgoing event are stored in one transaction.
• A crash before acknowledgement causes safe redelivery.

An uncertain provider result still requires reconciliation. For example, the provider may complete the capture but become unavailable before returning a response. The worker should query the operation using the same idempotency key before creating a new capture.

def recover_uncertain_capture(operation_id: str) -> None:
    provider_operation = payment_gateway.find_by_idempotency_key(
        operation_id
    )

    if provider_operation.status == "completed":
        database.payment_operations.mark_completed(
            operation_id=operation_id,
            provider_reference=provider_operation.reference
        )
    else:
        schedule_retry(operation_id)

Common Mistakes

Production Checklist

• Define acceptable failure behaviour: document whether loss, duplication, or delay is more harmful.
• Separate delivery from processing: identify what the broker guarantees and what the application must guarantee.
• Acknowledge after durable completion: avoid losing work during worker crashes.
• Expect duplicate delivery: design every important at-least-once consumer accordingly.
• Use stable message IDs: identify repeated deliveries of the same message.
• Use stable operation IDs: protect the actual business action across multiple messages or retries.
• Add database uniqueness constraints: enforce critical business invariants at the storage layer.
• Store deduplication and business updates atomically: prevent one from committing without the other.
• Use provider idempotency keys: protect payment, refund, shipping, and other external side effects.
• Define the exactly-once boundary: state which database, topic, or external operation is covered.
• Configure visibility timeouts: allow enough time for normal processing.
• Extend leases for long tasks: prevent concurrent processing of an active message.
• Use bounded retries: stop retrying permanent failures indefinitely.
• Configure dead-letter queues: isolate messages requiring investigation.
• Preserve event versions: reject stale updates after retries or replay.
• Plan deduplication retention: retain records for the maximum realistic retry and replay period.
• Monitor duplicate rates: unexpected growth may indicate acknowledgement or timeout problems.
• Monitor message age: identify consumers that are repeatedly retrying or falling behind.
• Test consumer crashes: terminate workers before, during, and after business-state persistence.
• Run reconciliation: compare critical internal records with external payment, shipping, or accounting systems.

Conclusion

At-most-once delivery avoids duplicate redelivery but may lose messages. At-least-once delivery avoids silent loss by retrying unacknowledged messages, but duplicate delivery becomes normal. Exactly-once processing is not a universal network guarantee; it is usually an exactly-once business effect implemented within a specific transaction or idempotency boundary.

Most production messaging systems use at-least-once delivery with idempotent consumers. This model accepts that brokers cannot always determine whether a failed consumer completed its work. Stable message IDs, business-operation IDs, uniqueness constraints, database transactions, provider idempotency keys, and reconciliation make repeated delivery safe.

The delivery guarantee should be chosen according to business impact. Occasional telemetry loss may be harmless, while duplicate payment capture or ledger insertion requires stronger safeguards. Clear processing boundaries are more valuable than broad claims of exactly-once delivery.

Key Takeaway

At-most-once may lose work, at-least-once may repeat work, and practical exactly-once processing is achieved by making repeated delivery produce only one durable business effect.

More Articles to Read

• Message Queues Explained: Producers, Consumers, and Brokers
• Event-Driven Architecture in Distributed Systems
• Designing Reliable Background Workers
• Dead-Letter Queues, Retries, and Poison Messages
• Transactional Outbox Pattern for Reliable Messaging
• Message Queue Best Practices for Production Systems