Message Queues & Event-Driven Architecture

Overview

A message queue decouples producers from consumers, but the mechanism that actually prevents two consumers from processing the same message concurrently is the visibility timeout: once a consumer receives a message, it becomes temporarily invisible to other consumers, and only an explicit delete before that timeout expires removes it for good. If the timeout expires first, whether from a slow consumer, a crash, or a lost delete request, the message becomes visible again and gets redelivered, which is exactly what "at-least-once delivery" means: a message is guaranteed to be delivered and processed eventually, but not guaranteed to be delivered exactly once. That guarantee shape is deliberate, not a flaw, and it means any consumer built on this model has to be idempotent, processing the same message twice has to produce the same outcome as processing it once, because duplicate delivery is a normal, expected part of the model, not a rare failure case.

Quick Reference

Syntax

messages = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=1)
# process the message...
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=messages['Messages'][0]['ReceiptHandle'])

Examples

# Idempotent processing - safe even if this message is delivered
# and processed more than once, which at-least-once delivery allows.
# claim_processing() atomically inserts order_id and returns False if
# it already exists, so two concurrent redeliveries can't both pass
# the check before either one claims it.
def process_order(message):
    order_id = message['order_id']
    if not claim_processing(order_id):
        return
    # idempotency_key ties the charge itself to this order, so even a
    # retry of charge_customer after a crash won't double-charge.
    charge_customer(order_id, idempotency_key=order_id)

Visual Diagram

Common Mistakes

• Assuming a message queue guarantees exactly-once processing, when the standard model (and the one most queues offer) is at-least-once, duplicates are expected.
• Setting the visibility timeout far shorter than actual processing time, causing legitimate in-progress messages to be redelivered and double-processed unnecessarily.
• Writing consumers that aren't idempotent, then being surprised by real-world duplicate side effects (double charges, duplicate emails) under normal operation, not just failure scenarios.
• Not using a dead-letter queue for messages that repeatedly fail processing, letting a poison message cycle indefinitely instead of being captured for investigation.

Performance

• A visibility timeout set too long delays legitimate retries after a real consumer crash, directly increasing end-to-end processing latency for affected messages.
• A visibility timeout set too short causes unnecessary redelivery and duplicate processing work under normal, still-in-progress conditions, wasted compute, not a correctness issue if the consumer is properly idempotent.

Best Practices

• Set the visibility timeout to comfortably exceed expected processing time, and extend it programmatically for genuinely long-running tasks rather than picking one static value blindly.
• Design every consumer to be idempotent, since at-least-once delivery makes duplicate processing a normal, expected event, not an edge case.
• Route messages that fail processing repeatedly to a dead-letter queue instead of letting them cycle indefinitely in the main queue.
• Delete messages promptly after successful processing to keep the in-flight message count low and avoid unnecessary redelivery windows.