Search
6 results for “event-driven”
Search results
Message Queues & Event-Driven Architecture
Why a visibility timeout, not a delete, is what actually protects a message from being processed twice, and why "at-least-once delivery" means your consumer has to handle duplicates even when everything is configured correctly.
What is a visibility timeout, and what problem does it solve that simply having multiple consumers poll a queue would create?
When a consumer receives a message, the queue doesn't delete it immediately, it starts a visibility timeout during which that message is hidden from other consumers, so a second consumer polling the same queue won't also pick up and process the same message concurrently. The consumer is expected to finish processing and explicitly delete the message before the timeout expires; if it does, the message never reappears. Without this mechanism, multiple consumers competing for messages would routinely double-process the same one, exactly the race condition visibility timeouts exist to prevent.
A consumer receives a message, starts processing, but crashes before deleting it. What happens to that message, and why is this actually the desired behavior?
Once the visibility timeout expires without the message being deleted, it automatically becomes visible again in the queue and can be picked up by the same or a different consumer for another processing attempt. This is deliberate, not a bug: the alternative (a crashed consumer's message vanishing permanently) would silently lose data, whereas reappearing after timeout guarantees eventual processing at the cost of a possible duplicate attempt, which is exactly the trade-off "at-least-once delivery" describes. Setting the visibility timeout too short causes premature, unnecessary reprocessing of messages still legitimately being worked on; too long delays legitimate retries after a real crash.
What does "at-least-once delivery" actually guarantee, and what does it not guarantee, and why does that mean a consumer must be idempotent?
At-least-once delivery guarantees a message will eventually be delivered and processed at least one time, it does not guarantee exactly one delivery, the same message can legitimately be delivered and processed more than once, for example if a consumer's deletion request is lost after it already finished processing, or a visibility timeout expires just as processing completes. Because duplicate delivery is a normal, expected outcome of this model rather than a rare edge case, a consumer has to be written so that processing the same message twice produces the same end result as processing it once (an idempotent operation), rather than assuming the queue itself will prevent duplicates.
Kafka vs RabbitMQ
How Kafka and RabbitMQ differ in messaging model, routing, replay, and throughput, and which fits event streaming versus task queues.
When would you choose AWS Lambda over EC2 for a workload?
Lambda fits event-driven, short-lived work, an API request, a file landing in S3, a queue message, where you want to pay only for actual invocation time and never manage a server at all; a function can run for at most 15 minutes and has no state between invocations. EC2 fits workloads that need to run continuously, need OS-level control, exceed Lambda's execution-time or memory ceiling, or need to retain in-process state between requests. The decision is about execution model and constraints, not maturity, a long-running stateful service is a worse fit for Lambda regardless of how "serverless-first" a team wants to be.