Transactional inbox/outbox messaging

Transactional inbox/outbox messaging ties messaging to your database transaction. An inbound message is processed once. An outbound side effect is recorded in the same transaction as the business write that caused it, and a relay delivers it after that transaction commits. A message is never acted on twice, and a rolled-back write leaves no side effect behind.

• Atomic with your transaction. The inbox dedup insert and the outbox entry insert each take part in the caller’s own transaction. If the business write rolls back, the recorded row rolls back with it.
• Exactly-once effects via the inbox. Deduplication is strict, by message id and source. A redelivered message is reported as a duplicate and its work is not invoked again.
• At-least-once delivery via the relay. The relay reads only committed rows, retries with backoff, and dead-letters after the attempt limit. It preserves ordering within an aggregate while different aggregates proceed concurrently.
• Pluggable destinations. Relay to a stable service target, a durable delayed job, or a Kafka topic, or contribute your own destination handler through the SPI.
• Near-immediate delivery. With the default LISTEN/NOTIFY strategy a newly committed row is claimed almost immediately. Periodic polling runs underneath as a safety net.
• Self-maintaining tables. Retention-bounded cleanup and stale-lease recovery run as cluster-singleton cron jobs instead of per-node timers.
• Observable publish attempts. An observer is told the outcome of every relay attempt, for every destination type, after the entry’s new state is recorded.

Reach for inbox/outbox messaging when a business write must deduplicate an inbound message, or must record an outbound side effect (a service call, a delayed job, a Kafka publish) that commits atomically with the write and takes effect only afterward. To coordinate several steps over time, use a durable workflow. For a recurring or one-shot background task with no transactional pairing, use a job. A job handler can still use inbox/outbox internally.

Record once, deliver reliably

One transaction carries both sides of the pattern: deduplicate the inbound message, apply the business write, and record the outbound confirmation. They commit or roll back together, and the relay delivers the confirmation only after commit. From the vertique-inbox-outbox-core module reference’s Kafka-consumer example:

pool.withTransaction(tx ->
    inboxService.processOnce(messageId, "payments", tx, () ->
        orderRepository.markPaid(orderId, tx)
            .compose(v -> outboxService.publish(tx, OutboxEntry.builder()
                .aggregateType("Order")
                .aggregateId(orderId)
                .eventType("send-order-confirmation")
                .destinationType(DestinationType.SERVICE)
                .destination("integration.notification.send-order-confirmation")
                .payload(orderConfirmation)
                .build()))
    )
).onSuccess(result -> {
    if (result.isDuplicate()) {
        log.info("Duplicate message {} skipped", messageId);
    }
});

Delivery semantics

The inbox and the outbox guarantee different things:

• Recording is atomic with the business transaction, on both sides. The inbox dedup insert and the outbox entry insert each take part in the caller’s own transaction. If the surrounding business write rolls back, the recorded row rolls back with it.
• The outbox insert is not deduplicated. A business retry that runs in a new transaction and records the same outbound side effect again records a second outbox row. The outbox does not collapse two calls that the application considers the same event.
• The relay delivers at least once. It reads only committed rows and hands each one to its destination. A retryable delivery failure returns the row to the pending state with an incremented attempt and a backoff-computed retry time, and the relay redelivers it on a later cycle.
• Inbox dedup is the framework’s one exactly-once effect seam. The inbox deduplicates strictly by message id and source. When the same pair arrives again, it is reported as a duplicate and the work supplier passed to it is not invoked again. The outbox side is at-least-once and not deduplicated, so an application that calls the outbox publish API more than once for one logical event gets more than one delivery. The destination side should be idempotent on its own terms. Destination-side idempotency below lists what each adapter guarantees.

Entry headers

An outbox entry carries text headers, set with headers(...) on the OutboxEntry builder. The Kafka adapter puts them on the published record. OutboxService.publish rejects an entry whose header has a null key, a null value, or a key that starts with the reserved vertique- prefix. The returned future fails with IllegalArgumentException and no row is inserted, so the surrounding transaction can still roll back. The message names the key and never the value.

Observe publish attempts

OutboxPublishObserver reports the result of each relay attempt. The relay calls it once per claimed entry per attempt, for every destination type, after the repository call that records the entry’s new state has settled. Use it for publish metrics, audit records, and alerts on dead-lettered entries. From the vertique-inbox-outbox-core module reference:

public interface OutboxPublishObserver extends OrderedExtension {
    default void onPublishCompleted(OutboxPublishCompletedEvent event, OutboxEnvelope envelope) {}
}

@Provides @IntoSet
static OutboxPublishObserver publishMetrics(PublishMetricsObserver observer) {
    return observer;
}

The event holds facts only: the entry id, destination type and destination, the attempt number and the attempt limit, the elapsed time, an outcome, and a disposition. It has no payload, headers, or failure message, so it is safe to log.

The outcome (SUCCESS, RETRYABLE_FAILURE, PERMANENT_FAILURE, UNRESOLVABLE) says how the attempt ended. The disposition (PUBLISHED, RETRY_SCHEDULED, DEAD_LETTERED, DEFERRED) says what the relay did with the entry. The two differ when a retryable failure uses up the last attempt, which is reported as RETRYABLE_FAILURE with DEAD_LETTERED. An entry whose destination type has no registered handler is reported as UNRESOLVABLE with DEFERRED.

dispositionRecorded is true only when the repository confirmed the state change.

An implementation works under these rules:

• It observes. It cannot change the entry’s state transition or the relay’s poll loop.
• It runs on the relay’s event-loop context and must not block. Several relay instances may call one observer concurrently.
• The envelope is the same instance the destination handler receives. Do not modify it, log it (its toString() prints the payload and headers), or keep it after the callback returns.
• An exception it throws is logged and swallowed. Later observers still run and the entry’s outcome does not change. An assertion that fails inside an observer therefore does not fail the test that triggered the publish.

The relay does not call the observer for an attempt that never settles: a handler future that never completes, or an attempt still in flight when the relay stops.

Claim scopes and per-aggregate ordering

Each OutboxDestinationHandler declares a claim scope that decides which of its own outbox rows a relay node may claim:

• ClaimScope.all() when any relay node can deliver every row of its destination type. Kafka topics are an example: any node with the Kafka adapter installed can publish to any topic.
• ClaimScope.destinations(supplier) when this node can claim only rows whose destination value is currently in the supplier’s live set. Service targets and delayed-job handlers registered on this node are examples.

The relay derives its full claim eligibility (RelayCapabilities) from the registered handler set at startup. A destination type with no registered scope claims nothing.

Within one aggregate, the relay preserves order. While one outbox entry for an aggregate id is being delivered, later entries for the same aggregate are excluded from claim until it finishes. Different aggregates may proceed concurrently. One consequence: a row of an unregistered destination type at the head of its aggregate’s queue blocks younger, otherwise claimable siblings of the same aggregate until a handler for that type is registered. This is the ordering guarantee at work.

Relay delivery strategies

The PostgreSQL relay supports two claim strategies, configured with inboxOutbox.relay.strategy:

• LISTEN_NOTIFY (the default). The relay listens for a PostgreSQL notification, so a newly inserted outbox row is claimed almost immediately. Periodic polling still runs underneath as a safety net for a missed notification or a dropped connection.
• POLLING. The relay claims on a fixed-interval timer only.

When LISTEN_NOTIFY is configured but the notification channel becomes unavailable, the relay falls back to polling-only mode on its own.

Retention

Published and dead-lettered outbox rows are removed after a retention period. inboxOutbox.cleanup.publishedRetentionDays (default 7) and inboxOutbox.cleanup.deadLetterRetentionDays (default 30) bound how long PUBLISHED and DEAD_LETTER rows survive, and inboxOutbox.cleanup.inboxRetentionDays (default 30) bounds how long processed inbox rows are kept.

Cleanup and stale-lease recovery run as cluster-singleton cron jobs: cron.jobs.outbox-cleanup (default every 6 hours) and cron.jobs.outbox-stale-lease-recovery (default every 30 seconds). For this maintenance to run, the application must install vertique-job-cron’s persistent (CronPersistenceModule) configuration. The in-memory one is not enough. Installing TransactionalMessagingPostgresqlModule without it fails at Dagger code generation.

Destination-side idempotency

Each destination adapter documents its own delivery guarantee. None of them is exactly-once on its own:

• The vertique-inbox-outbox-services adapter delivers to a stable service target over an event bus request/reply. The handoff is at-least-once, so the service should be idempotent or deduplicate inbound calls itself with the inbox.
• The vertique-inbox-outbox-delayed-job adapter enqueues a durable delayed job once the business transaction has committed. The job then runs under the delayed-job module’s own retry and dead-letter handling (see Scheduled and deferred jobs).
• The vertique-inbox-outbox-kafka adapter publishes to a topic, keyed by the aggregate id when one is present so that records for the same aggregate route to the same partition. Delivery is at-least-once. Consumers that need an exactly-once effect should deduplicate with the inbox on their own side.

The framework’s exactly-once guarantee is the inbox dedup seam described above. Expect at-least-once delivery from every adapter.

Composing with adapters and PostgreSQL

vertique-inbox-outbox-core has no compile dependency on services, job-delayed, or Kafka. Those arrive only through the destination adapter an application chooses:

Add	For
vertique-inbox-outbox-core	InboxService, OutboxService, and the OutboxDestinationHandler contract
vertique-inbox-outbox-postgresql	Persistence for both write APIs, plus the relay itself
vertique-inbox-outbox-services	Relaying to a stable service target over the event bus
vertique-inbox-outbox-delayed-job	Relaying to a durable delayed job
vertique-inbox-outbox-kafka	Relaying to a Kafka topic

Install vertique-inbox-outbox-core and vertique-inbox-outbox-postgresql together, then add the destination adapter or adapters the application relays to. There is no vertique-starter-inbox-outbox. An application composes these artifacts directly into its own Dagger component.

vertique-inbox-outbox-postgresql wires the database connection layer directly, without a starter. It depends on vertique-db-postgresql, and the application still installs both DbPostgresqlModule and DbFlywayModule in its own Dagger component. The adapter does not run Flyway migration for the application, but it ships its migration scripts at classpath:db/migration/inbox-outbox and an InboxOutboxMigrationsModule that declares them as a migration named inbox-outbox. With the opt-in named mode (flyway.instances), that declaration gives the inbox and outbox tables their own Flyway history, and an optional db.schemas.inbox-outbox setting places them in their own schema for both the migration and the repository’s SQL. In the default single-history mode, flyway.locations must instead list the inbox/outbox path alongside the application’s own migration location for the scripts to run. An application that already uses the PostgreSQL starter for its own repositories (see Persistence) adds the inbox/outbox PostgreSQL adapter alongside it. The module’s own reference documents the wiring. See Durable workflows for the same pattern with the workflow and job adapters.

Learn more

The module references below document the constructs, SQL, configuration, and extension points in full.

• vertique-inbox-outbox-core module reference: InboxService, OutboxService, and the destination-handler contract.
• vertique-inbox-outbox-postgresql module reference: persistence, claim eligibility, and the relay.
• vertique-inbox-outbox-services module reference: the service-target destination adapter.
• vertique-inbox-outbox-delayed-job module reference: the delayed-job destination adapter.
• vertique-inbox-outbox-kafka module reference: the Kafka destination adapter.
• Durable workflows
• Scheduled and deferred jobs
• Persistence
• Application model
• What is Vertique?

Continue reading

• Previous: Messaging with Kafka
• Next: Jobs and scheduling