Skip to content

Outbox Publisher

Overview

The Outbox Publisher is the second half of the Transactional Outbox Pattern. It is a highly resilient, distributed background service responsible for polling the outbox database table, publishing the events to the message broker, and marking them as processed.

Key Features

  • Guaranteed Delivery: Continuously polls for new events to ensure they are eventually published.
  • Horizontally Scalable: You can run multiple instances of the publisher service to increase throughput.
  • Distributed Coordination: Leverages other Athomic resilience patterns to coordinate work between instances:
    • Sharding: To distribute the processing of different aggregate_keys across all active workers.
    • Leasing: To ensure only one worker is processing events for a specific aggregate_key at a time, guaranteeing order.
    • Backpressure: To temporarily stop polling for a key that is causing repeated publishing failures.
    • Backoff: To reduce polling frequency when no events are found.

How It Works

The OutboxPublisher is a BaseService managed by the application's LifecycleManager. In its main run loop, it performs the following steps: 1. Queries the database for all distinct aggregate_keys with pending events. 2. Uses the ShardingService to filter that list down to only the keys this specific instance is responsible for. 3. Uses the BackpressureManager to further filter out any keys that are temporarily throttled. 4. For each remaining key, it acquires a lease and then processes all pending events for that key in strict sequential order. 5. It also polls for and processes any unordered (non-aggregate) events.

API Reference

nala.athomic.integration.outbox.publishers.base_publisher.OutboxPublisherBase

Bases: BaseService

Base implementation for the Outbox Publisher (Poller) Service.

This service is a high-resilience component that polls the Outbox storage, applies sharding, backpressure, and leasing mechanisms to ensure atomic, ordered, and distributed processing of events. It inherits lifecycle management from BaseService.

__init__(settings=None, storage=None, router=None, lease_manager=None, sharding_manager=None, backpressure_manager=None, connection_manager=None)

Initializes the publisher, establishing all necessary resilience and data access dependencies, prioritizing explicit DI over factory creation.