WalletPe

1. Context & Problem Statement

In distributed financial architectures, updating an account balance in a relational database while simultaneously publishing an event notification to a message broker introduces the Dual-Write Problem:

If the database commits successfully but Kafka is unreachable due to a transient network partition, downstream notification, auditing, and ledger replication services never receive the event. Conversely, if the message is published before the transaction completes and the database rolls back, downstream systems process phantom transactions.

WalletPe is a personal engineering exploration designed to evaluate architectural solutions to this challenge, focusing on zero financial balance discrepancies and idempotent transaction processing.

2. Requirements & System Constraints

• Atomicity: Wallet state mutations and event publication must succeed or fail as a single unit.
• At-Least-Once Delivery: Events must never be dropped, even across broker reboots.
• Idempotency: Downstream wallet consumers must tolerate duplicate event deliveries without balance corruption.
• Auditability: Every balance change requires an immutable ledger entry.

3. Architecture & Service Interactions

WalletPe implements the Transactional Outbox Pattern alongside an asynchronous Kafka relay worker.

sequenceDiagram
    participant User as Client App
    participant WalletAPI as Wallet Service
    participant DB as PostgreSQL (ACID Boundary)
    participant Relay as Debezium / Outbox Poller
    participant Kafka as Apache Kafka Topic
    participant Ledger as Ledger & Audit Consumer

    User->>WalletAPI: POST /api/v1/wallets/transfer (Idempotency-Key: k7-991)
    activate WalletAPI
    WalletAPI->>DB: BEGIN TRANSACTION
    WalletAPI->>DB: Deduct sender balance & credit recipient
    WalletAPI->>DB: INSERT INTO ledger_entries (...)
    WalletAPI->>DB: INSERT INTO outbox_events (type: TRANSFER_COMPLETED)
    WalletAPI->>DB: COMMIT TRANSACTION
    WalletAPI-->>User: 200 OK (Transfer Queued)
    deactivate WalletAPI

    Relay->>DB: Poll uncommitted outbox rows
    Relay->>Kafka: Publish event (transfer-events topic)
    Relay->>DB: Mark outbox row as PUBLISHED

    Kafka->>Ledger: Consume TRANSFER_COMPLETED
    activate Ledger
    Ledger->>Ledger: Verify Idempotency-Key in Redis
    Ledger->>Ledger: Persist analytical ledger snapshot
    Ledger-->>Kafka: ACK commit offset
    deactivate Ledger

4. Implementation Decisions & Status

Implemented Functionality

• Dual-Write Elimination: Wallet mutations and outbox_events records are written in the exact same local PostgreSQL ACID transaction.
• Idempotent Consumer Pipeline: Consumers verify a distributed idempotency key stored in Redis with a 24-hour TTL before executing downstream updates.
• Ledger Invariant Checking: Balance integrity checks ensure total debits match total credits across all transactions.

Planned Functionality (Roadmap)

• Distributed Dead Letter Queue (DLQ) Auto-Replay: Automated retry scheduling for poison-pill messages.
• Multi-Currency Conversion Service: External rate-provider integration with circuit-breaker protection (Resilience4j).

5. Failure Handling & Reliability

• Kafka Broker Outages: If Kafka becomes unavailable, the core wallet transfer API remains operational because events are safely buffered in the PostgreSQL outbox table. Once Kafka recovers, the relay drains the backlog without data loss.
• Network Retries: Idempotency keys prevent duplicate payments if a client times out and re-attempts the exact same HTTP request.