Data Flow Architecture

Overview

This document describes the flow of data through the BankLingo Core Banking System. It covers transaction processing flows, approval workflows, event-driven patterns, message queue interactions, process engine execution, and data synchronization mechanisms.

High-Level Data Flow

Transaction Processing Flow

1. Deposit Transaction Flow

Key Data Points:

Step	Data Written	Data Read	Cache Updated
1. Create PENDING	Transaction record, AccountHold	Account balance	Transaction state
2. Approval	Transaction update, Account balance	Transaction details	Transaction state
3. Direct SETTLED	Transaction record, Account balance	Account balance	Transaction state
4. Audit logging	AuditLog entries	-	-
5. Notification	ServiceBus message	-	-

---

2. Withdrawal Transaction Flow

Data Flow Characteristics:

• Balance Validation: Read-only queries with cache-first strategy
• Hold Placement: Write to AccountHolds table + Redis cache update
• Settlement: Account balance update + hold removal (atomic transaction)
• External Payment: Asynchronous call to NIBSS with retry logic
• Audit Trail: All state changes logged to AuditLog table

---

3. Cheque Transaction Flow (Two-Level State)

Cheque State Mapping:

Transaction State	Cheque State	Description	Balance Impact
PENDING	APPROVAL_PENDING	Awaiting transaction approval	None
PENDING	PENDING	Transaction approved, awaiting clearing	None
SETTLED	CLEARED	Cheque cleared successfully	Balance credited
CANCELLED	CANCELLED	Cancelled before approval/clearing	None
SETTLED	BOUNCED	Cheque bounced during clearing	None (or reversed if credited)

---

Approval Workflow Data Flow

Approval State Transitions

Approval Workflow Sequence

Database Operations Summary:

Phase	Writes (DB)	Reads (DB)	Cache Operations
Create PENDING	Transaction, AccountHold, TransactionImpact	Account	SET txn:id, SET hold:id
Approve	Account, GL_Entry, Transaction (UPDATE), ApprovalHistory, DELETE Hold	Transaction, TransactionImpact	DEL hold:id, SET txn:id, SET acct:id:balance
Reject	Transaction (UPDATE), ApprovalHistory, DELETE Hold	Transaction	DEL hold:id, SET txn:id, INCR availableBalance
Reverse	Account, GL_Entry (reversal), Transaction (UPDATE), ReversalHistory	Transaction, Account	SET txn:id, SET acct:id:balance

---

Event-Driven Data Flow

Domain Events Architecture

Event Types & Handlers

Transaction Events:

Event Type	Published By	Subscribers	Data Payload
TransactionCreated	All transaction handlers	Audit, Notification, Reporting	TransactionId, Type, Amount, AccountId, Status, Timestamp
TransactionApproved	Transaction Management	Audit, Notification, Reporting, Integration	TransactionId, ApproverId, ApprovalDate, PreviousState, NewState
TransactionRejected	Transaction Management	Audit, Notification	TransactionId, RejecterId, RejectionReason, Timestamp
TransactionSettled	Transaction Management	Audit, Reporting, Integration	TransactionId, SettlementDate, FinalAmount, BalanceImpact
TransactionReversed	Transaction Management	Audit, Notification, Reporting, Integration	TransactionId, ReversalReason, ReversedBy, ReversalDate, BalanceImpact

Cheque Events:

Event Type	Published By	Subscribers	Data Payload
ChequeDeposited	Cheque Handler	Audit, Notification	ChequeId, TransactionId, ChequeNumber, Amount, BankCode
ChequeCleared	Cheque Service	Audit, Notification, Reporting	ChequeId, ClearingDate, Amount
ChequeBounced	Cheque Service	Audit, Notification, Reporting	ChequeId, BounceReason, BounceDate, Charges

Process Events:

Event Type	Published By	Subscribers	Data Payload
ProcessStarted	Process Engine	Audit	ProcessInstanceId, ProcessDefinitionKey, StartDate, StartedBy
UserTaskCreated	Process Engine	Notification	TaskId, ProcessInstanceId, AssigneeId, TaskName
UserTaskCompleted	Process Engine	Audit, Process Engine	TaskId, CompletedBy, CompletionDate, Variables

---

Outbox Pattern for Reliable Messaging

Outbox Table Schema:

CREATE TABLE Outbox (
    Id UNIQUEIDENTIFIER PRIMARY KEY,
    EventType NVARCHAR(100) NOT NULL,
    EventData NVARCHAR(MAX) NOT NULL,
    CreatedAt DATETIME2 NOT NULL,
    Processed BIT NOT NULL DEFAULT 0,
    ProcessedAt DATETIME2 NULL,
    RetryCount INT NOT NULL DEFAULT 0,
    LastError NVARCHAR(MAX) NULL
)

CREATE INDEX IX_Outbox_Processed ON Outbox(Processed, CreatedAt)

---

Process Engine Data Flow

BPMN Process Execution Flow

Process Instance Data Model:

-- Process Instance
CREATE TABLE ProcessInstance (
    Id UNIQUEIDENTIFIER PRIMARY KEY,
    ProcessDefinitionKey NVARCHAR(100) NOT NULL,
    BusinessKey NVARCHAR(100),
    State NVARCHAR(50) NOT NULL, -- ACTIVE, WAITING, COMPLETED, TERMINATED
    StartTime DATETIME2 NOT NULL,
    EndTime DATETIME2 NULL,
    StartedBy UNIQUEIDENTIFIER NOT NULL
)

-- Process Variables (EAV pattern)
CREATE TABLE ProcessVariable (
    Id UNIQUEIDENTIFIER PRIMARY KEY,
    ProcessInstanceId UNIQUEIDENTIFIER NOT NULL,
    VariableName NVARCHAR(100) NOT NULL,
    VariableValue NVARCHAR(MAX),
    DataType NVARCHAR(50) NOT NULL
)

-- User Tasks
CREATE TABLE UserTask (
    Id UNIQUEIDENTIFIER PRIMARY KEY,
    ProcessInstanceId UNIQUEIDENTIFIER NOT NULL,
    TaskDefinitionKey NVARCHAR(100) NOT NULL,
    AssigneeId UNIQUEIDENTIFIER,
    State NVARCHAR(50) NOT NULL, -- CREATED, COMPLETED, CANCELLED
    CreatedAt DATETIME2 NOT NULL,
    CompletedAt DATETIME2 NULL
)

---

Data Synchronization Patterns

1. Cache Synchronization Strategy

Write-Through Pattern:

Cache-Aside Pattern (Read):

Cached Data Types:

Data Type	Cache Key Pattern	TTL	Invalidation Strategy
Account Balance	acct:{accountId}:balance	5 minutes	Write-through on transaction
Transaction State	txn:{transactionId}:state	1 hour	Write-through on state change
User Session	session:{sessionId}	30 minutes	Sliding expiration
Product Config	product:{productId}	24 hours	Manual invalidation on change
Process State	process:{instanceId}:state	1 hour	Write-through on state change
User Permissions	user:{userId}:permissions	15 minutes	Invalidate on permission change

---

2. Read Replica Synchronization

Replication Lag Handling:

• Typical lag: 1-5 seconds (asynchronous replication)
• Strategy: Direct critical reads to primary, non-critical to replica
• Read-your-writes: Cache recent writes, serve from cache if read immediately

---

3. Event Store Synchronization

Event Sourcing Read Models:

Read Model	Updated By	Purpose	Rebuild Time
AccountBalance	TransactionSettled, TransactionReversed	Query balance	~5 minutes
TransactionHistory	All transaction events	Query history	~30 minutes
AuditTrail	All audit events	Compliance reporting	~2 hours
DailyTransactionSummary	TransactionSettled	EOD reporting	~1 hour

---

Message Queue Data Flow

Service Bus Queue Architecture

Message Processing Flow:

Message Types & Payloads:

1. Notification Messages:

{
  "messageType": "TransactionApproved",
  "recipientId": "user-123",
  "channels": ["email", "sms", "push"],
  "templateId": "transaction-approval",
  "data": {
    "transactionId": "txn-456",
    "amount": 5000.00,
    "accountNumber": "1234567890",
    "approvalDate": "2026-01-01T10:30:00Z"
  },
  "priority": "high",
  "scheduledTime": null
}

2. Batch Processing Messages:

{
  "messageType": "EndOfDayBatch",
  "batchId": "batch-789",
  "batchType": "InterestCalculation",
  "parameters": {
    "businessDate": "2026-01-01",
    "productTypes": ["SAVINGS", "CURRENT"]
  },
  "estimatedDuration": "PT30M"
}

3. Integration Messages:

{
  "messageType": "PaymentInstruction",
  "externalSystem": "NIBSS",
  "operation": "InterBankTransfer",
  "payload": {
    "transactionId": "txn-789",
    "sourceAccount": "1234567890",
    "destinationAccount": "0987654321",
    "amount": 10000.00,
    "narration": "Payment for services"
  },
  "idempotencyKey": "idem-123-456",
  "retryPolicy": {
    "maxRetries": 5,
    "backoffSeconds": [10, 30, 60, 300, 600]
  }
}

---

Data Flow Performance Characteristics

Throughput Metrics

Flow Type	Average Latency	P95 Latency	P99 Latency	Throughput
Direct Transaction (no approval)	150ms	300ms	500ms	500 TPS
Approval Transaction (PENDING creation)	100ms	200ms	350ms	800 TPS
Approval Processing	200ms	400ms	700ms	300 TPS
Cheque Deposit	180ms	350ms	600ms	400 TPS
Cheque Clearing	250ms	500ms	900ms	200 TPS
Process Instance Start	300ms	600ms	1000ms	150 TPS
User Task Completion	250ms	500ms	800ms	200 TPS
Event Publishing	50ms	100ms	200ms	2000 TPS
Message Queue Publish	20ms	50ms	100ms	5000 TPS
Message Queue Consume	100ms	200ms	400ms	1000 TPS

Data Volume Estimates

Daily Transaction Volume:

• Deposits: 50,000 transactions/day
• Withdrawals: 40,000 transactions/day
• Cheques: 10,000 transactions/day
• Loans: 5,000 transactions/day
• Total: ~105,000 transactions/day

Database Growth:

• Transaction table: ~3 GB/year
• Audit logs: ~10 GB/year
• Event store: ~5 GB/year
• Total: ~18 GB/year

Cache Hit Rates:

• Account balance: 85% hit rate
• Transaction state: 90% hit rate
• User permissions: 95% hit rate
• Product config: 99% hit rate

---

Related Diagrams

• System Architecture - Overall system overview
• Component Architecture - Modules and components
• Deployment Architecture - Azure infrastructure
• Integration Architecture - External integrations
• Security Architecture - Security layers

---

Summary

The data flow architecture demonstrates:

• ✅ Transaction Processing - Clear flow from API to database with caching
• ✅ Approval Workflows - Two-phase processing with hold management
• ✅ Event-Driven Architecture - Reliable event publishing with Outbox pattern
• ✅ Process Engine Integration - BPMN execution with context loading
• ✅ Cache Synchronization - Write-through and cache-aside patterns
• ✅ Message Queues - Reliable async processing with DLQ handling
• ✅ High Performance - Sub-second latency for most operations
• ✅ Scalability - 500+ TPS capacity with horizontal scaling

Data Flow Characteristics:

• Consistency: Strong consistency for financial transactions
• Availability: 99.95% uptime with failover capabilities
• Partition Tolerance: Geo-redundant data replication
• Latency: P95 < 500ms for critical paths
• Throughput: 500+ transactions per second