Skip to content

Recovery Flow

This page explains how fyers-store recovers after restarts or WS reconnects.

What It Does

  • Persists runtime order/position state to a state DB.
  • Loads persisted state on startup.
  • Reconciles with REST snapshots before consuming WS updates.

Why It Exists

WS messages can be missed across process restarts or disconnects. Persisted state plus REST reconciliation prevents duplicate fills or missing positions.

Startup Recovery

  1. Broker loads persisted state from the state DB.
  2. Broker fetches REST snapshots (orderbook, tradebook, positions, holdings, funds).
  3. Broker reconciles and repairs state if possible.
  4. Only after reconciliation does it start the Order WS.

Reconnect Recovery

  1. WS reconnects with bounded backoff.
  2. Broker runs a REST reconcile before processing new WS messages.
  3. Dedupe logic ensures trades are not double-applied.

Failure Handling

  • If reconciliation fails, the broker raises ReconcileFailureError and stops trading.
  • Operators must resolve auth or connectivity issues before restarting.

Safety Implications

  • Recovery is deterministic and fail-fast to avoid trading with unknown positions.
  • The state DB must remain separate from the historical cache DB.

Scenario Example

Scenario: The process crashes during a partial fill. - Persisted state holds the last known fill watermark. - On restart, REST tradebook fills the gaps. - Broker dedupes by tradeNumber and avoids double-counting fills.