NorthwindAPI & backend
Rebuilding a payments API to handle ten times the traffic
A move from a struggling monolith to well tested services for a payments provider processing two million transactions a day.
- Client
- Northwind
- Industry
- Payments
- Role
- Senior backend engineer
- Year
- 2024
- Duration
- 8 months
Built with
- Node.js
- TypeScript
- PostgreSQL
- Kafka
- Kubernetes
Scope
- API design
- Backend
- Data migration
- Observability
- Peak traffic handled without an incident
- 10x
- Duplicate charges since launch
- 0
- Deploys, up from one a month
- 15/day
The challenge
Northwind's payment API ran on a single monolith that struggled every Black Friday. Deploys were risky, one slow partner bank could stall the whole system, and duplicate charges during retries cost the team hours of manual refunds.
The approach
We split the system along clear boundaries, starting with the parts that hurt most, and kept the public API unchanged so no customer had to migrate.
- Introduced idempotency keys on every write, so a retried request can never charge a card twice.
- Moved slow bank calls onto Kafka queues with automatic retries and dead-letter handling.
- Added tracing and service-level objectives, so problems show up on a dashboard before customers notice them.
A retried capture is now always safe:
await payments.capture({
paymentId,
idempotencyKey: `capture-${paymentId}`,
})The outcome
The new platform handled ten times its usual traffic on its first Black Friday without an incident. Duplicate charges dropped to zero, and deploys went from once a month to several times a day.
Elena brought calm to a migration that had stalled twice before. Rigorous with testing, generous with our team, and the results speak for themselves.