Last year our food delivery app double-booked riders three times in one week. Two orders assigned to the same person. Angry customers. Refunds. The on-call engineer (me) debugging at 2am.
The bug was simple: two app servers checked if a rider was available at the same millisecond. Both saw "available." Both assigned the order.
This is the classic race condition, and it breaks more systems than you'd expect.
Here's what our code looked like:
Looks reasonable. Check first, then update. The problem: another server can run the same SELECT between your SELECT and UPDATE. Both think they won.
This pattern is called "check-then-act" and it's broken by design in concurrent systems.
The solution is making the check and update a single atomic operation:
If the row was already booked, zero rows affected. If available, one row updated. No race window.
Check the affected row count. If it's zero, the rider was already taken—find another one.
Sometimes a single SQL statement isn't enough. Maybe you need to check inventory, validate payment, and reserve stock—all atomically.
This is where distributed locks come in. Redis with SET key value NX PX 30000 gives you a lock with expiration. Acquire the lock, do your work, release it.
The NX flag means "only set if not exists"—atomic check-and-set.
Even with proper locking, add database constraints. They're your last line of defense:
Now even if your application code has a bug, the database won't let you double-book.
Every time you have a shared resource that can only be assigned once:
The rider problem shows up everywhere. Payment processing. Inventory management. Seat reservations. Event tickets. Any scarce resource.
Get it wrong and you're debugging at 2am. Get it right and you sleep through the night.
Check-then-act is not atomic. That's the whole lesson.
— blanho
Partial failures, network lies, clock drift. Everything that makes distributed systems a nightmare.
ACID sounds simple until you learn what READ COMMITTED actually allows.
UUIDs aren't always the answer. Here's when they hurt more than help.
SELECT status FROM riders WHERE id = 123;
-- returns: AVAILABLE
UPDATE riders SET status = 'BOOKED' WHERE id = 123;UPDATE riders
SET status = 'BOOKED', order_id = 456
WHERE id = 123 AND status = 'AVAILABLE';if redis.set(f"lock:rider:{rider_id}", "1", nx=True, px=5000):
try:
# do the booking
finally:
redis.delete(f"lock:rider:{rider_id}")ALTER TABLE orders
ADD CONSTRAINT unique_rider_active_order
UNIQUE (rider_id) WHERE status = 'ACTIVE';