We build backend systems behind serious products.

Challenge

Background job processing needs guarantees beyond list-popping. Workers crash, networks partition, and poison messages can block the queue forever. The system needed at-least-once delivery, automated recovery from worker failures, poison-pill isolation, and real-time visibility into queue health.

Decision

Lua scripts for atomic acquisitionCustom Redis Lua combines RPOP + ZADD into one atomic transaction, closing the microsecond gap where a crash could lose a task between pop and claim.

Challenge

Naive in-process rate limiters break across multiple servers. Real SaaS products need shared Redis state, atomic enforcement under concurrency, and different limits per customer tier — without duplicating the enforcement logic.

Decision

Three algorithms, one Redis Lua coreSliding Window Counter (anonymous), Sliding Window Log (free), and Token Bucket (pro) share the same middleware path — tiers swap algorithms, not code paths.

Challenge

Most upload demos stream video through the API and call ffmpeg inline — concurrent uploads pin the CPU the HTTP server needs. Production systems need direct-to-storage ingest, async retryable transcoding, and ownership-safe streaming without exposing raw bucket URLs.

Decision

Presigned uploads bypass the APIThe client gets a presigned S3/MinIO URL and PUTs the file straight to object storage — the upload service never buffers video bytes.

Challenge

Closest-in-vector-space is not best-answer. Research papers need exact term matching for acronyms and notation, multi-user sharing without duplicate embeddings, safe deletion that does not break other users, and conversational follow-ups that still retrieve the right chunks.

Decision

3-stage CPU reranker (BM25 → RRF → Cross-Encoder)Wide recall (top-30) then precision funnel to top-5 keeps the LLM focused and avoids lost-in-the-middle — without requiring a GPU.

Challenge

Centralized log ingestion at scale requires decoupling producers from consumers, handling burst traffic, deduplicating entries, and providing real-time visibility into ingestion health — all without losing data under load spikes.

Decision

Redis Streams over Pub/SubRedis Streams provide persistence, consumer groups, and acknowledgment — unlike Pub/Sub which drops messages if no consumer is listening.