Back to portfolio
Journal

Field notes from shipping data and AI systems

Less thought leadership, more delivery notes. Architecture choices, failure patterns, and the tradeoffs that show up once real users arrive.

Production AIData platformsDelivery patterns

Featured note

01AI delivery notes

RAG in Production: Chunking, Retrieval Quality, and the Problems the Demo Hides

A RAG assistant becomes a product the moment someone relies on it during real work. From that point on, the engineering around the model matters more than the demo.

Context window limits, suboptimal chunking, retrieval evaluation, hybrid search, observability, and cost control — the real engineering behind a RAG system that works past the demo.

RAGRetrievalEvaluationLLM Ops
7 min read

What matters

01

Index like a real pipeline

Stable IDs, source versions, and diff-based ingestion matter before prompt tuning does.

02

Score retrieval separately

If the right evidence is not showing up, the model never had a fair chance.

03

Plan for refusal and tracing

A trustworthy assistant cites, abstains, and leaves behind a debuggable trail.

Working rule

Trace every answer

First fix

Ingestion quality

Failure to avoid

Confident guesswork

More notes

Platform delivery notes02

Building a Data Platform People Actually Trust

The hard part is not choosing a warehouse. It is creating a platform where finance, ops, and product stop arguing about whose number is correct.

Data PlatformdbtSnowflakeGovernance

Simplify the stack on purpose

A smaller platform with clear ownership is easier to scale than a clever one with overlapping tools.

8 min read
Read article