LLM Product Engineering

Shape language-model behavior through prompts, schemas, context, retrieval, and tools.

You'll build: A grounded feature prototype with a typed user contract and observable tool or retrieval behavior.

1. Prompt Engineering: Techniques & Patterns~2h
2. Structured Outputs: JSON, Schema Validation, Constrained Decoding~2h
3. Embeddings & Vector Representations~1h
4. Context Engineering: Windows, Budgets, Memory, and Retrieval~2h
5. RAG (Retrieval-Augmented Generation)~2h
6. Function Calling & Tool Use~1h

Measure quality and control cost and safety before production exposure.

You'll build: An evaluation suite with guardrail tests and a quality, latency, and cost scorecard.

1. Evaluation & Testing LLM Applications~1h
2. Caching, Rate Limiting & Cost Optimization~1h
3. Guardrails, Safety & Content Filtering~1h

Package the feature as a production-style application behind operational controls.

You'll build: A deployed LLM feature with routing, limits, observability hooks, and documented failure behavior.

1. Building a Production LLM Application~2h
2. AI Gateways — LiteLLM, Portkey, Kong AI Gateway, Bifrost~1h

Show how the feature advances through controlled exposure with rollback evidence.

You'll build: A product case study with eval results, cost tradeoffs, release gates, and a rollback decision.

1. Shadow Traffic, Canary Rollout, and Progressive Deployment for LLMs~1h