AI experimentation

The experimentation loop AI-native products run before writing a spec: one idea, one prompt, real test cases, several models, every response reviewed. How to start your first experiment, how to structure prompt iterations, and how to turn what you read into something you can trust in production.

Newsletter10 SEPT 2026

The AI cost trap

"Build first, optimize later" works for SaaS — and quietly bankrupts AI products. Why your most engaged users are the ones losing you the most money, and the pre-ship experimentation loop that stops you from locking in a 10x-too-expensive architecture.

Madalina Turlea

Article16 JUN 2026

How to move past vibe checks: scaling manual AI testing into systematic evaluation

Vibe checks are the right way to start testing AI — and the wrong way to keep going. The step-by-step path from a few happy-path tests to systematic, automated evaluation.

Madalina Turlea

Article14 MAY 2026

What is AI experimentation, and why do you need it?

One idea. One prompt. Five real cases. Several models. Read every response. That's where AI-native products start. An 8-step playbook for product thinkers running their first experiment.

Madalina Turlea

Newsletter03 MAY 2026

Why ship and learn just does not work for AI features

Why the traditional 'ship and learn' approach fails for AI features.

Madalina Turlea

Newsletter10 APR 2026

The expert test: how to identify high-value AI features for your product

How to identify which AI features will deliver the most value.

Madalina Turlea

Article15 JAN 2026

Building an expense-policy AI agent: what we learned reverse-engineering Ramp

Three prompts, six models, 126 runs. What we learned trying to reverse-engineer Ramp's expense-policy AI agent — including why a more detailed prompt actually made GPT-5 worse.

Madalina Turlea

Article15 JAN 2026

How to run an AI experiment, step by step

From idea to evaluation: how to write the first prompt, build realistic test cases, pick the right mix of models, review the answers side by side, and turn your notes into something you can scale.

Madalina Turlea

Article15 JAN 2026

How to build a test dataset for an AI feature

A good test set is the difference between a feature that looks fine in a quick check and one you can actually trust. How to replicate real usage, design inputs that try to break the feature, and grow the set as you learn.

Madalina Turlea

Article09 JAN 2026

Prompt Engineering Techniques: Part 4

From craft to system: how to test prompts rigorously, compare approaches on real data, and build repeatable processes for prompts running inside a product or workflow.

Maria-Fontica Marinescu