A kitchen app where AI agents propose and I decide

A kitchen app where AI agents propose and I decide

AI agents

Human in the loop

Interaction design

Trust and control

Role

Solo; designed and built it

Scope

Agent architecture, interaction design, front end, measures

Team

Me, working with Claude (see 007)

Timeline

Side project, since November 2026

Outcome

Seven agents, one review screen, Critic send-backs, undo for every run

Status

In progress, stage 4 of 6

In short

Pantry Chef is a kitchen app I built for myself. It tracks what’s in my fridge, freezer and pantry, and suggests what to cook before things go off.

The first version used one Claude prompt per feature. I rebuilt it as a team of AI agents with a clear rule: agents propose, I decide, and anything they change can be undone.

001

The agent edition: Adding groceries, with each agent step shown live in the activity panel
The agent edition: Adding groceries, with each agent step shown live in the activity panel

The agent edition. Adding groceries, with each agent step shown live in the activity panel.

Why rebuild something that worked

The first version did its job. I pasted a shopping list, and Claude sorted it into the fridge, freezer and pantry. But when it got something wrong, like frozen peas going into the pantry, nothing caught it. I was the only check, and I couldn’t see why it had done what it did.

I also wanted to work through the question I think matters most in AI products right now: when AI acts on someone’s behalf, how do they stay in control? My own kitchen was a safe place to find out.

002

Agents propose, I decide

I split the work between seven specialist agents, each with one job and its own tools: Intake, Inventory, Chef, Nutrition, Planner, Shopper and Taste memory. An orchestrator reads plain-language requests and hands each part to the right agent. Three rules shape how they work:

  • Read freely, never write: agents can look anything up, but every change they want comes back to me as a proposal.

  • Nothing changes until I approve: everything the agents want to change arrives on one review screen, grouped by agent.

  • Undo for everything: each approved run is saved as one change set, so a single tap puts the kitchen back exactly as it was.

Not everything needs an agent. Known flows, like adding groceries, run as fixed steps in code, and so do the Critic’s basic checks: fast, cheap and predictable. Claude handles only the parts that need judgement, and agents use tools to look things up, like what’s in the kitchen, instead of relying on one long prompt.

Every step is visible in an activity panel: which agent did what, which tools it called and what went in and came out, what the Critic flagged, and what I decided. It turns a black box into something I can follow, question and debug.

003

How it fits together: An orchestrator and fixed workflows hand work to specialist agents

How it fits together. An orchestrator and fixed workflows hand work to specialist agents. The Critic checks their output, and nothing reaches the kitchen until I approve it.

One sentence, four agents

This is what the old version couldn’t do. I can type or say what happened, like “I made the tofu pudding, it was a bit too sweet, 4 stars, and half the honey’s gone”. The orchestrator splits that into separate jobs:

  • Inventory: honey from 75% to 38% left

  • Nutrition: log the pudding to this week’s cooking

  • Taste memory: remember to use less honey in desserts, for every recipe from now on

  • Chef: rewrite the saved recipe so it’s less sweet

Before, that was five separate actions across three tabs, and the app never remembered that I like desserts less sweet. Now it’s one review screen. I untick anything I don’t want, apply, and one tap undoes the lot.

004

Pantry Chef’s proposed changes, grouped by agent, with the Critic’s check at the top
Pantry Chef’s proposed changes, grouped by agent, with the Critic’s check at the top
The agent activity trail for the same request, step by step
The agent activity trail for the same request, step by step

One sentence in, one review out. Every change the agents propose, grouped by agent and checked by the Critic (left), and the hand-offs behind it in the activity trail (right).

Anatomy of one request: What happens behind “I made the tofu pudding…”, step by step

Anatomy of one request. What happens behind “I made the tofu pudding…”, step by step. Steps 1 to 11 only produce proposals; nothing is written until I approve at step 12.

A Critic that checks the work

Everything goes past the Critic before I see it: sorted groceries, recipes, week plans and proposed kitchen changes. It works in two layers:

  • Rules in code: cheap, instant and predictable. Is this food? Is it past its use-by? Is it already on the shopping list? Do I own the cookware the recipe needs?

  • A Claude review: for grocery lists, it compares what came back with what I actually sent, to catch an item that went missing or an amount that was misread.

When something fails a serious check, the Critic sends it back once. A recipe that needs a frying pan I don’t own goes back to Chef to be rewritten for a saucepan. Anything still wrong is flagged where I’ll see it.

005

Pantry Chef’s grocery table with Critic flags on the rows they’re about and one-tap fixes
Pantry Chef’s grocery table with Critic flags on the rows they’re about and one-tap fixes
The agent activity panel showing the same Critic checks
The agent activity panel showing the same Critic checks

The Critic at work. Flags on the rows they’re about, with one-tap fixes (left), and the same checks in the activity trail (right).

How I built it with AI

I built it myself because a mockup can’t show what matters most in an AI product: what happens when the AI gets something wrong. I wanted to see real failures, in my own kitchen, and design for them.

I worked with Claude in stages. I set the shape of the system: the agent roles, the rule that agents propose and I approve, undo for every run, and what the Critic should catch. Claude wrote the code, and I reviewed each stage against how I wanted it to behave. Every stage was tested end to end before it went live, in a separate copy of the app, so my real kitchen kept working throughout.

My first real run showed me where the tests weren’t enough. The Critic caught Chef listing a clay pot as an ingredient, but Chef’s revision didn’t fix it. I decided the flag should stay visible rather than be quietly dropped, and had equipment moved out of the ingredient list in code, so the Critic doesn’t depend on Claude getting it right.

006

What could go wrong

Letting AI act for someone only works if you plan for it getting things wrong. These are the risks I designed for, and the ones still open:

  • Confidently wrong: an agent can misread a list or invent an ingredient. Nothing it proposes is written until I approve, the Critic checks first, and undo covers what slips through.

  • Slow: one request can take several Claude calls. Known flows skip the orchestrator, simple checks run as code, and the trail shows what’s happening while I wait.

  • Cost: every check is another call. Simple jobs like storage tips and shopping lists use a lighter, cheaper model, and the Critic’s first layer is plain code.

  • Claude unavailable: the kitchen, lists and plans still work by hand. Only the agent features pause.

  • Too much to review: one sentence can become 22 proposals, and at that volume it’s easy to approve without reading. Changes are grouped by agent, the Critic flags only what matters, and I can untick anything before applying.

  • Still open: my own direct edits can’t be undone yet, and every measure so far comes from one user: me.

007

Results so far and what’s next

My first real run produced the clay pot fix above. It also sorted a 14-item receipt photo, matched 2 items to names I already use, and turned one “I made this” into 22 proposed changes across five agents. Each review also records what I kept, changed or undid, so the system measures itself:

  • Intake accuracy: the share of fields I didn’t need to edit before adding

  • Approvals per agent: how often I accept each agent’s proposals, which shows which one to improve next

  • Undo rate: how often I reverse a run after approving it

Four of six stages are live. Next, I’m adding a measures view in Insights, starting with food thrown out per month, the outcome the whole app exists to reduce. After that, a public demo anyone can try, and undo for my own edits.

008

Air Blaster 9000

A VR Controller You Can Feel

Ask Federico