“The data team's queue is two weeks deep — and your decision is today.”

The struggle

Trial-to-paid conversion dropped four points last month. You need to decide by Thursday whether to pull the onboarding rework forward, and answering that means knowing which step, which segment, and whether it’s real or an artifact of the pricing test. So you file a ticket. It gets triaged into a queue that’s two weeks deep behind a finance close and a board deck, and the analyst who eventually picks it up will need a 30-minute call to understand what you meant by “drop-off,” then send back a chart that answers a slightly different question than the one you asked.

The obvious fix is “learn SQL,” and it fails for a boring reason: you don’t need SQL, you need one number in the next four hours, and by the time you’d be fluent enough to get it safely you’d also need to know your warehouse’s schema, its event tables, and which of the three users tables is the real one. So the decision gets made on instinct with a data ticket still open behind it — which is the actual cost, not the wait. This playbook gets you a defensible answer to a bounded question the same day, and is explicit about the questions you should still put in the queue.

The chain

Step 1 — Ask the question where the events already live · Amplitude

Start in Amplitude, not in a spreadsheet. Build the funnel or retention chart for the question as asked — for a funnel, the ordered steps, the conversion window, and the date range that brackets the change (building a funnel chart). Then ask the agent to do the segment hunting for you, because that’s the part that takes an analyst an afternoon:

Using my trial-to-paid funnel for the last 90 days: which single step lost the most
conversion between the first 60 days and the last 30? Then break that step down by
platform, plan, and signup source, and tell me which segment accounts for most of the
drop. Show the numbers for each segment, and tell me which events and properties you used.

The Global Agent docs are worth two minutes here: the agent builds charts and answers follow-ups from plain language, takes 30–60 seconds per real question, and Amplitude itself tells you to validate anything AI-built before it feeds a decision. Take that literally — before you go further, rebuild the one chart that matters by hand and confirm you get the same number. If they disagree, the agent picked different events than you meant, and everything downstream inherits that error.

You’re done with this step when you have one anomalous slice: Android trials on the Team plan, stalling at workspace creation.

When to skip: if the chart answers your question outright, stop here and go write the memo. Half of these questions die at step 1, which is the point. And if your events aren’t in a product-analytics tool at all — the numbers you need live in the warehouse — swap this step for get the SQL for a funnel question in plain English and have an analyst read the query before you run it.

Step 2 — Interrogate the slice, with the code showing · Julius

Amplitude tells you where. It won’t tell you whether the shape is real. Export the anomalous slice as CSV — user-level rows, with the properties and timestamps you care about — and upload it to Julius (hands-on walkthrough). Then:

This CSV is user-level data for Android trials on the Team plan over the last 90 days.
Columns are described in the header row; ask me if any are ambiguous instead of guessing.

1. Describe the data first: row count, date range, missing values per column, and any
   duplicate user rows. Tell me if anything looks truncated or filtered.
2. Compare the last 30 days against the prior 60 on the workspace-creation step, and tell
   me whether the difference is larger than the week-to-week variation in this data.
3. Break the last 30 days down by signup week to show whether this is a step change on a
   specific date or a gradual slide.
4. Flag any confounds you can see IN THIS DATA — a cohort that appears suddenly, a property
   that changes mid-period, an incomplete final week.

Rules: show me the code for every step so I can check it. State your assumptions
explicitly and label anything estimated or modeled as an estimate. Do not fill in missing
values silently, and do not report a difference as meaningful without showing the
variation you compared it against.

The “show me the code” rule is the entire reason to use a tool like this instead of asking a chatbot to eyeball a table. The code is your audit trail: you can read it, an analyst can read it in ninety seconds, and it’s what turns “the AI said” into “here’s exactly how this number was computed.” If you can’t follow the code, that’s the signal to ask, not to proceed. (Analyze a CSV export without a spreadsheet is the lighter version of this step.)

Step 3 — Write the memo the decision actually needs · Claude

Any frontier model runs this step. ChatGPT and Gemini do this job as well as Claude does; the prompt is plain text in, text out. We name Claude because that’s what we field-tested with (see last field-tested above) — not because it’s required. Use whichever you already pay for.

Paste the Julius output — findings and the code — into Claude:

Below is an analysis of trial-to-paid conversion for Android Team-plan trials, including
the code that produced it. Write a one-page decision memo for my product leadership.

Structure:
- The decision on the table (one sentence).
- What the data shows — every number cited verbatim from the analysis below.
- What the data does NOT say: the questions this analysis cannot answer, and the
  alternative explanations it does not rule out.
- My recommendation, and my confidence in it.
- "What would change this recommendation": the specific finding that would flip it.
- What I'd need from the data team to be certain, and how long that would take.

Rules: every number must appear verbatim in the analysis I gave you — do not round,
re-derive, annualize, or extrapolate. If a number I need for the memo isn't in the
analysis, write [NOT IN ANALYSIS] rather than supplying one. Label anything the analysis
marked as an estimate as an estimate in the memo too. Under 400 words.

The “what would change this recommendation” line does more work than it looks like it does. It tells your reader you know the limits of a same-day analysis, and it pre-commits you to changing your mind on a specific trigger instead of defending the memo later. (Decision memo from an analysis is the standalone prompt.)

Step 4 — Check every number against source (10 minutes, not optional)

Open the memo next to the Amplitude chart and the Julius output, and verify each figure by eye. Three specific checks:

1. Verbatim, not re-derived. A percentage that appears in the memo but not in the analysis was computed by the LLM. Delete it or recompute it yourself.
2. Estimated numbers labeled as estimates. If sampling was on in Amplitude, your totals and uniques are extrapolations from a 10% user sample, not counts — Amplitude’s own docs say so. Write ”≈” and say why.
3. The date range is honest. Truncated final week, timezone boundary, conversion window shorter than your actual sales cycle — any of these produce a real-looking drop that isn’t one.

Then send the memo to your data team at the same time you send it to leadership, with the Julius code attached and one line: “decision was today, here’s what I did — tell me if I got it wrong.” This is what keeps self-serve from becoming a credibility problem. You’re not going around them; you’re bringing them a checkable artifact instead of a ticket.

Where it breaks

• Metric definitions are not yours to change. Self-serve means answering a question with existing definitions. It does not mean deciding what “active user” means, what counts as activation, or where the trial window starts. Redefine a company metric in a memo and you’ve created a second version of the number that someone will cite in six months. Emilie Schario’s JTBD piece makes the case plainly: metrics management is a data-team job with named business and technical owners. If your answer requires a new definition, that’s a queue item, and it always was.
• A modeled number quoted as a fact will cost you more than the two-week wait. This is the failure that ends self-serve at a company. Query-time sampling extrapolates totals and uniques from a 10% user sample; Julius will happily produce a projection if you don’t forbid it. Say “≈2,400, estimated from a 10% sample” in the exec review and you keep your credibility. Say “2,400” and get corrected by an analyst in front of your VP, and you lose the right to run your own numbers.
• Sampling and date-range traps, specifically. The last period is almost always incomplete and always looks like a decline. A funnel’s conversion window silently excludes users who convert later — shorten it and conversion drops, and nothing about the product changed. Timezone boundaries shift day-level counts. Any one of these can manufacture the exact four-point drop you’re investigating, which is why step 2’s prompt asks about the incomplete final week and step 4 checks the range by hand.
• Unread code is just a confident guess. If you skip reading Julius’s code because it looks plausible, you’ve reintroduced everything this workflow was supposed to remove — you just got the wrong answer faster and with a chart attached. Nobody downstream can audit what you didn’t look at.
• Some questions genuinely need the queue. Anything joining product events to revenue, contracts, or support systems. Anything that will become a board number or a company KPI. Anything where being wrong is expensive and slow to detect. The point of getting fast at the bounded questions is that the data team’s two weeks get spent on the ones that actually deserve two weeks.