AI Insights — AI Product Coach
An AI-powered review of your product, run against your actual responses. Find what's working, fix what isn't.
Most analytics tell you what happened. AI Insights tells you why — and exactly what to change.
Every product you build on Productised is a hypothesis: that these questions, in this order, will collect the right information, and that the AI will do something valuable with it. AI Insights tests that hypothesis against your actual response data. It reads your conversations, evaluates your outputs, and comes back with a structured diagnosis of where your product is performing and where it's creating friction.
This is not a dashboard of numbers. It is an AI coach that has reviewed your product's work — and has specific feedback.
What AI Insights does
Click Generate Insights and the AI reviews up to 20 of your most recent responses. It reads the full conversation for each session — the questions asked, the answers given, and the outcome generated — and produces a structured analysis across five dimensions.
Output quality score
A score from 1–10 with a written observation. This measures how well the AI's generated outcomes matched what each conversation actually warranted. A high score means the outcomes felt earned — relevant, specific, grounded in what the person shared. A low score means there's a gap: the outputs are generic, or the system prompt is producing something that doesn't reflect the conversation.
This is your most important signal. If your output quality is low, conversion will be low — because personalised-feeling outcomes drive action, and generic ones don't.
Response themes
Recurring patterns across your responses, organised into three categories:
- Positive signals — what's consistently working. Confirm these and protect them when you iterate.
- Opportunities — directions you could develop. These are patterns that suggest an angle you haven't fully exploited yet.
- Warnings — consistent problems worth addressing. These are patterns that are actively hurting your product's performance.
Themes are derived from the actual content of your responses, not from completion rates or time-on-page. They reflect what your audience is actually thinking and saying.
Confusion points
Questions where people's responses were consistently vague, off-topic, or incomplete — with a pattern description and a concrete suggestion for improving that specific question.
If the same question is producing weak answers across multiple sessions, the question is the problem. Confusion points identify exactly where to rewrite.
Prompt improvement suggestions
Specific, actionable suggestions for improving your AI Product's system prompt or individual node configurations — based on the gap between what the AI collected and what it produced.
These suggestions are grounded in your actual data. They're not generic prompt-writing advice — they're responses to specific patterns the AI observed in your conversations and outputs.
Per-response insights
For each sampled response: the inferred intent behind their answers, an output match rating (strong / partial / weak), and a specific observation about that session.
This is useful for understanding outliers — both exceptional sessions and underperforming ones — and for spotting patterns the aggregate view might miss.
How to use AI Insights effectively
Run it at meaningful milestones, not constantly. The analysis is most useful when you have a real sample to work with — 20–30 responses gives you reliable patterns. Running it after every 5 sessions will produce noise. Running it after a campaign, a launch, or a product update will produce signal.
Treat it as a product iteration loop. The workflow is: build → get responses → run AI Insights → implement one or two suggestions → repeat. Each pass tightens the product. Over time, your output quality score climbs, your confusion points disappear, and your completion rate follows.
Prioritise prompt suggestions over everything else. The output quality score and prompt improvement suggestions are the highest-leverage outputs. A single change to your system prompt — informed by what the AI actually observed in your conversations — can shift your output quality score significantly and improve every session that follows.
Use confusion points to rewrite questions. If the AI flags a specific question as consistently producing vague answers, rewrite it before your next traffic push. A clearer question produces richer inputs, which produces better outputs, which produces higher conversion.
Why this exists
The hardest part of building an AI product isn't the launch — it's the iteration. Most products get deployed, get traffic, and then sit static while their owners wonder why completion rates aren't climbing. There's no obvious feedback loop.
AI Insights creates that loop. It makes the invisible visible: the questions that confuse people, the outputs that feel generic, the system prompt gaps that are costing you conversions. And it gives you the specific changes that will fix them — not guesswork.
The best products on Productised aren't the ones that were built perfectly the first time. They're the ones that were iterated on. AI Insights is how you do that.