Using AI to summarise customer feedback: what to check

The summary says customers want an e-wallet

This is an illustrative situation. You are the PO of an online cake-ordering app called Mầm Ngọt (a fictional name). After two weeks, customer feedback is scattered across several places: app store reviews, messages to the Facebook page, support tickets and a few emails from customers ordering for their companies. Around 120 rows in all. You collect them in a spreadsheet, ask an AI tool to summarise them, and within minutes you have a neat report.

The summary says four things. The biggest theme is late delivery, with 48 mentions. Customers want to pay with an e-wallet, backed by a quote that sounds very real: “The app needs an e-wallet now or I am leaving”. 25 customers have problems with the app. And overall sentiment is positive. The summary does not say which rows each theme came from. Your manager reads it and suggests putting an e-wallet feature into the backlog (the ordered list of work the team could do) for the next sprint.

Before agreeing, you search the spreadsheet for that quote. No row says anything like it. The rows about payment tell a different story: customers transferred money but the order still shows “awaiting payment”. This article covers where an AI-generated feedback summary goes wrong, and how to check it before it drives a decision.

Why do AI feedback summaries go wrong?

Summarising feedback really is something AI does fast. It reads all 120 rows in seconds, groups similar comments and gives each group a readable name. The difficulty is that the summary looks so tidy that the reader cannot see where it guessed, merged or skipped.

• AI can produce content that is not there. NIST, the US National Institute of Standards and Technology, calls this risk “confabulation”: false or made-up content stated with confidence. In feedback work, that can be a theme nobody raised or a quote nobody wrote.
• AI can miss and misread. NN/g, the user experience research group, writes that AI can miss, misinterpret or even manufacture insights when analysing research data, and recommends treating AI clustering as a first pass, not the finished analysis.
• Techniques only reduce errors. Anthropic, the company that makes the AI tool Claude, states in its documentation that ways of reducing “hallucination” do not eliminate it, and advises always validating critical information.
• Readers trust too easily. NIST also names the risk of people relying too much on machine output. A summary with numbers and quotes looks more certain than a messy 120-row spreadsheet, even when it is less accurate.

For a PO, the cost is not in the report. It is that one theme in the summary can go straight into the backlog and become weeks of the team's work. So the themes to check most carefully are the ones that will drive a decision.

Where does an AI summary of customer feedback go wrong?

The table lists six failure modes, how each one shows up and how to check it. You do not need all six checks for every theme. Use them on the themes about to be discussed for prioritisation.

The last row is a risk to watch for, not a measured result. This article has no data on how well AI reads everyday Vietnamese. NIST does name the risk of model performance differing between groups and languages. In a comment like “ship nhanh ghê, đặt sáng tới chiều mới có” (roughly “wow, fast delivery, ordered in the morning and it came in the afternoon”), the sarcasm sits in the context, which a human reader is better placed to catch than an automatic label.

How to check an AI-generated feedback summary

One approach is to work through four steps. AI speeds up step 2. The other three are the PO's job, and step 1 decides whether you can check anything at all.

Step 1, prepare the data. It is easy to skip because nothing about it is exciting. Each feedback row needs a row ID so it can be traced, a customer ID so you can count people, a channel column (app store, Facebook page, support ticket, email) and a customer group column if you have one (new, returning, company orders). At the same time, anonymise: remove names, phone numbers, emails, addresses and order IDs. The customer ID is a pseudonym: the table linking IDs to real people stays inside company systems and never goes into the AI tool with the data. That way customer support can still follow up when needed. For channels without an internal customer ID, such as app store reviews or Facebook page messages, you can count by the writer's account and note the limitation: one person may use several accounts. And only put the data into an AI tool if your company's policy allows it.

Step 2, let the AI draft the groups. A usable summary has to show which rows each theme came from, by row ID. Anthropic's documentation points the same way at the level of principle: ground the output in word-for-word quotes from the source and retract claims that have no supporting quote. A theme without row IDs is a theme you cannot check.

Step 3, check each theme that will drive a decision. Open the source rows by ID and read a sample. For small themes, read them all. Ask three questions: do these rows really describe the same problem, how many people raised it rather than how many times, and who are they and which channel did they use. Any quote used in the report must be found word for word in the data.

Step 4, compare with a manual read. Read a small random sample yourself, say a few dozen rows, and note the themes as you understand them before looking at the summary again. If your manual read finds a theme the AI did not list, or the sentiment labels are far off, read that part more closely. There is no standard sample size. Choose one where every channel has a few rows in the sample.

Counting people in step 3 works the same way as synthesising user interviews: group similar rows and count the people who said it, not the number of mentions. If the summary includes percentages or trends over time, also check the calculation, as you would with any numbers an AI has analysed.

• What to ask in a user interview
• Checking numbers analysed by AI

Which themes are solid enough for a decision?

Use this filter for each theme in the summary. A “yes” moves you forward. A “no” means doing the action beside it first.

Passing the filter does not mean the work happens now. It only means the theme is trustworthy enough to compare against everything else in the backlog. Whether to build it still depends on goals, cost and what is already waiting.

• Prioritising the backlog when everything is urgent

Example: rechecking the Mầm Ngọt summary

Back to the opening situation. The original summary had no source row IDs, so under the filter above none of its themes could be used for a decision. The PO starts again from step 1: adds row IDs, customer IDs, channel and customer group columns, anonymises, and has the rows grouped again so every theme lists its source row IDs. Then the PO checks each theme and reads a sample by hand. The table sets the original summary beside what the PO finds after those four steps. The data is a fictional feedback set for the Mầm Ngọt app, and every number is illustrative.

After the check, the story is completely different. There is no evidence for an e-wallet feature. What is worth discussing is the payment status bug, the OTP problem and late delivery, each with a count of people, context and row IDs anyone can open. The company-order group is small, but it is recorded instead of disappearing.

Notice that every cell in the “Checked version” follows the article's own steps: a count of people, context where it matters, and source row IDs. None relies on an unverified quote. After the team fixes these problems, it is still worth measuring whether that kind of feedback actually drops, rather than going on a feeling.

• Measuring a feature after launch

Which feedback data should stay out of AI tools?

Customer feedback holds more personal information than it seems: names, phone numbers in messages, delivery addresses, order IDs, sometimes screenshots showing account details. NIST lists data privacy among the risks of generative AI, including information leaking or people being re-identified.

• Ask about your company's policy first: which tools are allowed, which kinds of data can go in, who approves. If it is unclear, ask your manager, your legal team or IT.
• Anonymise before summarising, not after. Replace names and contact details with customer IDs, and remove order IDs and addresses.
• Do not assume how an AI tool stores or uses data. Vendor terms can change, so check your company's policy and the vendor's current terms.
• Follow the personal data protection rules your company applies. This article is not legal advice.

• Data you should not put into AI tools

Template: a review sheet for an AI-generated feedback summary

Use this sheet whenever you receive a feedback summary that AI helped produce, whether you made it or someone sent it to you. Fill in the first part once for the whole data set. Repeat the middle part for every theme that will drive a decision. Copy it and replace the parts in square brackets.

DATA SET
Sources and channels: [app store / Facebook page / support tickets / email / other]
Time range: [from date] to [date]
Feedback rows: [number]  |  Distinct customers: [number]
[ ] Anonymised: no names, phone numbers, emails, addresses or order IDs left
[ ] Company policy allows this kind of data in the tool that was used
[ ] Every row has a row ID, customer ID, channel and customer group (if available)

EACH THEME (repeat for every theme that will drive a decision)
Theme name: [name]
Customers: [number]  |  Mentions: [number]
Channel and customer group: [for example: returning customers, via support tickets]
Source row IDs: [list of IDs]
Original rows read myself: [number] of [total rows in theme]
[ ] The rows I read describe the same problem; no split needed
[ ] Every quote used in the report was found word for word in the data
[ ] Mentions per customer checked; anyone writing repeatedly is noted separately
Conclusion: [use / split / drop / unverified]

WHOLE SUMMARY
[ ] I read a random sample by hand covering every channel: [number] rows
[ ] My manual read found no important theme missing (if it did, note it here: [theme])
[ ] I skimmed the rows that fit no theme and the low-volume channels
[ ] Sentiment labels were compared with the manual sample; mislabelled rows: [number]
[ ] Small customer groups and dissenting views are recorded, even with few people
[ ] Percentages and trends (if any) have had their calculation checked

Checked by: [name]  |  Date checked: [date]

The line on “mentions per customer” catches exactly the late-delivery problem in the example: one person writing many times can make a theme look bigger than it is. “Unverified” is also a valid conclusion. It tells readers of the report which parts they should not lean on yet.

Common mistakes when using AI to summarise feedback

• Using a quote from the summary without finding the original. One false quote can persuade more than ten accurate rows.
• Counting mentions instead of people. One frustrated customer writing many times can turn a single incident into a “trend”.
• Accepting labels that are too broad. “Poor experience” does not tell the team what to fix.
• Reading only the big themes. Small customer groups, low-volume channels and dissenting views can vanish from a summary.
• Trusting automatic sentiment scores. Sarcasm, abbreviations and slang can be read the wrong way round.
• Putting data that has not been anonymised into an AI tool, or using a tool your company has not approved.
• Skipping your own reading. If you have never read a single original feedback row, you do not really know what customers are saying.

Summary

• AI groups feedback and names themes quickly, but it can invent themes or quotes, lose frequency, merge different problems, drop small groups, lose context and misread sentiment.
• Prepare the data first: anonymise, and give every row a row ID, customer ID, channel and customer group.
• Every theme that will drive a decision must trace back to source rows, have a sample of originals read, have people counted and agree with a small manual read.
• A theme you cannot trace to the data is dropped or marked unverified. The final decision still belongs to the PO.

This article is not affiliated with Anthropic. Claude is a trademark of Anthropic, PBC.

I am writing more PO situations in this same format, with templates you can use straight away. If you would like the next one by email, subscribe to the newsletter below.

• Get new articles by email

Frequently asked questions

Can AI summarise customer feedback accurately?

AI groups feedback and names themes quickly, but it can miss things, misread them, or produce themes and quotes that are not in the data. Treat an AI summary as a first draft, not a conclusion.

How do I check an AI-generated feedback summary?

For every theme that will drive a decision, trace it to the original feedback rows, read a sample of the originals, count people instead of mentions, and compare with a small manual read of your own. A theme you cannot trace to the source data is dropped or marked as unverified.

Should I put customer feedback into an AI tool?

Only if your company's policy allows it, and anonymise first: remove names, phone numbers, emails, order IDs and anything else that identifies a customer. If the policy is unclear, ask your legal or IT team before you start.

Sources to check

• NIST AI 600-1: Generative AI Profile
• Anthropic: Reduce hallucinations
• NN/g: Accelerating Research with AI

Related articles

• Using Claude as a PO: where it helps, what you still do: AI drafts fast. This article shows four mistakes to check and a checklist to use before a document reaches the team.
• Can AI replace a Product Owner? What still needs a PO: A balanced answer for new POs: AI helps with drafting, summarising and suggesting, while accountability for product value and the decisions behind it stays with a person.
• What data should you keep out of AI tools at work?: Six kinds of data a PO should keep out of AI tools, how to filter each piece before pasting, an anonymised complaint example and a ready-to-use checklist.
• Numbers from AI analysis: what to check before reporting: How a PO checks numbers an AI tool calculated: reconcile totals, check the denominator and the period, recompute a few key figures, and label what is still uncertain.