What Three AIs Said When We Asked Them to Audit WIMM

Most app testimonials are written by friends, family, and the founder's mom. Star ratings on the App Store get gamed. Five-star reviews on Trustpilot are the ones companies survey for right after a happy moment. The whole genre is suspect, and shoppers know it. Customer reviews on a software company's own website are worth roughly what you would expect. The company picks the happiest ones, edits them lightly, and decides which to show.

Photo by Markus Winkler on Unsplash

So we tried something different. We took three frontier AI models, Claude (Anthropic), GPT-4o (OpenAI), and DeepSeek, gave each one the public WIMM marketing site, and asked for an honest review. We did not edit their answers. We published all three, including the criticism, on a dedicated /reviews page. The scores landed at Claude 4 out of 5, GPT-4o 3 out of 5, and DeepSeek 4 out of 5.

Why this is not theater

A normal "AI testimonial" generator works like this. The company writes flattering copy, an AI rephrases it, and the result is posted as if it came from an outside reviewer. That is theater, and most people see through it.

What we did is different. The AIs were not asked to praise WIMM. They were handed the actual public marketing site as their only input, told to compare WIMM honestly against YNAB, Monarch, and Rocket Money, and instructed not to be polite. Two of the three flagged real product weaknesses. None of them invented a feature. None of them adjusted a score because we asked nicely.

There is one caveat we want to be upfront about. The AIs were given the marketing site, not the live product. They cannot test transaction import, watch a budget go red, or click through the Plaid Link flow. So the reviews audit how well WIMM's pitch holds up under scrutiny, not how the app behaves in your hands. The next round of the audit will run against the live demo too.

What we told the AIs

The prompt has three parts. A role, a corpus, and a task. Anyone can reproduce it.

The role. A professional product reviewer who has just spent two weeks exploring the public marketing site.

The corpus. The actual content of wimm.money injected between delimiters. Every feature page, every comparison page, the FAQ, and the security page. We strip scripts, styles, navigation, and footer, cap each page at 4,000 characters, and concatenate them with page headers. About 48,000 characters of context total. The model is told its review must be grounded in what those pages say, not in invented features.

The task. Write a 300 to 500 word honest review covering what WIMM does well, what is weak or missing compared to YNAB, Monarch, or Rocket Money, who it is the right fit for, and who it is not. Be specific. Be useful, not polite. Do not invent statistics. End with a 1 to 5 star rating and a one-sentence summary.

Photo by Markus Winkler on Pexels

The full seed script lives in the repo at apps/web/scripts/seed-ai-testimonials.ts for anyone who wants to read it byte by byte.

Reviewing software with AI is a real practice now

Feeding a product to an AI critic is not a gimmick we invented. Reading code, spec, and product surfaces with a large language model to flag gaps and risks is becoming standard engineering practice. This short IBM Technology talk walks through how AI gets used to review and secure software, which is the same idea applied to our marketing pitch instead of our code.

What the three models said

The full reviews are on /reviews. The short version of each:

Claude (Anthropic, claude-sonnet-4-6) gave WIMM 4 out of 5. It praised the AI categorization story specifically because three engines route each transaction through Claude, GPT-4o, and DeepSeek. It flagged the missing native mobile app as the most notable gap, while acknowledging that WIMM ships as an installable PWA.

GPT-4o (OpenAI) gave WIMM 3 out of 5. The most critical of the three. Its headline summary: "WIMM offers privacy-focused financial tracking with useful budgeting tools, but lacks advanced features." Strongest on the privacy and pricing story, weakest on positioning against feature-dense competitors.

DeepSeek (deepseek-chat) gave WIMM 4 out of 5. It led with pricing as the standout. Direct quote: "WIMM's pricing is genuinely refreshing: $9/month flat for an entire household (up to 5 members), with no per-seat fees." It also liked the envelope budgeting depth and the privacy posture.

The three models had no coordination, no access to each other's responses, and no incentive to agree. The fact that two landed on 4 and one on 3 is information.

What they agreed on

Across the three reviews, the same strengths kept coming up.

Photo by Towfiqu barbhuiya on Pexels

• Honest pricing. $9 a month, flat, for a whole household, with no auto-renewal traps. All three commented on this.
• Multi-engine AI categorization. Three engines (Claude, GPT-4o, DeepSeek again), with the transparency about which model categorized what standing out.
• Debt Reducer. Avalanche, snowball, and what-if scenario tools. Two of three reviewers called this rare at this price point.
• Envelope budgeting without the YNAB tax. All three appreciated the YNAB-style discipline at a below-YNAB price.

They converged on the same weaknesses, too.

• No automated bill negotiation. Rocket Money does this. WIMM does not, and we are not planning to.
• Investment account sync is balance only. No portfolio analysis. Fair criticism.
• PayPal-routed subscriptions show up with verbose descriptions. This was DeepSeek's most specific complaint, and it is a real gap we are working on.

What surprised us

The first time we ran this experiment, before we tightened up the marketing copy, the average rating was 2.3 out of 5, and the reviews accused us of missing features that were actually shipped. Goals, Household Sharing, Reports, even Plaid sync, were all flagged as absent. They were not absent. They were just under-marketed.

That is the second value of the exercise. If three sophisticated AI reviewers cannot find your features by reading your site, no human visitor is going to either. We rewrote the homepage feature grid, added dedicated pages for the missed features, and ran the experiment again. The rating climbed from 2.3 to 3.7. Every previously missed feature showed up by name in the new reviews. That is honest signal about marketing quality, surfaced cheap and fast.

What we did with the feedback

When the audit highlighted features that already exist but were not surfaced on the site (Plaid coverage, Household Sharing, Savings Goals), we treated that as a marketing problem and shipped feature pages for them.

When the audit highlighted real product gaps (a native iOS and Android app, more analytics in some areas), we treated that as a roadmap signal. The mobile gap led directly to the PWA install article and the bottom-nav redesign that shipped at the end of May.

When the audit flagged things we explicitly disagree with (more aggressive tracking would help our marketing, more AI everywhere would be more impressive), we kept our position and noted the disagreement.

Why we published the criticism

A confident review surface is itself a marketing claim. Anyone can hide negative feedback. Posting it next to the positive feedback says we read it, we did not panic, and we are not going to bury it. Shoppers detect that and trust it. Models do too, apparently. One reviewer explicitly cited the transparency of the reviews page as a positive in the next run.

This is not a trick that works only for budgeting apps. If you run a product, the simplest path to better marketing is to feed your live site to three AI critics and see what they say. They are not your users, but they read like skeptical first-time visitors, and they catch the failures internal teams are too close to see.

You can check this yourself

If you are evaluating WIMM and want a sanity check, you do not have to take our word for any of it. Open the reviews page, read the full unedited text from each model, then go look at the marketing pages they were reading. The user-submitted reviews live there too, gated by an account-age check and a captcha so the page stays signal, not spam. If a claim does not hold up, write us at support@wimm.money. The whole point of publishing the prompt and the scores is to be auditable.

If you want to try the product before deciding, the live demo opens straight to the dashboard with sample data, no account, no card, no bank connection required: app.wimm.money/demo.