Red flags in a growth marketer's case study, including mine
Seven ways a case study misleads without technically lying, and how to audit one in ten minutes. Applied to my own headline number at the end.
the short answer: growth case studies are marketing about marketing, and the common distortions are not lies. they are omissions with a particular shape. here are seven, how to test for each in about ten minutes, and the same test applied to my own number.
1. A percentage with no starting point
“increased signups 300%” means nothing without the base. three to twelve is a 300% increase and is noise.
test: ask for the absolute numbers. real ones come back immediately.
2. A result with no timeframe
“cut CAC by 40%” over what period, and did it hold? most acquisition costs improve temporarily when spend drops, then revert when it scales again.
test: ask what the number was ninety days later. this one catches more than any other question on the list.
3. Blended numbers presented as paid performance
blended cost of acquisition includes users who arrived free. an operator with a large organic base looks dramatically better on blended than their paid work justifies.
test: ask what share was paid versus organic. an honest answer is specific and slightly deflating.
4. Attribution that assumes credit
if a brand ran a podcast campaign, a launch and a price change in the same quarter, the growth marketer’s channel gets credited with the whole lift in the case study.
test: ask what else was happening at the same time. anyone who says “nothing” was not paying attention.
5. No failed engagements anywhere
everybody has them. a portfolio with only wins means the losses were filtered, which tells you the case studies were selected rather than reported.
test: ask directly what did not work and what it cost. hesitation here is the answer.
6. Category advantage sold as method
some products spread on their own. a social consumer app in a large young market grows in ways a compliance tool never will, regardless of who runs marketing.
test: ask which parts of the result came from the product or the market rather than the work. a good operator separates these unprompted.
7. The system is invisible
the most consequential omission. if a case study describes outcomes but never the mechanism, you cannot tell whether it repeats. what you want to know is the testing cadence, the kill thresholds, the decision rules.
test: ask what the client was left with after the engagement ended.
Now mine
my headline is: ZuAI from 10,000 to 2,000,000 users in eleven months at a $0.02 blended CAC, on roughly $300k a month.
running the seven tests on it honestly:
starting point: stated, 10,000. good.
timeframe: stated, eleven months. good.
blended: yes, and that is the biggest caveat. blended includes organic, and ZuAI had genuine organic strength because students showed each other results. i did not buy two million users at two cents. i took the full breakdown apart in the $0.02 CAC arithmetic.
attribution: the product was improving during the same period. some of that growth is not mine.
failures: i have run market tests that returned nothing and the honest conclusion was that the market was too thin rather than that we learned something valuable. i also burned 40+ reddit accounts learning the rules, which cost time and traffic.
category advantage: substantial. a consumer AI study app aimed at students is close to the easiest possible case for social distribution. the method transfers; the number does not.
the system: creative volume above the industry benchmark, kill thresholds written before spend, paid budget only following proven creatives. that part is the transferable content, and it is the part worth asking me about.
Why publish this
because the alternative is a headline that performs well in a search result and then underdelivers in a first conversation.
if a case study cannot survive its own author auditing it, it should not be the basis of a hiring decision. mine survives with real caveats attached, which is the most anyone should claim.
if you want to run these seven tests on my work directly, let’s talk. that is a good use of a free call.