How many creators before you judge the channel

judge the channel on a batch, never on a handful. five creators is noise: one lucky video or five quiet ones tells you nothing about whether seeding works for your app. run a real batch, score the batch’s combined output against what it cost, and only then decide. individual creators miss constantly inside a channel that is working.

Why a handful is noise

short video outcomes follow a power law, so small samples lie in both directions. in any honest batch, most videos land near zero, a few do fine, and one or two carry the entire result. sample five creators and you are rolling dice on whether a carrier lands in your sample. hit one and you will overbuild the channel on luck; miss and you will kill a channel that was one batch away from working. neither conclusion came from evidence. the distribution is the reason agencies can show you a highlight reel for any product: someone’s video always pops. your question is whether the batch pays, not whether a video can pop.

Test in batches, judge the batch

the unit of decision is the batch: a set of creators booked in the same window, measured over the same period, judged as one line item. i test 150+ creatives monthly on the paid side, and the same rule holds there: no single ad is ever the verdict, the batch is. for seeding, book enough creators that one outlier cannot dominate the read, run them inside a tight window so the comparison is fair, and total it: what did the batch cost, what did it produce in views, installs and comment quality, against your baseline.

then run a second batch before any big conclusion, with the sourcing filter tightened by what the first batch taught you. two batches is the minimum for separating “the channel fails” from “my first sourcing pass was bad”, and those are very different problems with very different fixes. the batch logic is the same discipline i wrote up for paid creative in the creative testing system, applied to people instead of ads.

What early signal looks like

early signal is any video meaningfully above that creator’s own baseline, plus the right kind of comments. specifically:

• a seeded video outperforming the creator’s recent median. their baseline is the control group you get for free; beating it means your product added energy to their content.
• comments asking what the app is called, or tagging a friend. that is purchase-adjacent attention, and it shows up before installs do.
• saves and shares outrunning likes. people file away things they intend to act on.
• an install bump lining up with a posting window, even a small one you cannot fully attribute yet.

what is not signal: raw view counts on their own, likes, or how polished the video looks. a beautiful video with silent comments is a portfolio piece, not a channel.

give the batch a fixed reading window too, and score every video at the same age, day 7 or day 14 after posting. short video keeps accruing views for weeks, so comparing a three-day-old post against a three-week-old one inside the same batch quietly rigs the verdict.

The kill criteria

decide the kill line before the first batch, because after the batch you will negotiate with yourself. a workable rule: two consecutive batches with zero videos above creator baselines and no detectable install movement means stop and change something structural, the niche, the brief or the product’s demo-ability, before spending on batch three. a channel that produces no signal twice is telling you something no third batch will unsay.

the other kill criterion runs the opposite direction: kill individual creators fast, even in a winning channel. a creator with two misses is off the roster candidates list without ceremony, because the batch only pays if the misses stay cheap. keep the channel alive on batch evidence, keep the roster alive on individual evidence, and never confuse the two decisions.

what counts as “detectable install movement” depends on measurement you set up before the first post went live. that setup has its own page later in this playbook; without it, every kill decision is a guess.

this is one piece of the creator seeding playbook. the breakdown call is free. Let’s talk Growth.