One A/B Test Doubled Activation at Vidyard — After PMs Stopped Chasing Signups

An A/B test of a new onboarding experience more than doubled Vidyard's activation rate — not because of a growth hack, but because a product manager paused before running experiments and asked why the metric had moved in the first place.

That is the core argument from a first-person account by a Vidyard product manager published on ProductLed. The piece walks through how a product-led SaaS company can chase signups, see retention stall, and still miss the underlying product-market fit problems causing the churn.

The cautionary tale opens with Adam, a fictional CEO of a new SaaS company. His team hired a growth team, set KPIs, and launched a freemium product. Signups went through the roof. Retention did not follow. A product manager named Amy investigated, expecting to surface low-hanging fruit. Instead she found the team, while shipping quick experiments, had left product-market fit problems and UX issues on the table — the actual cause of the retention gap.

Quick wins or cheap growth hacks?

The author draws a hard line between two things teams routinely conflate.

A quick win puts the customer first, takes little time to execute, generates value, and removes a known pain point — think fixing confusing copy, cutting unnecessary steps from a task, or adding a small delight that builds brand affection.

A cheap growth hack puts the business first. It pushes the target metric up immediately but ignores user experience. Examples from the piece: burying a cancel button behind surveys and "are you sure?" dialogs, or tricking users into an annual membership when they wanted one month.

The distinction is the customer, not the speed.

What changed at Vidyard's activation rate?

When the pandemic pushed more users toward video communication, Vidyard saw a surge of signups — and its activation rate shifted. Asked to fix it, the author resisted diving into experiments and instead diagnosed three causes:

• Paid advertising launched with messaging that set expectations the in-product onboarding could not sustain.
• More people worked from home and felt uncomfortable in their new casual environment.
• New users were exploring video without buying into it the way earlier customers had.

The result: more signups, but lower-intent users. The lesson is diagnostic, not tactical. Ask why users behave as the data shows, and whether your solution improves the product daily or merely makes destructive actions harder.

How does R.I.C.E. keep strategy honest?

The R.I.C.E. framework — reach, impact, confidence, effort — scores each idea so teams can sort by highest impact against lowest effort, balancing quick wins with longer-term bets.

The framework has a known failure mode. On first use, grand ideas sink to the bottom of the list because their effort scores are too high. The author's advice: break the problem down and ask whether a minimum viable product can ship ahead of the full vision.

That is exactly what happened at Vidyard. Asked to drive user acquisition, the author doubted pure acquisition would deliver. The team instead targeted a great first-time onboarding experience — harder to sell internally, but better positioned to lift activation and retention.

How do you create buy-in?

To prove the onboarding bet, the team used a no-code tool called app queues, which builds prompts and flows inside a product without development effort. They shipped an MVP of the ideal onboarding flow and ran an A/B test. Activation more than doubled, making every acquisition effort more valuable.

Putting those results on the table convinced the business to fund the area — and to create a dedicated activation team. Evidence first, org chart second.

Why siloed growth teams fail

The author calls tiger teams solely responsible for growth the number one mistake product-led companies repeat. Three failure modes:

• Wrong people at the table — the team sees numbers, not the people behind them.
• Duplicated effort — a disjointed growth team and product team end up building the same thing, costing money and time.
• Broken communication — the resulting disjointed user experience directly damages the metrics you are trying to drive.

Vidyard's current structure is collaborative instead. Squads carry a dev lead, design lead, and product lead, plus partners from marketing, sales, and data. Each squad owns one funnel step: acquisition, activation, conversion, or paid retention. Diverse expertise produces one cohesive user experience; collaboration between squads produces compounding effects, such as marketing building a complementary email flow that pulls users back into an onboarding experience product is building.

Companies that silo see initial success and then dissolve when sustained growth never arrives.

What to do Monday morning

The author closes with three steps before driving any metric upward: put yourself in the customer's shoes and ask why the data looks as it does; question the metric itself and challenge the ask if a different focus is more impactful; and get diverse opinions in the room to collaborate and challenge each other.

Consumers are increasingly aware of cheap wins that create no value, which makes the hacks riskier and value-driven growth more durable. As funding discipline pushes SaaS companies toward retention over raw signup counts, the PMs who diagnose before experimenting will be the ones who keep their funnels compounding.