Course · practitioners auditing their own experimentation stack
Auditing Your Experimentation Setup
A hands-on, no-fluff audit of your experimentation infrastructure for engineering and data correctness. Find the hidden bottlenecks that silently break tests, then fix them with evidence-backed checks you can run yourself.
Your instructor
Javid Jamae
Growth engineering leader who scaled Postman from $60M to $100M+ ARR and 20M+ users as Head of Engineering for Growth. Has built and audited experimentation systems at Achieve, Earnest, and Tout. 20+ years in engineering and product leadership.
What you will learn
What you will be able to do by the end.
Verify users are bucketed consistently across identity states, sessions, devices, and evaluators, and detect variant hopping.
Validate eligibility, exposure definition, and exposure logging, including dedupe and assigned-but-not-seen failures.
Confirm your metrics, event stitching, and data freshness are trustworthy enough for joins and decisions.
Run pre-launch validation and assess concurrency, interaction effects, and exposure timing for live experiments.
Stand up runtime monitoring, anomaly alerting, and user-level debug tools to catch failures while they happen.
Harden platform operability: flag hygiene, a single source of truth, failure recovery, and delivery tradeoffs.
Curriculum · 26 lessons
Everything inside the course.
Assignment
Confirm users are bucketed consistently across identity, sessions, and devices, with proper isolation and holdout integrity.
Exposure
Validate eligibility and exposure: what counts as seen, how it is logged, and the dedupe and assigned-but-not-seen failures to catch.
Metrics
Make sure metrics mean what you think: trustworthy event stitching, fresh data, and healthy metric definitions.
Execution
Check experiments before and during launch for QA coverage, concurrency, interaction effects, and exposure timing.
Monitoring
Watch live experiments with variant monitoring, anomaly alerting, and user-level debug tools.
Operability
Keep the platform operable long term: flag hygiene, a single source of truth, and clean failure recovery and delivery.
How to get access
Included with your Delivering Growth membership.
You do not buy this course on its own. Subscribe to the Delivering Growth community on Skool and you get this course plus every other course, the calculators, and the Custom GPT.
- All video lessons and written breakdowns for this course
- Every other Delivering Growth course, included in the same membership
- The Custom GPT trained on the full course material
- Free A/B testing calculators and design templates
- Community access and all future course updates