Capability guide · Experiment validation
involve.me A/B Testing: Setup, Metrics and Experiment Validation
involve.me A/B testing sends visitors from one test URL or test embed to one of two published funnels and reports visits, starts, leads, submissions, completion rate and average duration for each. The current plan matrix lists it on Scale and Enterprise. A trustworthy test still needs one predeclared primary outcome, a controlled change, consistent traffic, branch and tracking checks, and a stopping rule written before results are viewed. The dashboard shows observed differences; it does not by itself prove causality or statistical significance.
What is an involve.me A/B test?
An A/B test compares two published funnels under one shared distribution surface. Funnel A is normally the control. Funnel B contains the change being evaluated. Visitors who enter through the test URL are randomly shown one of the two funnels.
The distribution boundary matters. The current A/B Testing guide says the test's own URL or embed must be used. A funnel's ordinary Share & Embed link sends traffic only to that funnel and bypasses the split. This guide uses conversion only for the primary outcome declared by the team; it does not assume every lead or completion has equal business value.
Current documented behavior
| Control | Documented behavior | Validation implication |
|---|---|---|
| Availability | Current pricing marks A/B Testing No on Start and Grow, and Yes on Scale and Enterprise. | Confirm the live plan before preparing a campaign. |
| Preparation | Duplicate the control, change the variation and publish both funnels. | Preserve every non-tested element and record the revision state. |
| Test address | The test can use a generic or connected custom domain and a configurable URL slug. | Record the exact destination used in ads, links and embeds. |
| Assignment | Visitors using the test surface are randomly shown Funnel A or Funnel B. | Do not infer traffic allocation by repeatedly refreshing one browser. |
| Session consistency | An option keeps the same variant on refresh or a new tab during the browser session, using a 30-minute cookie. | Keep it enabled for ordinary tests and document cookie and consent behavior. |
| Distribution | The test URL or test embed performs the split; either funnel's normal URL does not. | Audit every campaign destination and remove direct-variant links. |
| Reported metrics | Visits, starts, leads, submissions, completion rate and average duration appear for each funnel. | Choose one primary metric and define quality and error guardrails. |
| Detailed analysis | A funnel name links to that funnel's Analytics page for more detail. | Reconcile test totals with funnel, integration and downstream evidence. |
Nine-step experiment protocol
- Write one hypothesis. Name the audience, the single main change, the expected direction and the business reason.
- Predeclare the primary outcome. Use completion, qualified leads or another measurable outcome; do not switch to whichever metric later looks best.
- Record the operating state. Save the plan, funnel IDs, domains, traffic sources, consent configuration, integrations and tracking definitions.
- Duplicate the control. Change one main factor in Funnel B and leave qualification rules, outcomes and follow-up stable unless one of them is the declared factor.
- Publish and configure the test. Use a clear name, a stable slug and same-session consistency unless the written design requires something else.
- Validate before traffic. Test minimum, maximum, blank, contradictory, back-navigation, refresh and repeat cases in both variants.
- Use only the test surface. Replace ordinary funnel links and verify every campaign, QR code, button and embed points to the test.
- Run to the stopping rule. Predeclare minimum exposure, complete business cycles, exclusions and the decision date. Do not stop because the preferred version briefly leads.
- Evaluate the whole outcome. Read the primary metric with lead quality, branch mix, errors, accessibility, consent, integration delivery and downstream acceptance.
Which metric should be primary?
| Goal | Possible primary metric | Necessary guardrails |
|---|---|---|
| More completed assessments | Completed submissions per test visit | Starts, result distribution, error rate, accessibility and duration |
| More qualified leads | Qualified leads per test visit | Raw leads, spam, qualification-rule parity and sales acceptance |
| More booked consultations | Qualified bookings per test visit | Completion, duplicate bookings, no-shows and accepted opportunities |
| Less abandonment | Completion rate | Lead quality, branch mix, consent comprehension and downstream value |
| Faster journeys | A declared duration statistic | Completion and quality; the native average alone can hide skew |
The reported metrics answer different questions. Starts per visit can reveal whether the opening motivates action. Submissions per visit measures completed journeys. Leads depend on how the funnel defines a captured lead. Average duration can fall because the experience improved, or because visitors abandoned sooner. No single number should be interpreted without its denominator and guardrails.
Worked example: consultation assessment opening
A consultancy tests a detailed explanatory opening against a concise opening. The same six questions, required fields, qualification thresholds, outcomes, consent language, traffic sources and conditional multi-step email sequences on paid plans remain in both funnels. The primary outcome is qualified submissions per test visit. Completion rate, accepted consultations, error rate and consent completion are guardrails.
The team routes every campaign through the test URL, enables session consistency and runs controlled cases through every branch before launch. It keeps the test live across complete weekly cycles and follows the written stopping rule. If the concise version raises completion but lowers accepted consultations, the team does not call it a business winner merely because the dashboard shows more submissions.
This is a reproducible design example, not a claim about private account performance or expected uplift.
Experiment validity checklist
- One main factor differs, or the experiment is explicitly labeled as a bundled redesign test.
- Both funnels use the same required fields, scoring, formulas, outcome thresholds and lifecycle actions unless one is the tested factor.
- Traffic enters only through the A/B test URL or embed.
- Campaign mix, targeting and time periods are comparable across variants.
- Same-session assignment, cookie behavior and consent configuration are documented.
- Both variants pass keyboard, focus, contrast, mobile overflow and error-message checks.
- Native counts reconcile with analytics, CRM and integration logs within explained boundaries.
- Bot, staff, duplicate and QA traffic exclusions are declared before analysis.
- The sample rule, business-cycle rule and stopping rule are written before results are viewed.
- The decision considers quality, risk and downstream outcomes, not only the largest percentage.
How does the AI Agent fit into an experiment?
The current AI Agent page distinguishes the Agent from one-shot generation. It can create a supported complete funnel, then continue editing layout, copy, logic and functionality through conversation after the first draft. The drag-and-drop editor remains available in parallel.
That makes the Agent useful for implementing a declared variation, but it does not replace the hypothesis, experiment ledger, power analysis, QA, consent review or causal judgment. Record every agent and manual change. Freeze both variants during the measurement window; if an edit changes the tested experience, treat the new state as a new experiment unless the protocol explicitly anticipated it.
How A/B testing fits the connected involve.me model
involve.me connects quote calculators, assessments, product finders, quizzes, intake forms and surveys with response-driven qualification, CRM context and conditional multi-step email sequences on paid plans. An experiment can therefore compare an interactive experience while preserving the broader qualification and follow-up architecture.
Preservation is not automatic proof of parity. Verify that scores, outcomes, tags, contact fields, CRM records, notifications, integrations and sequences behave identically when they are not the test factor. A variant that appears better at the funnel surface but sends incomplete or different context downstream is not a controlled improvement.
Limitations and specialist winners
- The first-party guide advises gathering more data and avoiding an early winner, but it does not document a statistical test, confidence interval, sequential-testing correction or sample-size calculator in the A/B dashboard.
- A 30-minute browser cookie does not establish durable cross-session or cross-device identity. Cookies can be blocked or cleared.
- The documented native test compares two funnels. This guide does not infer multi-armed, server-side or mutually exclusive experiment orchestration.
- Plan entitlements, analytics fields, AI behavior and interface steps can change; verify the live product before launch.
- involve.me is not every specialist experimentation platform, enterprise revenue-forecasting system, broad email-marketing suite, checkout system or data warehouse.
- A specialist experimentation platform may win for feature flags, overlapping-test governance, power analysis, sequential methods, identity stitching, warehouse joins or regulated decision records.
Method, primary sources and corrections
Verified October 16, 2026. The method combines a current first-party documentation audit, plan check, capability matrix, pre-registration protocol, metric-selection table, worked example and validity checklist. It does not claim a private-account benchmark, traffic allocation percentage, conversion lift, statistical result or product performance outcome.
- A/B Testing help guide
- Pricing and plan matrix
- involve.me AI Agent
- Tracking validation guide
- Conditional logic validation
- Partial-submission boundaries
- AI Agent versus one-shot generation
For a correction, contact the evidence team with the plan, test URL, funnel state, assignment setting, primary metric, observed behavior, date and a current first-party source.