Skip to main content

Guided Path

Guided Path (the Guided tab in the Research Path zone, Ctrl+3) turns a plain-language description of your study into a recommended, executable analysis plan with academic citations. You describe the data and the question; Statios produces a pre-mapped sequence of analysis steps — screening, assumption checks, the primary test, post-hocs, effect sizes, reporting — each step annotated with why it is there and which published methodology source says so. Running a step executes the real engine handler and grafts the result into your Research Path; assumption outcomes (a failed Shapiro–Wilk, for example) automatically light up the pre-planned alternative branch.

The output is a recommended path with sources — never "the correct answer." Guided Path is decision support: it makes the defensible options and their literature visible. It is not a substitute for statistical review, and for publication-critical work a statistician or methodologist should still review the plan.

The AI never chooses methods​

Guided Path is built on a strict separation:

  • The AI only interviews. The chat asks clarifying questions ("Were the before and after numbers collected from the same patients?") and translates your answers into a typed study profile — goal, outcome type, pairing, group sizes, and so on. Its replies are validated against a strict schema; any method advice it tries to smuggle into the profile is stripped. Every profile field stays visible as an editable chip, so you can correct it at any time — corrections re-plan instantly.
  • Versioned rules decide. The recommendation comes from the Methodology Knowledge Base (MKB): a registry of decision rules formalized from published method-selection literature (test-selection tutorials, canonical textbooks, reporting guidelines, primary method papers). Rule matching is deterministic — the same profile always produces byte-for-byte the same plan, with or without any AI provider configured.

Where the literature disagrees with itself (classical assumption-pretest ladders vs. robust-by-default Welch tests; Cronbach's alpha vs. McDonald's omega; the Mauchly pre-test vs. always reading the corrected F), the MKB encodes both positions. The planner selects one arm deterministically and presents every other matching arm as a strength-marked, cited alternative — the recommendation never pretends the field is unanimous.

The full registry is reviewable as a printable rulebook: docs/guided-rulebook.md in the source tree, regenerated from the registry by python scripts/generate-guided-rulebook.py, with every rule's conditions, steps, handlers, rationale and full references, plus a reviewer sign-off section. A CI test fails if the committed rulebook ever drifts from the registry.

What MKB v1 covers​

The current knowledge base (MKB 0.2.0, 70 rules, ~120 cited sources) covers:

  • Group comparisons — 2 and k groups, paired and independent, the parametric and rank-based ladders, permutation tests, one-sample comparisons, factorial (two-way) designs, ANCOVA as a GLM;
  • Repeated measures — paired t / Wilcoxon, repeated-measures ANOVA with sphericity handling, Friedman, mixed (between×within) designs via linear mixed models, McNemar / Cochran's Q for repeated binary outcomes;
  • Association / correlation — Pearson, Spearman, Kendall, point-biserial, partial correlation, correlation-matrix multiplicity, non-monotone description;
  • Chi-square family — independence, the N−1 and Fisher's-exact small-sample arms, goodness-of-fit, ordered-category trend;
  • Linear regression — diagnostics-after-fit doctrine (heteroscedasticity, functional form, residual normality, influence, multicollinearity), sample-size planning, hierarchical vs. stepwise model building, clustered data;
  • Logistic regression and GLM outcomes — binary, ordinal (proportional odds), multinomial, count outcomes (Poisson → negative binomial → zero-inflated), events-per-variable screening, fit assessment;
  • Reliability and factor analysis — alpha/omega/Spearman–Brown, EFA suitability and retention (parallel analysis), EFA vs. PCA, CFA, EFA/CFA sample splitting.

Universal screening rules (descriptives, plots-first, missingness, outlier flags) are prepended to every plan.

What it does NOT cover (honest disclosure)​

Profiles outside the rules' scope get an empty plan, not an invented one. Survival analysis, time series, causal inference, SEM and full multilevel-modeling doctrine are v2 scope — the engine has handlers for many of these, but no reviewed methodology rules yet, so Guided Path stays silent rather than guessing.

Within the v1 domains, these are consciously out, even though related engine handlers may exist:

  • Equivalence / non-inferiority testing (TOST) — v1 rules test only difference hypotheses; absence of evidence is not treated as evidence of absence, and no TOST path exists.
  • MANOVA — only thin coverage; Box's M, robustness doctrine and the follow-up question (descriptive discriminant analysis vs. univariate ANOVAs) are deferred.
  • Bayesian alternatives — bayes.* handlers exist, but the frequentist/Bayesian fork carries no methodology rules yet.
  • Missing-data remedies — screening rules quantify missingness and test MCAR, and multiple imputation is flagged as the standard remedy, but there is no imputation-strategy advice (which variables, how many imputations, pooling rules).
  • Item response theory, measurement invariance, DIF — scale-development rules stop at classical test theory (alpha/omega/EFA/CFA).
  • Survey weights and design effects — survey.* handlers exist; no rule considers sampling design in test choice.
  • Study-wise multiplicity planning — rules adjust within a family (post-hocs, correlation matrices) but there is no alpha-spending logic across analysis families.

Additional gaps disclosed inside individual rules (as rationale text, not as planned steps): power analysis as a planning workflow, agreement analysis (ICC/Bland–Altman/kappa as distinct from correlation), trimmed-means robust tests beyond Welch, exact small-sample p-values beyond Fisher/permutation, and paired k-level categorical outcomes (Stuart–Maxwell).

How the recommendations are validated​

Three independent layers:

  1. Scenario fixtures. A committed benchmark suite of study profiles with literature-expected plans (python/tests/guided_fixtures.json). Every selectable rule is the selected rule of at least one realistic scenario; every branch arm (the on-fail and on-pass routes) is pinned; rules that can never win selection (because their documented conflict partner always outranks them) are pinned as alternatives plus step-inventory goldens. The planner must reproduce all of it byte-for-byte in CI, and a key-order-shuffled profile must yield the identical plan (determinism).
  2. Per-rule mutation testing. CI corrupts every rule in the registry, one at a time — flipping a matching condition, dropping a recommended step — and asserts that at least one fixture fails for every corruption. "Every rule is pinned by a benchmark" is a tested claim, not a slogan.
  3. Engine validation. The handlers that execute the plan steps are the same ones validated against R in the golden fixture suite — see Validated Analyses for the exact tolerance commitments and scope limitations.

What validation does not claim: the fixtures pin the planner to the corpus's reading of the literature — they cannot prove that reading is the only defensible one. That judgment is what the rulebook review by a human methodologist is for, and why disagreements are resolved as rule edits with new sources, in version-controlled history.