# InstaBizIntel User Manual Welcome to InstaBizIntel, a self-hosted platform that brings business intelligence, professional statistics, econometrics, structural equation modeling, machine learning, and qualitative research together in one browser-based workspace. This manual teaches you the platform from the ground up. You do not need prior experience with statistical software; every concept is explained, every worked example is spelled out step by step, and every result is interpreted for you. --- ## 1. Introduction ### 1.1 What InstaBizIntel is InstaBizIntel is a complete data-science and research environment that you run on your own server. Think of it as a combination of several well-known tools rolled into a single, consistent web application: | If you have used… | InstaBizIntel gives you… | |---|---| | Tableau / Power BI | Charts, dashboards, cross-filtering, public links | | SPSS / SAS / JMP / StatCrunch | A statistics workbench with 304 analyses in 23 groups | | Stata | Panel-data econometrics (fixed/random effects) | | EViews | Time-series econometrics (VAR, VECM, GARCH, ARIMA) | | SmartPLS | PLS-SEM (structural equation modeling) | | RapidMiner / DataRobot | Visual machine-learning workflows and Auto Model | | NVivo | Qualitative coding and text analysis | | Prophet / EViews forecasting | A Forecasting Lab (decomposition hybrids, deep models, statistical model comparison) | | Covidence / Rayyan / VOSviewer / bibliometrix | A Literature Review + Meta-Analysis suite (search, PRISMA, cluster maps, appraisal, drafting) | | G*Power / statcheck / OSF | A Rigor Guard (power, assumption checks, reporting checklists, reproducibility, preregistration) | | DoWhy / Zenodo / RO-Crate | Causal DAG estimation with refutation tests; Trust Scores, Inspector verification and one-click reproducibility bundles with DOI deposit | | JASP personas / G*Power / Qualtrics | Teaching mode (Socratic tutor), validated survey templates → questionnaire + PLS model, careless-response screens, thesis pipeline | | ChatGPT / Claude connectors | An MCP server exposing the whole platform (35 tools) to any AI client | Everything lives in one place, shares the same data, and is described in the same vocabulary, so a chart, a regression, and a machine-learning model can all be built from the very same dataset without exporting anything. ### 1.2 Who it is for - **Business analysts** who need dashboards and KPIs. - **Students and researchers** running statistical tests, econometrics, or SEM for a thesis or paper. - **Data scientists** building and explaining predictive models. - **Qualitative researchers** coding interviews, reviews, or open-ended survey text. - **Anyone** who wants to ask a plain-English question and let the built-in AI Analyst choose and run the right analysis. Because the platform is self-hosted, your data stays on your infrastructure. Each person works inside private **Projects**, and a shared **Sample Library** lets everyone learn on the same example datasets. ### 1.3 The module map InstaBizIntel is organized into modules. You will meet each of them in this manual: 1. **Projects & Data** — the container for your work and the tools to bring data in and clean it. 2. **Business Intelligence** — charts and dashboards. 3. **Statistics Workbench** — **304 statistical analyses** in 23 groups, organised in the sidebar into 12 families (Descriptive & Exploratory; Compare Groups; Correlation & Association; Regression & Modeling; Multivariate & Classification; Causal Inference; Consumer & Market Research; Time Series & Forecasting; Design & Quality (DOE/SPC); Survival & Clinical; Reliability Engineering; Bayesian), each with relevant visualizations. See the complete catalogue in §22. 4. **Panel Data** — Stata-style fixed/random effects. 5. **Time-Series Econometrics** — EViews-style unit roots, VAR, VECM, plus the full volatility family (ARCH/GARCH/EGARCH/GJR-GARCH), exponential smoothing (ETS), STL and ARIMA. 6. **Regression Variants** — counts, quantiles, instrumental variables, survival. 7. **PLS-SEM** — SmartPLS-class structural equation modeling. 8. **Machine Learning** — a visual node canvas with **50 operators (37 models)** plus one-click Auto Model and an Instant baseline. See the operator reference in §23. 9. **Qualitative Analysis** — NVivo-class coding and text queries. 10. **AI Analyst & InstaGenie** — a central agent (AI Analyst) and an on-every-page agent (**InstaGenie**) that run any of the above from a plain-English or spoken request. 11. **Reports** — a premium document generator that assembles analysis results, charts and PLS diagrams into a branded report and exports to PDF and Word (.docx). 12. **Rigor Guard & the audit trail** — Trust Scores, Inspector verification, reproducibility bundles, executable preregistration, multiverse, causal DAGs, hypothesis tournaments and the publication gate (§§26–30). 13. **Teaching & Thesis** — validated model templates, careless-response screening, sample-size rules, instructor process logs and a proposal-to-viva thesis pipeline (§31). 14. **MCP server** — use everything above from ChatGPT, Claude or any AI client (§32). 15. **Finance Lab** — investment analytics on live market data: performance and risk, portfolio optimisation, technicals, finance-tuned news sentiment, an evidence-linked AI brief, fundamentals with DCF, filing Q&A, event studies and a research note (§34). > **New in September 2026?** Jump to **§26 What's new** for a one-page map of the latest features. --- Also in the sidebar (each has its own section): **Ask Data** and **Decomposition** (§3), **Reports** (§13), **Forecasting Lab** (§17), **Finance Lab** (§34), **Lit Review + Meta** (§18), **Rigor Guard** (§19, §27–30), **Prediction Profiler** (§20), **Teaching & Thesis** (§31), **Connections** (§16) and the **SQL Editor / Alerts** data tools (§24). ## 2. Getting started ![Your projects — each project is a private workspace with its own data, analyses and dashboards.](/manual-img/01-projects.png) ### 2.1 Creating or opening a project A **Project** is a private workspace that holds its own datasets, analyses, dashboards, and exported artifacts. Nothing leaks between projects, so you can keep a class assignment, a client engagement, and a personal experiment cleanly separated. When you open InstaBizIntel you land on the **Projects** screen. 1. To start fresh, click **New Project**, give it a name (for example, `Retail Analytics 2026`) and an optional description, then click **Create**. 2. To continue earlier work, click any project card to open it. Once inside a project, everything you import or build belongs to that project. > **Plans.** Every new account starts a **15-day free trial** of the full platform (no card needed). After that, descriptive statistics, data import, charts and dashboards stay free; inferential statistics, ML, forecasting, PLS-SEM, qualitative and the other premium modules require **Premium at ₹800/month** (30 days per payment, via **Billing**). The gate fires when results are produced, so you can always set an analysis up first. ### 2.2 The workspace shell After opening a project you see the **workspace shell**. It has three parts: - **The left sidebar** groups every feature into sections — **Analyze** (AI Analyst, Ask Data, Decomposition, Charts, Dashboards, Reports), **Research** (Statistics, PLS-SEM, Forecasting Lab, Lit Review + Meta, Rigor Guard, Prediction Profiler, ML Workflows, Qualitative, Teaching & Thesis), **Data** (Datasets, Connections, SQL Editor, Alerts), and **Manage** (Manual, Settings). - **The main canvas** in the center shows whatever you are working on: a data grid, an analysis form, a chart, or a results document. - **The command palette**, opened with **⌘K** (macOS) or **Ctrl+K** (Windows/Linux), lets you jump anywhere or launch any action by typing. Try typing "linear regression", "import CSV", or "new dashboard" and pressing Enter. The palette is the fastest way to move around once you know what you want. ### 2.3 Importing data Every analysis starts with data. From the sidebar choose **Data → Datasets**. The Datasets page shows twelve source tiles — Upload a file, From URL / Google Sheets, From a web page, Cloud storage, Paste / enter data, Documents & transcripts, Market data, Macro · sentiment · on-chain, Research data, Web / YouTube / Reddit capture, Reference library (RIS / BibTeX) and Sample library. The four you will use first are below; §16 covers the rest. #### Upload a file Click **Upload a file** and pick a file. Supported formats: | Category | Extensions | |---|---| | Spreadsheets | `.csv`, `.tsv`, `.xlsx`, `.xlsm`, `.xltx`, `.xls`, `.ods` (multi-sheet workbooks become one dataset per non-empty sheet) | | Statistical packages | `.sav` / `.zsav` / `.por` (SPSS), `.sas7bdat` / `.xpt` (SAS), `.dta` (Stata) — variable/value labels are captured | | Columnar / structured | `.parquet`, `.json` / `.jsonl` / `.ndjson`, `.xml`, SQLite (`.sqlite`, `.db`, `.sqlite3`) | | Delimited text | `.txt`, `.dat`, `.tab`, `.psv` (delimiter auto-detected) | InstaBizIntel reads the file, detects column types, and profiles it automatically on import. Report-style Excel sheets that start with a title banner and use a two-row header (a group label above repeated column names such as CO1–CO6) are recognised: the real header row is found automatically and group labels are folded into the column names (for example `2025-2026 CO Targets (%) CO1`). #### Import from a URL Paste a direct link to a data file (for example a CSV published on the web). The platform downloads and parses it the same way as an uploaded file. #### Import from a connected SQL database Go to **Data → Connections** (editor role or above) and create a connection — PostgreSQL, MySQL / MariaDB, Microsoft SQL Server, ODBC (generic), MongoDB, REST / JSON API or OData feed. Then, on the connection card, pick a table/collection (or write a **custom SQL** query for SQL sources) and give the new dataset a name. The result of the query becomes a dataset. This is ideal when your data already lives in a warehouse. #### Import from the Sample Library The **Sample Library** is a shared collection of ready-made teaching datasets. Click **Sample library** on the Datasets page, pick a dataset, and click **Import**. A copy is placed in your project so you can experiment freely. This manual uses the Sample Library throughout. Here is what each dataset contains: | Dataset | Size | Columns | Best used for | |---|---|---|---| | `sales` | 5,000 rows | order_date, region, category, quantity, unit_price, revenue, cost, profit, channel | BI, descriptives, charts, dashboards | | `panel_firms` | 720 rows (60 firms × 12 years) | firm, year, size, employees, rnd_intensity, marketing, sales_growth | Panel data | | `macro_ts` | 240 monthly rows | date, gdp, consumption, interest_rate, inflation, market_return | Time-series econometrics | | `tam_survey` | 350 respondents | peou1-3, pu1-3, int1-3 (7-point items), age, gender | PLS-SEM | | `sem_satisfaction` | 400 respondents | imag1-3, expe1-3, qual1-3, val1-3, sat1-3, loy1-3 (7-point items), age, gender, segment (New/Returning/Loyal) | PLS-SEM — ECSI-style model, mediation, moderation, multi-group (MGA) | | `reviews` | 120 rows | review (free text), segment, region, rating | Qualitative | | `clinical` | 300 patients | months, event (1=event, 0=censored), age, treatment, biomarker, sex | Survival analysis | | `experiment` | 180 subjects | method (Control/Method A/Method B), score, site, pretest | Compare means / ANOVA | | `churn` | 2,400 customers | tenure_months, monthly_charges, total_charges, contract, internet_service, payment_method, tech_support, senior_citizen, churn | ML classification, Auto Model | | `housing` | 1,600 homes | living_area_sqft, bedrooms, bathrooms, house_age_years, quality_score, garage_cars, neighborhood, sale_price | ML regression | | `credit` | 2,000 loans | annual_income, loan_amount, debt_to_income, credit_score, employment_years, loan_purpose, home_ownership, defaulted (~18%) | Imbalanced classification, ROC/AUC | | `iris_species` | 210 flowers | sepal_length, sepal_width, petal_length, petal_width, species | Multi-class classification, clustering | | `heart` | 320 patients | age, sex, chest_pain_type, resting_bp, cholesterol, fasting_bs, resting_ecg, max_heart_rate, exercise_angina, st_depression, st_slope, vessels_colored, thalassemia, heart_disease | Classification, Auto Model | | `segments` | 600 customers | age, annual_income_k, spending_score, visits_per_month (unlabeled) | Clustering (K-means/DBSCAN/GMM) | | `wine` | 1,200 wines | alcohol, volatile_acidity, sulphates, citric_acid, residual_sugar, ph, density, quality | Regression, feature importance | The `churn`, `housing`, `credit`, `iris_species`, `heart`, `segments` and `wine` datasets are the **Machine learning** category — ready targets for the ML Workflows canvas, Auto Model, and the worked examples in §15. ### 2.4 The data grid Open any dataset from **Data → Datasets** to see the **data grid** — a spreadsheet-like view of your rows and columns. The **Preview** tab shows the first 100 rows; long cells expand on click. Column types (shown on the **Profile** tab as DuckDB types such as VARCHAR, BIGINT, DOUBLE, DATE, TIMESTAMP, BOOLEAN) are detected on import. To sort, filter or reorder columns use the **Transforms** tab. #### Dataset tabs and actions Every dataset page has seven tabs — **Preview**, **Insights**, **Profile**, **Transforms**, **Semantic**, **Row security**, **Refresh** — and four header buttons: **Download CSV**, **Synthesize**, **New chart**, **Delete**. - **Insights** — one-click *Key findings*: automatically computed bullet insights about the dataset, with an AI summary button. - **Semantic** — define named **measures** (e.g. `sum(profit) / sum(revenue)`), **hierarchies** (drill-down paths such as year → quarter → month) and field **synonyms/descriptions** that Ask Data, InstaGenie and the chart builder use; **Import OSI** merges an Open Semantic Interchange / dbt-metrics / LookML-style YAML/JSON model. - **Row security** — per-dataset row-level security rules (`column op value`, where value may be `{{attribute}}` resolved from each member's attributes set in Settings; unresolvable attributes fail closed). Saving requires the admin role. - **Refresh** — a cron schedule (presets from every 15 min to weekly, or custom), a **Refresh now** button and a refresh-run history. Uploaded files re-run their transform steps; database/REST sources re-import first. No daily cap. - **Synthesize** — creates a privacy-preserving synthetic twin (Gaussian copula) with a chosen row count and a fidelity & privacy report, for teaching and sharing (§31.6). ### 2.5 Profiling Every dataset is profiled on import, giving you an instant summary. For each column you get: - Its type and how many values are missing. - For numbers: minimum, maximum and a small histogram (mean/SD come from the Descriptive analyses or the Insights tab). - For text/categories: the number of distinct values and the most frequent ones. - For dates: the earliest and latest values. Profiling is the first thing to check with any new dataset. It tells you what you are working with and flags problems (unexpected missing values, a numeric column that imported as text, outliers) before they derail an analysis. ### 2.6 The transform pipeline Real data is rarely clean. The **transform pipeline** lets you reshape a dataset through an ordered list of **steps**. Open a dataset and choose the **Transforms** tab. Steps are added from a Power-Query-style ribbon grouped into **Manage columns**, **Reduce rows**, **Transform**, **Group & reshape**, **Combine queries** and **Data science**; applied steps are listed on the right. Each step is applied to the output of the previous one, so the order matters and the whole recipe is repeatable and reversible. Available steps (25): | Ribbon group | Step | What it does | |---|---|---| | Manage columns | **Remove columns** | Drop columns | | Manage columns | **Rename** | Change a column's name | | Manage columns | **Change type** | Cast to VARCHAR, INTEGER, BIGINT, DOUBLE, DECIMAL, DATE, TIMESTAMP or BOOLEAN | | Manage columns | **Add column** | Calculated column from a SQL expression | | Manage columns | **Index column** | Add a running row number | | Reduce rows | **Filter rows** | Keep rows meeting a condition | | Reduce rows | **Sort** | Order rows by one or more keys | | Reduce rows | **Remove duplicates** | Dedupe on all or chosen columns | | Reduce rows | **Keep N rows** | Keep the first N rows | | Reduce rows | **Bernoulli / Poisson sample** | Include every row independently with probability *p* (Bernoulli), or with probability proportional to a size column such as invoice amount (Poisson / PPS). Adds `inclusion_prob` and `ht_weight` (= 1/π) so totals can be estimated without bias; reproducible via a seed. Pair it with **Statistics → Audit sampling — Horvitz-Thompson estimates** | | Transform | **Split column** | Split one column into several on a delimiter | | Transform | **Merge columns** | Concatenate columns with a delimiter | | Transform | **Replace values** | One step holds any number of find → replace rules across one or several columns (substring or whole-cell); rules on the same column run in order and earlier rules are kept when you add more | | Transform | **Format text** | Trim, case changes and similar text operations | | Transform | **Extract** | Text before/after a delimiter or first N characters | | Transform | **Fill down/up** | Propagate the last non-null value | | Group & reshape | **Group by** | Aggregate rows (sum, avg, count…) | | Group & reshape | **Pivot** | Turn a name/value pair into wide columns | | Group & reshape | **Unpivot** | Turn wide columns into tall key/value rows | | Combine queries | **Merge (join)** | Join another dataset on a key (left/inner/…) | | Combine queries | **Append (stack)** | Union rows from another dataset | | Data science | **Impute missing** | Fill nulls with mean, median, mode or a constant | | Data science | **Cap outliers** | Clip values to mean ± k·SD (default k = 3) | | Data science | **Standardize** | Rescale to mean 0, SD 1 | | Data science | **Encode category** | Integer-code a categorical column | Each step shows a live preview. When you are happy, click **Apply & save**; the transformed dataset becomes available to every module. A common beginner recipe is: **Retype** a date column, **Filter** out test rows, **Impute** a few missing numbers, then **Calculated column** to add a metric you need. --- ## 3. Business intelligence The BI module turns rows of data into charts and dashboards that anyone can read. ![The Charts builder — 24 visual types with a live preview, analytics overlays and per-column conditional formatting.](/manual-img/70-charts.png) ### 3.1 Chart types InstaBizIntel offers 24 chart types: bar, line, area, combo (bar + line), pie, donut, scatter, table, KPI, waterfall, funnel, gauge, treemap, heatmap, histogram, box plot, pivot table, geographic map, radar, sankey, sunburst, bubble, ribbon and 3-D scatter. You pick **Dimensions** and **Measures** (with an aggregation: sum, avg, min, max, count, count_distinct — or a saved semantic measure) from dropdowns and the preview redraws live. An **Analytics** panel adds trend lines, moving averages and reference lines; tables support per-column conditional formatting; a **Drill-down hierarchy** can be attached to bar/line/area/combo/pie/donut/treemap charts. #### The Format pane (Power BI / Tableau-style customisation) Below **Options** in the builder sits a **Format** card. Everything in it is saved with the chart, so dashboards, reports, PNG/PDF exports and public links all honour it. Sections open and close by clicking their heading: | Section | What you can set | |---|---| | **Title & text** | Chart **title**, **subtitle** and a **caption / source note**; title alignment, size and colour; base font size | | **Colours** | 15 palettes — InstaBizIntel, Vibrant, Cool, Warm, Earth, Monochrome, **Power BI classic**, **Tableau 10**, **Office / Excel**, Pastel, **Colour-blind safe (Okabe–Ito)**, Grayscale (print), Viridis, Red–Blue diverging, Corporate navy — plus a **custom colour list** (click *copy palette to edit*, then change any swatch with the colour picker or a hex code), **per-series / per-slice colours** (each series, category value or pie slice gets its own picker), a single colour for one-series charts and KPI tiles, and a chart **background** colour | | **Axes** | X and Y **axis titles** (second Y title for dual-axis combos), Y min / max, category-label rotation, hide gridlines, logarithmic axis, hide category labels, **horizontal orientation** for bar/line/area | | **Legend** | Top, bottom, left, right or hidden | | **Data labels** | Show/hide, position (outside end, inside, inside end, inside base), label size; **stack totals** on stacked bars; pie/donut slice labels as name, percent, value or combinations | | **Series style** | Bar width and **rounded bars**; smooth / step lines, point markers, line width, area opacity; **gradient fill**; sort bars by value | | **Pie / donut** | Inner radius, rose (Nightingale) style, **centre text** for donuts | | **KPI tile** | Prefix / suffix (e.g. ₹, %), value size and colour | | **Table style** | Striped rows, **totals row**, compact rows, header colour and text colour, **rename column headers** — in addition to the existing data bars, colour scales, icon sets and rules | | **Friendly names** | The *rename column headers* map (`columnLabels`) also renames chart series in legends and tooltips, radar axes and bubble/scatter axis titles — so `sum_GrossAmount` can read **Revenue** everywhere. A chart that carries its own title no longer repeats the tile name on shared dashboards. | | **Decimals** | Fixed number of decimals for axes, labels, tooltips and tables (works with the Number-format setting) | **Reset formatting** clears the Format keys but keeps query-level options (filters, stacking, hierarchy, conditional formats). ### 3.1d Map visualisations — filled, bubble and flow maps Chart type **Map** now has three modes (Format pane → *Map*): | Mode | Query | What it draws | |---|---|---| | **Filled map** (choropleth) | dimension = state / country name, measure = value | Colours each region by the measure. Regions: **India** (36 states & union territories; spellings such as *Orissa*, *Uttaranchal*, *Jammu & Kashmir*, *Pondicherry* are matched automatically) or **World** (countries). *Auto* picks India when the names are Indian states. Click a state to cross-filter the dashboard. | | **Point / bubble map** | dimensions = name, latitude, longitude; measures = size (1st), colour (2nd) | One bubble per city, store, branch or site. Latitude/longitude columns are detected by name (*Latitude*, *lat*, *StoreLat* …) or chosen explicitly. Optional ripple animation and point labels. | | **Flow map** | dimensions = origin name, origin lat, origin lon, destination name, destination lat, destination lon; measure = flow weight | Animated origin → destination lines (width = weight) with destination bubbles sized by total inflow — delivery networks, migration, trade lanes. | Other map options: show region names, pan & zoom, zoom level, a three-stop colour scale, empty-area colour, maximum bubble size; the Number format (₹, K/M, decimals) applies to the legend and tooltips. Maps work in dashboards, public links and Data Stories exactly like any other chart, and respect dashboard filters routed through the Data model (e.g. a *Zone* filter on a state dimension reaches the city, store and shipment maps). ### 3.1a Ask Data (natural-language charts) **Analyze → Ask Data** lets you type a question such as *Total sales by region* or *Monthly trend of orders* against a chosen dataset; InstaBizIntel picks the chart type, builds the query, explains its choice and renders the result. Click **Save chart** to keep it for dashboards. ### 3.1b Decomposition tree **Analyze → Decomposition** breaks one measure (sum/avg/count of a column) down one dimension at a time, Power-BI style. Click a bar to drill into it, add the next level yourself, or press **AI split** to let the platform choose the dimension that explains the most variation. **Reset tree** starts over. ### 3.1c Filters — the Tableau / Power BI filter set Every chart can carry any number of filters (builder → **Filters → + filter**). Pick the field (including related-table fields such as `Products › Category`), then the filter **type**: | Type | What it does | Tableau / Power BI equivalent | |---|---|---| | **Pick values (list)** | Tick values from the column's distinct list (with search, all/none), or type them; **include** or **exclude** the selection | General filter · Basic filtering | | **Compare value** | =, ≠, >, ≥, <, ≤ a value | Condition · Advanced filtering | | **Range** | between two values or dates; leave one end empty for *at least / at most* | Range of values · Between | | **Wildcard / text** | contains, does not contain, starts with, ends with, wildcard (`North*`, `?ast`), regular expression — case-insensitive | Wildcard filter · Contains / Starts with | | **Null / blank** | is null, is not null, is blank (null or empty), is not blank | Special · Is blank | | **Top N** | Top or bottom *N* items, or top *N %* of items, of the field **by** an aggregate (sum, avg, count, count distinct, min, max) of another field, e.g. *top 10 products by sum of revenue* | Top filter · Top N | | **Condition (by field)** | keep the field's members whose aggregate meets a test, e.g. *regions where sum(profit) > 1.5M* | Condition → By field | | **Relative date** | last / next *N* days, weeks, months, quarters or years; this period; previous *N* complete periods; to date (YTD, QTD, MTD); optional anchor date instead of today | Relative date filter · Relative date | | **Advanced (AND / OR)** | several conditions on the same field combined with ANY (OR) or ALL (AND) | Power BI Advanced filtering | | **Measure result (after aggregation)** | filter the chart's own aggregated measure, e.g. *sum_revenue > 6M*, applied after grouping (HAVING) | Measure filter · Visual-level filter | Filters combine with AND. Each filter row shows a one-line summary of what it does. The same filter vocabulary is available to dashboards: the dashboard filter bar offers **list**, **multi-select**, **date range**, **Top N** (viewers choose N, top/bottom, count or %, and the measure), **relative date** and **text** kinds, and all of them work on public share links and flow across related tables through the Data model (§3.6). ### 3.2 Worked example: revenue by region (bar chart on `sales`) Let us build your first chart. 1. Make sure `sales` is in your project (add it from the Sample Library if not). 2. In the sidebar choose **Analyze → Charts → New chart** and pick dataset `sales`. 3. In the chart builder, set **Chart type** to **Bar**. 4. Under **Dimensions**, choose **region**. 5. Under **Measures**, choose **revenue** and set the aggregation dropdown to **sum**. 6. The preview renders automatically; click **Save chart** when you are happy. You now see one bar per region, each bar's height equal to the total revenue for that region. A realistic result looks like this: ``` Region Total revenue East $412,500 West $388,900 Central $301,200 South $274,600 ``` **How to read it.** The tallest bar (East) is the region contributing the most revenue; the shortest (South) the least. Because we summed revenue, the chart answers "where does our money come from?" To answer "where is each order biggest?" you would switch the aggregation from **Sum** to **Average**, which usually reorders the bars. Sort the bars descending (the **Sort & limit** panel) so the ranking is obvious at a glance. Save the chart with a name like `Revenue by Region`. Saved charts can be reused on dashboards. ### 3.3 Building a dashboard A **dashboard** is a canvas of several charts and metric tiles shown together. 1. Choose **Analyze → Dashboards**, type a name in **New dashboard name** and click **Create**. 2. Pick a saved chart from the **Add chart…** dropdown and click **Add** (charts are built in the Charts builder or Ask Data, not inline). 3. Drag the tile's corner to **resize** it, and drag its body to **reposition** it. Drag the ⠿ header to move a tile; dashboards use a 12-column snapping grid so tiles align neatly. 4. Add a few more tiles — for example `Revenue by Region`, a line chart of revenue over `order_date`, and a KPI tile showing total profit. Two further toolbar controls: an **auto-refresh** dropdown (No auto-refresh / 30s / 1m / 5m) re-queries every tile on a timer, and **History** opens a governed version history — every save snapshots the previous layout (last 25 kept) with a diff of added/removed/moved tiles, and any version can be restored (the restore itself is versioned). Global dashboard **filters** (dropdown or multi-select per field) sit above the grid. ### 3.4 Linked exploration — cross-filtering, multi-select & highlight (brushing) Dashboards support **linked exploration** across every tile — the professional, JMP/Tableau-style capability where a selection in one chart drives the others. A **Filter / Highlight** toggle in the dashboard toolbar chooses how the selection behaves. - **Cross-filtering (Filter mode)**: click an element (say region *South*) on any chart and every other tile instantly **filters** to that selection — the other charts re-aggregate to just those rows (notice their axes rescale). A `region = South ✕` chip shows the active selection; click ✕ to clear. ![Filter mode — selecting South filters every tile to South (the contract and churn charts re-aggregate and rescale).](/manual-img/111-brush-filter.png) - **Multi-select**: click several elements on the same field to build a set — the chip shows `region ∈ {South, West}` and all tiles filter to that set (an `IN` filter). Click a selected element again to remove it. - **Highlight / brushing (Highlight mode)**: switch the toggle to **Highlight** and the same click **keeps all data visible** and instead **dims the non-selected marks** across every chart grouped by that field — so you see the selection *in context* within the full distribution, simultaneously, on every linked chart. This is classic linked brushing. ![Highlight mode — selecting South highlights it in context across every region-grouped chart while the others keep full context.](/manual-img/112-brush-highlight.png) - **Bookmarks**: save the current selection (filters + cross-selection + mode) as a named view to return to it in one click. - **Drill-through**: use the ⌕ button on a tile to open the underlying rows behind it; charts with a **Drill-down hierarchy** set in the builder (from the dataset's Semantic model, e.g. year → quarter → month) also drill down on click. ### 3.5 Publishing a public read-only link When a dashboard is ready to share: 1. Open the dashboard and click **Share**, then **Create link**. 2. InstaBizIntel generates a **public read-only URL** automatically. Anyone with the link can view but not edit. The shared dashboard is **fully interactive**: viewers can change the dashboard filters, **click a bar, slice or point to cross-filter every tile** (click again to deselect, several values can be selected), switch between **Filter** and **Highlight** (linked brushing) and **Reset view**. Their selections are never saved, so every viewer explores their own copy; PDF/PNG exports and email subscriptions still capture the dashboard as published. 3. Optionally type a password in **Optional password for new link** before creating it; protected links show a 🔒. You can also add **Email subscriptions** that send the dashboard as a PDF on a cron schedule (needs SMTP on the server). 4. Use the **PNG** / **PDF** toolbar buttons to download the dashboard for reports and slides. > Publishing makes a dashboard visible to anyone with the link. Only publish data you are comfortable sharing, and add a password for anything sensitive. ### 3.6a Data Stories — sequenced, publishable narratives **Where:** Analyze → **Data Stories**. A story is an ordered set of **scenes** built from your saved dashboards and charts — the equivalent of a Tableau *story* or Power BI *bookmark* deck, with a few things they cannot do: every scene stays **live and clickable**, the same chart carried across consecutive scenes **morphs** between filter states, captions can be **drafted by AI from the scene's own data**, and the whole story publishes as a **public link**, a **reading page**, a **PDF**, a **PNG** or a **PowerPoint** file. | Scene type | What it shows | |---|---| | **Title** | Opening / section card (title, subtitle, callout badge, author & organisation from Story settings) | | **Dashboard** | A saved dashboard with a **saved filter state** (every filter kind incl. Top N and relative dates), an optional **click selection** (cross-filter or highlight) and **spotlight tiles** — only the spotlighted tiles are shown, re-flowed to fill the scene; other tiles can be hidden | | **Chart** | One chart, large, with its own filters and a caption panel (right / left / bottom / hidden) | | **Compare** | Up to three charts rendered under **two filter states side by side** (e.g. *2023 vs 2024*, *North vs South*, *before vs after promotion*) | | **KPI strip** | A row of KPI tiles or small charts with shared filters | | **Text** / **Image** | Narrative-only (Markdown) or a picture with caption | **Editor.** Scenes on the left (reorder, duplicate, delete, *+ Add scene*), a live preview in the centre (click charts to test cross-filtering), the inspector on the right: title, **callout badge**, **caption** (Markdown) with **✨ AI caption from the data** — the assistant runs the scene's charts with the scene's filters and writes 2–4 grounded sentences — **speaker notes** (with ✨ AI notes), caption position, transition (fade / slide / zoom), autoplay seconds and background. *Story settings* hold the subtitle, accent colour, author, organisation, default transition and autoplay timing. **Save** snapshots the previous version (**History** → restore, last 25 kept). **Presenting.** ▶ **Present** opens the full-screen player: `←`/`→`/space to move, `F` fullscreen, `N` speaker notes, `P` autoplay, `Home`/`End`. A progress bar and dot navigator sit at the bottom. **Sharing and downloading** (toolbar, like dashboards): **Share** creates public links (`/s/`, optional password) that open in Present mode with a **Read** toggle for the scrollytelling page (`?mode=scroll`) and `?notes=1` for notes; viewers can still click charts to cross-filter. **PDF** exports one 16:9 page per scene, **PNG** the whole reading page, **PPTX** one slide per scene (rendered scene as the picture, caption/notes as speaker notes) — ready for PowerPoint or Google Slides. *API:* `GET/POST /api/workspaces/{ws}/stories`, `PUT/DELETE …/stories/{id}`, `…/bundle`, `…/share`, `…/narrative`, `…/export?format=pdf|png|pptx`, `…/versions`; public `GET /api/public/stories/{token}` and `POST …/charts/{chart_id}/data`. ### 3.6 Data model — relationships between datasets (schema diagram) Open **Data → Data model**. Like Power BI's model view, it shows every dataset in the project as a box with its columns and lets you draw **relationships** between them on key columns (for example `Orders.region_id → Regions.region_id`). A multi-sheet workbook becomes one dataset per sheet, so a typical model is a fact sheet linked to several lookup sheets. - **Auto-detect relationships** scans all datasets for matching key columns (same name, or `_id` patterns), measures how many values match and whether each side is unique, and proposes relationships with a cardinality (*:1, 1:1, 1:*, *:*) and a confidence. Accept them one by one or all at once. - **Rename datasets** from the diagram (✎ on a box): a multi-sheet workbook arrives as ` — `, and short names such as *Sales* or *Products* keep legends readable; charts using the old name are updated automatically. **Auto-arrange** lays the most-connected (fact) tables in the centre with lookups around them. - **Add a relationship** by hand: pick the two datasets and columns, click **Check match** to see the match percentage, unmatched rows and the suggested cardinality, then **Add**. Relationships can be deactivated or removed; drag boxes to arrange the diagram. **Save model** stores everything. - **Using related fields.** In the chart builder a **Fields** pane lists the base table (⌂) and every related table (🔗) with their columns, like Power BI's Fields list: hover a field and add it as a dimension, measure or filter. The dimension, measure and filter pickers also gain groups such as *Related: Regions*; fields are written **Regions › region_name**. If you pull fields from a table on the *many* side, the builder offers **make … the base table**, which rewrites the chart so the fact table drives the aggregation. The query engine joins along the shortest relationship path automatically, so one chart can combine columns from several tables. A warning appears when you pull fields from the *many* side of a relationship, because rows then repeat and sums can inflate. - **Cross-table dashboard filters.** A dashboard filter or click-selection on a column that exists only in a related dataset is pushed through the relationship to every chart, so cross-filtering works across tables, including on public share links. - The AI analyst and InstaGenie can use related fields too, with the same `Dataset › column` syntax. --- ## 4. Statistics workbench The statistics workbench is the heart of the platform: **304 analyses** grouped by purpose — a library on par with SAS, SPSS, JMP and StatCrunch combined. The workflow is identical for every one of them, so once you learn it you can run any analysis. Most analyses also emit a **relevant visualization**, and every result can be sent to a report (§13.3). The complete list is in §22; representative worked examples with real output are in §15. ### 4.1 The universal workflow 1. **Open the workbench** (sidebar → **Research → Statistics**) and pick a dataset. 2. **Pick an analysis** from the two-level sidebar. The 304 analyses are organized into thirteen collapsible **families** — *Descriptive & Exploratory*, *Compare Groups*, *Correlation & Association*, *Regression & Modeling*, *Multivariate & Classification*, *Causal Inference*, *Consumer & Market Research*, *Time Series & Forecasting*, *Design & Quality (DOE / SPC)*, *Survival & Clinical*, *Reliability Engineering*, *Bayesian* and *More methods* (multivariate extras such as CATPCA, TwoStep cluster, DEA, topic models, fsQCA) — with a search box that filters all of them. Click a family to expand its groups. 3. **Assign variables** by dragging columns into role slots (for example *Dependent*, *Factor*, *Test variable*, *Grouping variable*). The form only lets you drop a column into a role it fits. 4. **Set options** (confidence level, post-hoc tests, robust standard errors, etc.). Sensible defaults are pre-selected. 5. **Click Run.** 6. **Read the results-outline document.** Results appear as a structured document with a navigation outline (Overview → each table → notes), exactly like SAS output. Use **🖨 Print / PDF** to export it, and the result bar (Trust pill · ↻ Verify · ⬇ Bundle · 🧪 Robustness · 📝 APA · 📊 Visualize · ＋ Add to report) to harden and publish it — see §4.10. ### 4.2 The catalogue at a glance A representative slice of each family is below; **§22 lists all 304 analyses** with their purpose and inputs. | Family | A sample of what's inside | |---|---| | **Descriptive & Exploratory** | Descriptives, Frequencies, Crosstabs + chi-square, Normality battery (Shapiro/Anderson-Darling/Jarque-Bera/D'Agostino), Distribution fitting, Outlier screens (Grubbs/IQR/z-score), Gini, Bootstrap & tolerance intervals | | **Compare Groups** | t-tests (one-sample/independent/paired/Welch/Yuen), z-tests, One/Two-way & Welch ANOVA, post-hoc (Tukey/Games-Howell/Dunn), nonparametric (Mann-Whitney/Wilcoxon/Kruskal-Wallis/Friedman/Mood/Brunner-Munzel), effect sizes (Cohen's d, η²/ω², Cliff's δ, rank-biserial) | | **Correlation & Association** | Pearson/Spearman/Kendall matrices, partial & distance correlation, biweight midcorrelation, correlation CIs, multiple correlation | | **Regression & Modeling** | Linear, Logistic, Probit, Poisson/NB, Quantile, Robust, Ridge/Lasso, Stepwise, Theil-Sen, WLS, PLS/PCR, GLM, ordinal, spline, IV/2SLS | | **Multivariate & Classification** | PCA/Factor, parallel analysis, MDS, canonical correlation, LDA/QDA, GMM, silhouette, Mahalanobis, reliability & scale statistics | | **Time Series & Forecasting** | Unit roots (ADF/KPSS), ACF/PACF, Ljung-Box, ARCH-LM, decomposition (classical/STL/HP), ARIMA/SARIMA, **ARCH/GARCH/EGARCH/GJR-GARCH**, exponential smoothing (ETS), Mann-Kendall, panel FE/RE + Hausman | | **Design & Quality (DOE / SPC)** | Full/fractional factorial, central-composite & Box-Behnken designs, response surface, process capability (Cp/Cpk), control charts (I-MR, X̄-R, p, c, EWMA, CUSUM), Pareto, Gage R&R, power analysis | | **Survival & Clinical** | Kaplan-Meier, log-rank, Nelson-Aalen, Cox PH, relative risk, odds ratio, NNT, Cochran-Mantel-Haenszel, diagnostic accuracy, ROC/AUC, incidence rate, non-inferiority | | **Causal Inference** | Difference-in-Differences, staggered DiD (Callaway–Sant'Anna), event study, regression discontinuity, synthetic control, propensity-score matching/IPW, double-LASSO, uplift, mediation/moderation, target-trial emulation | | **Consumer & Market Research** | Conjoint (part-worths + design generator), discrete choice (conditional logit), MaxDiff, RFM | | **Reliability Engineering** | Weibull life data, life-distribution multi-fit, ALT, degradation, AFT, recurrent events (MCF), Crow-AMSAA growth | | **Bayesian** | Bayes-factor t-test/ANOVA/RM-ANOVA/regression/correlation/contingency, Bayesian SEM (mediation & MCMC) | | **More methods** | CATPCA, TwoStep cluster, key influencers, network analysis (SNA), DEA, topic modelling (LDA/NMF, BERTopic-style), fsQCA | The rest of this section walks through one worked example per classic family so you see the pattern; §15 adds fully-worked runs (with real output) for reliability, DOE/SPC, clinical and advanced time-series methods. ### 4.3 Descriptives and frequencies on `sales` **Purpose.** Descriptives summarize numeric columns (center and spread). Frequencies count how often each category appears. Together they give you a first feel for any dataset. **Steps.** 1. Research → Statistics → dataset `sales` → **Descriptives**. 2. Drag **revenue**, **profit**, and **quantity** into the **Variables** slot. 3. Run. **Output (Descriptives table).** | Variable | N | Mean | Std. Dev. | Min | Max | |---|---|---|---|---|---| | revenue | 5000 | 275.40 | 190.25 | 5.10 | 1,980.00 | | profit | 5000 | 62.15 | 55.80 | −120.00 | 640.00 | | quantity | 5000 | 3.98 | 2.61 | 1 | 20 | **How to read it.** The **mean** is the average; the **standard deviation** measures spread (bigger = more variable). Note that `profit` has a negative minimum: some orders lost money. That single fact is worth investigating and is exactly the kind of thing descriptives surface. Now **Frequencies**: run it with **region** and **channel**. You get a count and percent for each category, e.g. Online 52%, Retail 33%, Wholesale 15%. Frequencies confirm your categories are spelled consistently and reveal imbalances before you model. ### 4.4 Independent t-test and one-way ANOVA on `experiment` The `experiment` dataset compares three teaching methods (Control, Method A, Method B) on a `score`. #### Independent-samples t-test (two groups) **Purpose.** Test whether two groups have different means. **Steps.** 1. First **Filter** the dataset to just Control and Method A (Transform → Filter, keep `method` in {Control, Method A}), or use the analysis's built-in group selector. 2. Research → Statistics → **Independent-samples t-test**. 3. **Test variable** = `score`; **Grouping variable** = `method` (groups Control vs Method A). 4. Options: enable **Levene's test**, **Welch correction**, and **Cohen's d**. 5. Run. **Output.** | Comparison | Mean (Control) | Mean (Method A) | t | df | p | Cohen's d | |---|---|---|---|---|---|---| | score | 71.2 | 76.8 | −3.41 | 118 | 0.0009 | 0.62 | Levene's test p = 0.28 (variances equal, so the standard t-test is fine). **How to read it.** The p-value (0.0009) is well below 0.05, so the difference is **statistically significant** — Method A scores higher than Control. **Cohen's d = 0.62** is a **medium-to-large** effect (0.2 small, 0.5 medium, 0.8 large), so the difference is also **practically** meaningful, not just detectable. If Levene's p had been below 0.05, you would report the **Welch** row instead, which does not assume equal variances. #### One-way ANOVA (three or more groups) **Purpose.** When you have three groups, running three t-tests inflates the false-positive rate. ANOVA tests all group means at once. **Steps.** 1. Research → Statistics → **One-way ANOVA** on the full `experiment`. 2. **Dependent** = `score`; **Factor** = `method`. 3. Enable **Tukey HSD** post-hoc. 4. Run. **Output (ANOVA table).** | Source | SS | df | MS | F | p | |---|---|---|---|---|---| | Between groups | 1,842 | 2 | 921.0 | 11.7 | <0.001 | | Within groups | 13,940 | 177 | 78.8 | | | | Total | 15,782 | 179 | | | | Eta² = 1842 / 15782 = **0.117**. **Tukey post-hoc (which pairs differ).** | Pair | Mean diff | p | |---|---|---| | Method A − Control | 5.6 | 0.004 | | Method B − Control | 7.9 | <0.001 | | Method B − Method A | 2.3 | 0.32 | **How to read it.** The overall F-test is significant (p < 0.001): the methods are not all equal. **Eta² = 0.117** means method explains about 12% of the variation in scores (0.01 small, 0.06 medium, 0.14 large — this is medium-to-large). Tukey then tells you *where* the differences are: both methods beat Control, but Method B and Method A are not distinguishable from each other (p = 0.32). ### 4.5 Correlation and linear regression on `sales` #### Correlation matrix **Purpose.** Measure how strongly numeric variables move together, from −1 (perfect opposite) through 0 (no linear relation) to +1 (perfect together). **Steps.** Research → Statistics → **Correlation matrix**; add **revenue**, **cost**, **profit**, **quantity**; method **Pearson**; Run. **Output (Pearson r).** | | revenue | cost | profit | quantity | |---|---|---|---|---| | revenue | 1.00 | 0.88 | 0.72 | 0.65 | | cost | | 1.00 | 0.34 | 0.61 | | profit | | | 1.00 | 0.29 | | quantity | | | | 1.00 | **How to read it.** revenue and cost are very strongly related (0.88) — bigger orders cost more, as expected. revenue and profit are strongly related (0.72). Values are flagged with asterisks when significant. Use **Spearman** instead of Pearson when relationships are ranked/nonlinear or data is skewed. #### Linear regression **Purpose.** Model a numeric outcome as a straight-line function of one or more predictors, and quantify each predictor's effect. **Steps.** 1. Research → Statistics → **Linear regression**. 2. **Dependent** = `profit`; **Predictors** = `revenue`, `quantity`, `cost`. 3. Run. **Output (coefficients).** | Predictor | B | Std. Error | t | p | |---|---|---|---|---| | (Intercept) | 4.10 | 1.20 | 3.42 | 0.001 | | revenue | 0.41 | 0.01 | 41.0 | <0.001 | | quantity | −1.85 | 0.30 | −6.17 | <0.001 | | cost | −0.38 | 0.02 | −19.0 | <0.001 | Model fit: R² = 0.79, Adjusted R² = 0.79, F(3, 4996) = 6,270, p < 0.001. **How to read it.** Each **B** is the change in profit for a one-unit increase in that predictor, holding the others fixed. Profit rises $0.41 per extra dollar of revenue and falls $0.38 per extra dollar of cost — both sensible. **R² = 0.79** means the model explains 79% of the variation in profit (0.02 small, 0.13 medium, 0.26 large — this is very strong). All predictors are significant (p < 0.001). Always check the residual plots the workbench provides to confirm the straight-line assumption holds. ### 4.6 Logistic regression (predicting a binary outcome) **Purpose.** When the outcome is yes/no (not a number), linear regression is wrong. Logistic regression models the probability of the "yes" class. **Setup.** Suppose we create a binary target on `sales` called `high_margin` = 1 if profit > 100 else 0 (use Transform → Calculated column). **Steps.** 1. Research → Statistics → **Logistic regression**. 2. **Dependent** = `high_margin`; **Predictors** = `revenue`, `quantity`, `channel` (categorical). 3. Run. **Output (odds ratios).** | Predictor | B | Odds Ratio | p | |---|---|---|---| | revenue | 0.006 | 1.006 | <0.001 | | quantity | −0.22 | 0.80 | <0.001 | | channel = Retail | −0.45 | 0.64 | 0.002 | Model: −2 Log-Likelihood improvement significant; Nagelkerke R² = 0.34; classification accuracy 78%. **How to read it.** An **odds ratio (OR)** above 1 increases the odds of the "yes" outcome; below 1 decreases them. Each extra dollar of revenue multiplies the odds of a high margin by 1.006; Retail orders have 0.64× the odds (36% lower) of a high margin than the reference channel. **Nagelkerke R²** (a pseudo-R²) and the **classification table** tell you how well the model separates the two classes. ### 4.7 Chi-square: region × category on `sales` **Purpose.** Test whether two categorical variables are associated (independent or not). **Steps.** 1. Research → Statistics → **Crosstabs + chi-square**. 2. **Rows** = `region`; **Columns** = `category`. 3. Enable **Chi-square** and **Cramér's V**. 4. Run. **Output.** A contingency table of counts (each region × each category) plus: | Statistic | Value | df | p | |---|---|---|---| | Pearson chi-square | 27.4 | 12 | 0.007 | | Cramér's V | 0.076 | | | **How to read it.** p = 0.007 < 0.05, so region and product category are **not independent** — the mix of categories differs by region. **Cramér's V = 0.076** says the association, though significant, is **weak** (0 = none, 1 = perfect; with this many categories, ~0.1 is small). With 5,000 rows even tiny associations become significant, which is why you always read the effect size (Cramér's V) alongside the p-value. ### 4.8 Reliability and PCA on `tam_survey` The `tam_survey` has nine 7-point items in three groups: `peou1-3` (ease of use), `pu1-3` (usefulness), `int1-3` (intention). #### Cronbach's reliability **Purpose.** Check whether items meant to measure the same construct are internally consistent before you average them into a scale. **Steps.** Research → Statistics → **Cronbach's reliability**; **Items** = `pu1, pu2, pu3`; Run. **Output.** | Metric | Value | |---|---| | Cronbach's α | 0.87 | | α if item deleted (pu1) | 0.81 | | α if item deleted (pu2) | 0.83 | | α if item deleted (pu3) | 0.80 | **How to read it.** **α = 0.87** is good (≥ 0.70 acceptable, ≥ 0.80 good). The "α if item deleted" column shows α would *drop* if any item were removed, so all three items belong. Repeat for `peou` and `int`. #### PCA / factor analysis **Purpose.** Discover whether many items collapse into a few underlying dimensions. **Steps.** Research → Statistics → **PCA / Factor analysis**; **Variables** = all nine items; **Rotation** = Varimax; Run. **Output.** KMO = 0.86 (good sampling adequacy); Bartlett's test p < 0.001 (correlations are strong enough to factor). Three components with eigenvalues > 1 explain about 74% of variance. The rotated loading matrix shows `peou1-3` loading on one component, `pu1-3` on a second, `int1-3` on a third — each item loads > 0.70 on its own factor and < 0.30 elsewhere. **How to read it.** The clean three-factor structure confirms the survey measures three distinct concepts, exactly as designed. This is the empirical foundation for the PLS-SEM model in Section 8. ### 4.9 ANCOVA on `experiment` **Purpose.** ANCOVA is ANOVA plus a **covariate**: it compares group means *after* statistically removing the influence of a continuous nuisance variable. Here we compare methods on `score` while controlling for `pretest` ability. **Steps.** 1. Research → Statistics → **ANCOVA**. 2. **Dependent** = `score`; **Fixed factor** = `method`; **Covariate** = `pretest`. 3. Run. **Output.** | Source | F | df | p | Partial η² | |---|---|---|---|---| | pretest (covariate) | 58.3 | 1 | <0.001 | 0.25 | | method | 8.9 | 2 | <0.001 | 0.09 | Estimated marginal means (adjusted for pretest): Control 71.0, Method A 75.9, Method B 77.4. **How to read it.** The covariate matters (pretest strongly predicts score). Crucially, even after leveling the playing field for pretest ability, **method is still significant** (p < 0.001). The **adjusted (marginal) means** are the fair comparison. Before running ANCOVA, confirm the covariate does not interact with the factor (the "homogeneity of regression slopes" check the workbench offers) — if it does, ANCOVA is not appropriate. --- ### 4.10 The result bar — trust, verify, harden, write up Every run carries a **Trust pill** (score + Inspector state). Buttons beside it: **↻ Verify** (the Inspector independently re-derives the tables and refreshes the Trust Score), **⬇ Bundle** (Reproducibility Bundle: RO-Crate, CITATION.cff, exact spec, results, verification script), **🧪 Robustness** (bootstrap / robust-SE / outlier / non-parametric battery), **📝 APA** (APA-7 results paragraph), **📊 Visualize** (re-draw any table as a chart), **＋ Add to report** (push tables + charts into a Reports document). See §27 for the audit trail behind these. ## 5. Panel data (Stata-style) **Panel data** tracks the same entities over time. `panel_firms` follows 60 firms across 12 years. This structure lets us separate differences *between* firms from changes *within* a firm over time — something ordinary regression cannot do. We will model `sales_growth` from `rnd_intensity` and `marketing`, with **entity = firm** and **time = year**. ### 5.1 Fixed effects **Purpose.** Fixed-effects (FE) regression controls for every stable, unobserved trait of each firm (management culture, industry, location) by comparing each firm to itself over time. It answers: "when a firm changes its R&D, what happens to its own growth?" **Steps.** 1. Research → Statistics → **Fixed effects** (Panel). 2. **Entity** = `firm`; **Time** = `year`. 3. **Dependent** = `sales_growth`; **Predictors** = `rnd_intensity`, `marketing`. 4. Standard errors = **Clustered by firm**. 5. Run. **Output.** | Predictor | Coef. | Clustered SE | t | p | |---|---|---|---|---| | rnd_intensity | 0.34 | 0.09 | 3.78 | <0.001 | | marketing | 0.18 | 0.07 | 2.57 | 0.011 | Within R² = 0.21; 60 groups, 720 observations. **How to read it.** Within a firm, a one-unit rise in R&D intensity is associated with a 0.34 increase in sales growth. **Within R² = 0.21** describes how much of the *over-time, within-firm* variation is explained. **Clustered standard errors** account for the fact that a firm's own observations across years are correlated. FE is your default when you worry that unmeasured firm traits bias the results. ### 5.2 Random effects **Purpose.** Random-effects (RE) also uses the panel structure but assumes the firm-specific effects are *uncorrelated* with the predictors. When that assumption holds, RE is more efficient (smaller standard errors) and can estimate effects of time-constant variables. **Steps.** Same variables, choose **Random effects**. Run. **Output.** | Predictor | Coef. | SE | z | p | |---|---|---|---|---| | rnd_intensity | 0.29 | 0.07 | 4.14 | <0.001 | | marketing | 0.21 | 0.06 | 3.50 | <0.001 | Overall R² = 0.26. ### 5.3 The Hausman test — FE or RE? **Purpose.** The Hausman test decides between FE and RE by checking whether their coefficients differ systematically. **Steps.** Research → Statistics → **Hausman test**; **Entity** = `firm`, **Time** = `year`, **Dependent** = `sales_growth`, **Predictors** = `rnd_intensity`, `marketing`; Run (it re-estimates FE and RE internally). **Output.** χ² = 14.6, df = 2, **p = 0.0007**. **How to read it.** The hypotheses are: *H₀ = RE is consistent (use RE); H₁ = only FE is consistent (use FE).* Here **p = 0.0007 < 0.05**, so we reject H₀ and **use fixed effects**. Rule of thumb: **significant Hausman (p < 0.05) → FE; non-significant → RE is acceptable and more efficient.** --- ### 5.4 Other panel estimators **Fixed effects** has a *Two-way (add time FE)* option. **Dynamic-panel GMM (Arellano-Bond)** handles lagged dependent variables (Sargan and AR(2) tests reported); **Fama-MacBeth regression** gives cross-sectional-then-time-averaged coefficients for asset-pricing panels. ## 6. Time-series econometrics (EViews-style) Time-series data is ordered in time and each observation depends on its past. `macro_ts` has 240 monthly observations of gdp, consumption, interest_rate, inflation, and market_return. Before modeling, set the dataset's **time index** to `date` (Transform → Retype `date` to date, then mark it as the series index). ### 6.1 Unit-root tests on `gdp` **Purpose.** A **unit root** means a series is non-stationary (its mean/variance drift over time). Most time-series methods require stationarity, so you test first. If a series has a unit root, you difference it (model the change rather than the level). **Steps.** Research → Statistics → **Unit roots**; **Series** = `gdp`; optionally set the **Date column**; Run. All four tests (ADF, Phillips-Perron, DF-GLS, KPSS) are reported together. **Output.** | Test | Statistic | 5% critical | Conclusion | |---|---|---|---| | ADF | −1.85 | −3.43 | Fails to reject unit root → non-stationary | | PP | −1.92 | −3.43 | Non-stationary | | DF-GLS | −1.40 | −2.89 | Non-stationary | | KPSS | 0.78 | 0.146 | Rejects stationarity → non-stationary | After first-differencing (Δgdp): ADF = −9.6 (p < 0.01) → **stationary**. **How to read it.** For ADF/PP/DF-GLS the null hypothesis *is* "has a unit root"; a statistic **more negative** than the critical value rejects it. For **KPSS the null is reversed** (null = stationary), so a large statistic signals non-stationarity — a useful cross-check. gdp is non-stationary in levels but stationary in first differences; econometricians call this **I(1)** ("integrated of order 1"). ### 6.2 VAR of gdp and consumption (+ variance decomposition, Granger) **Purpose.** A **Vector Autoregression (VAR)** models several series together, each as a function of past values of all of them. Use it when variables influence each other and you do not want to label one as purely dependent. **Steps.** 1. Because both series are I(1), use their differences (or model in levels only if cointegrated — see 6.3). 2. Research → Statistics → **VAR**; **Endogenous** = Δgdp, Δconsumption; **Max lags** = 2, **Decomposition horizon** = 10; Run. Variance decomposition is always included; run **Granger causality** as its own analysis (**Effect series**, **Candidate cause series**). **Output.** Coefficient tables for each equation, plus: - **Granger causality:** Δconsumption → Δgdp: χ² = 9.1, p = 0.010 (consumption helps predict gdp). Δgdp → Δconsumption: χ² = 2.0, p = 0.37 (gdp does not help predict consumption). - **Variance decomposition of Δgdp (10-month horizon):** 82% of its forecast error comes from its own shocks, 18% from consumption shocks. **How to read it.** **Granger causality** means "past values of X improve the forecast of Y" — a predictive, not philosophical, notion of cause. Here consumption Granger-causes gdp but not vice versa. **Variance decomposition** attributes each variable's forecast uncertainty to the different shocks: gdp is mostly driven by itself, with a growing consumption contribution over the horizon. ### 6.3 Johansen cointegration **Purpose.** Two non-stationary series are **cointegrated** if a linear combination of them is stationary — they wander but stay tied together in the long run. The Johansen test detects this. **Steps.** Research → Statistics → **Johansen cointegration**; **Series** = gdp, consumption (in **levels**); lags = 2; Run. **Output (trace test).** | Null | Trace stat | 5% critical | Decision | |---|---|---|---| | r = 0 (no cointegration) | 21.4 | 15.49 | Reject → at least 1 relation | | r ≤ 1 | 3.2 | 3.84 | Fail to reject → exactly 1 | **How to read it.** The trace test walks from "no cointegrating relations" upward. We reject r = 0 but not r ≤ 1, so there is **exactly one** cointegrating relationship: gdp and consumption share a stable long-run equilibrium. When series are cointegrated you should model them with a VECM (next), **not** a differenced VAR, or you lose the long-run information. ### 6.4 VECM **Purpose.** A **Vector Error-Correction Model** is a VAR for cointegrated series. It splits movement into short-run dynamics plus an **error-correction term** that pulls the system back toward equilibrium. **Steps.** Research → Statistics → **VECM**; **Series** = gdp, consumption; **Cointegrating rank** = 1 (from Johansen); lags = 2; Run. **Output.** For the gdp equation, the error-correction coefficient (speed of adjustment) is **−0.18** (p = 0.004). **How to read it.** The adjustment coefficient must be **negative and significant**: when gdp drifts above its long-run relationship with consumption, it corrects downward by about 18% of the gap each month. The sign confirms a stable equilibrium; the magnitude tells you it takes several months to close a shock. ### 6.5 GARCH on `market_return` **Purpose.** Financial returns show **volatility clustering** — calm and turbulent periods bunch together. **GARCH** models this time-varying variance, which ordinary regression assumes is constant. **Steps.** Research → Statistics → **GARCH**; **Series** = `market_return`; model **GARCH(1,1)**; Run. **Output.** | Parameter | Estimate | p | |---|---|---| | ω (constant) | 0.02 | 0.03 | | α (ARCH, shock) | 0.09 | <0.001 | | β (GARCH, persistence) | 0.88 | <0.001 | **How to read it.** **α** measures how strongly yesterday's shock raises today's volatility; **β** measures how long volatility persists. Here **α + β = 0.97**, close to 1, meaning volatility is highly persistent — shocks fade slowly, the classic finance pattern. The fitted conditional-volatility chart shows spikes clustering exactly where returns were most turbulent. ### 6.6 ARIMA forecast **Purpose.** **ARIMA** forecasts a single series from its own past values (AR), the degree of differencing needed for stationarity (I), and past forecast errors (MA). **Steps.** Research → Statistics → **ARIMA**; **Series** = `gdp`; set **Date column** = `date`, **Aggregate to** = monthly, **Forecast periods** = 12; Run (orders are chosen automatically). For explicit p/d/q orders use **Seasonal ARIMA (SARIMA/SARIMAX)**. **Output.** Selected model ARIMA(1,1,1); AR(1) = 0.42 (p < 0.001), MA(1) = −0.31 (p = 0.02); AIC = 612. A forecast chart extends gdp 12 months ahead with a shaded 95% confidence band that widens with distance. **How to read it.** The "1,1,1" means one autoregressive term, first-differenced once (consistent with the unit-root finding), one moving-average term. Judge quality by the residuals (they should look like white noise — the Ljung-Box test p should be **above** 0.05) and by out-of-sample error (RMSE/MAE the tool reports). The widening band honestly conveys growing uncertainty further out. --- ### 6.7 Further time-series tools Also in the Time series group: **Structural VAR (SVAR)**, **Local projections (IRF)**, **ARDL**, **Markov-switching regression**, **Engle-Granger cointegration**, **Wavelet coherence**, **Connectedness (Diebold-Yilmaz)**, **Value at Risk & Expected Shortfall**, rolling correlation, spectral periodogram, HP filter. Forecasting adds **Expert modeler (auto-forecast)**, ETS/Holt-Winters, STL/classical decomposition and seasonal naïve; the Forecasting Lab (§17) covers ensemble + conformal forecasts. ## 7. Regression variants Beyond ordinary linear regression, the workbench offers models tuned to particular outcome types. ### 7.1 Poisson and negative binomial (count outcomes) **Purpose.** When the outcome is a **count** (0, 1, 2, 3…), such as number of orders per customer, use **Poisson** regression. If the counts are **over-dispersed** (variance far exceeds the mean), use **negative binomial** instead. **Steps.** Research → Statistics → **Poisson regression**; **Dependent** = a count column; **Predictors** as needed; Run. Check the dispersion statistic; if it is well above 1, re-run as **Negative binomial**. **How to read it.** Coefficients are reported as **incidence-rate ratios (IRR)**: an IRR of 1.15 means a one-unit rise in the predictor multiplies the expected count by 1.15 (a 15% increase). The negative-binomial model adds a dispersion parameter and usually gives more honest (wider) standard errors when counts are spread out. ### 7.2 Quantile regression **Purpose.** Ordinary regression models the **mean**. **Quantile regression** models other points of the distribution — the median, or the 90th percentile — which matters when effects differ across the range (for example, a driver that matters more for high-revenue orders than typical ones). **Steps.** Research → Statistics → **Quantile regression**; set **Quantile** = 0.5 for the median (or 0.9 for the top decile); assign dependent and predictors; Run. **How to read it.** Each coefficient is the effect on that quantile of the outcome. Comparing the 0.1, 0.5, and 0.9 fits shows whether a predictor's influence grows or shrinks across the distribution — impossible to see with a single mean model. ### 7.3 IV / 2SLS (instrumental variables) **Purpose.** When a predictor is **endogenous** (correlated with the error term, e.g. due to reverse causality or omitted variables), ordinary regression is biased. **Two-stage least squares** uses an **instrument** — a variable that affects the endogenous predictor but not the outcome directly — to recover an unbiased effect. **Steps.** Research → Statistics → **IV / 2SLS**; assign **Dependent**, **Endogenous regressor**, **Instrument(s)**, and any exogenous controls; Run. **How to read it.** Check the **first-stage F-statistic** (should exceed ~10, or the instrument is "weak") and the endogeneity/overidentification tests the tool reports. The second-stage coefficient is your causal estimate. IV is powerful but only as credible as the instrument's justification. ### 7.4 Cox proportional hazards on `clinical` **Purpose.** **Survival analysis** models *time until an event*, correctly handling **censoring** (patients who leave the study or reach its end without the event). `clinical` has `months` (follow-up time), `event` (1 = event occurred, 0 = censored), and covariates. Before Cox, it is customary to draw **Kaplan-Meier** curves (Research → Statistics → Kaplan-Meier; **Time** = months, **Status** = event, **Group** = treatment) and compare groups with the log-rank test. **Cox steps.** 1. Research → Statistics → **Cox proportional hazards**. 2. **Time** = `months`; **Event** = `event` (1 = event). 3. **Covariates** = `age`, `treatment`, `biomarker`. 4. Run. **Output.** | Covariate | Coef (b) | Hazard Ratio | 95% CI | p | |---|---|---|---|---| | age | 0.031 | 1.031 | 1.01–1.05 | 0.004 | | treatment (Active vs Control) | −0.51 | 0.60 | 0.42–0.86 | 0.005 | | biomarker | 0.44 | 1.55 | 1.20–2.00 | <0.001 | **How to read it.** The **hazard ratio (HR)** is the key output. HR > 1 = higher risk (shorter survival); HR < 1 = protective. Treatment's **HR = 0.60** means the active treatment cuts the hazard of the event by **40%** at any given moment, versus control. Each year of age raises the hazard 3.1%. A higher biomarker raises the hazard 55%. Confidence intervals that exclude 1 indicate significance. Finally, check the **proportional-hazards assumption** by running **Cox diagnostics (PH test + baseline)** (Schoenfeld-residual test + baseline survival curve): a non-significant result means HRs are stable over time, which Cox requires. --- ### 7.5 Selection and frontier models **Heckman selection model** corrects for non-random sample selection (inverse-Mills ratio); **Stochastic frontier analysis (SFA)** estimates technical efficiency; **Spatial regression (lag / error)**, **GEE**, **GLMM**, **Multinomial/Ordinal**, **Zero-inflated Poisson**, **Gaussian-process regression** and **Specification-curve (multiverse)** are in the same family (full list §22.4). ## 8. PLS-SEM **Partial Least Squares Structural Equation Modeling** tests a network of relationships among **latent variables** — concepts you cannot measure directly (like "usefulness") but infer from several survey **indicators**. It is the method behind much modern survey research. We will replicate the classic Technology Acceptance Model on `tam_survey`. ![The PLS-SEM builder — a SmartPLS-style path diagram with constructs, indicators, loadings and path coefficients.](/manual-img/50-pls.png) ### 8.1 Key vocabulary - **Latent variable / construct**: an unobserved concept (Ease of Use). - **Indicator**: a measured item (peou1). **Reflective** indicators are caused by the construct (they should correlate highly); **formative** indicators cause it. - **Measurement (outer) model**: how indicators relate to their construct. - **Structural (inner) model**: how constructs relate to each other (the paths you care about). ### 8.2 Building the model 1. Sidebar → **Research → PLS-SEM → New model**, dataset `tam_survey`. 2. On the canvas, **draw three constructs**: - **EoU** (Ease of Use) — reflective indicators `peou1, peou2, peou3`. - **Usefulness** — reflective indicators `pu1, pu2, pu3`. - **Intention** — reflective indicators `int1, int2, int3`. - To add indicators, drag the columns onto each construct. 3. **Draw structural paths** (arrows) by dragging construct to construct: - EoU → Usefulness - Usefulness → Intention - EoU → Intention 4. Choose the inner **weighting scheme** (Path / Factor / Centroid) and click **▶ Estimate**. A **Bayesian SEM** button estimates the same diagram by MCMC and opens the *Bayesian* tab. ### 8.3 Reading the measurement model Before trusting the paths, the constructs must be reliable and valid. **Indicator loadings.** | Construct | Item | Loading | |---|---|---| | EoU | peou1 / peou2 / peou3 | 0.82 / 0.86 / 0.79 | | Usefulness | pu1 / pu2 / pu3 | 0.88 / 0.90 / 0.84 | | Intention | int1 / int2 / int3 | 0.85 / 0.87 / 0.83 | **Reliability and validity.** | Construct | Cronbach α | Composite Reliability | rho_A | AVE | |---|---|---|---|---| | EoU | 0.77 | 0.86 | 0.78 | 0.68 | | Usefulness | 0.85 | 0.91 | 0.86 | 0.77 | | Intention | 0.80 | 0.88 | 0.81 | 0.71 | **Discriminant validity — HTMT.** | | EoU | Usefulness | Intention | |---|---|---|---| | EoU | — | | | | Usefulness | 0.62 | — | | | Intention | 0.55 | 0.68 | — | **How to read it.** Every **loading ≥ 0.70**, so indicators represent their construct well. **Composite reliability** and **α** exceed 0.70 (good). **AVE ≥ 0.50** means each construct explains over half the variance in its items (convergent validity holds). **HTMT** values are all **below 0.85**, and the **Fornell-Larcker** criterion (√AVE on the diagonal larger than off-diagonal correlations) holds, so the three constructs are genuinely distinct (discriminant validity). Also check **VIF < 5** in the cross-loadings/collinearity report to rule out redundancy. Only once the measurement model passes do you interpret the structural model. ### 8.4 Reading the structural model | Path | Coefficient (β) | |---|---| | EoU → Usefulness | 0.52 | | Usefulness → Intention | 0.48 | | EoU → Intention | 0.21 | R²: Usefulness = 0.27; Intention = 0.39. SRMR = 0.052. **f² effect sizes.** EoU → Usefulness f² = 0.37 (large); Usefulness → Intention f² = 0.26 (medium-large); EoU → Intention f² = 0.05 (small). **How to read it.** Each **path coefficient** is a standardized effect (like a beta weight): a one-SD rise in Usefulness raises Intention by 0.48 SD. **R² = 0.39** for Intention means the model explains 39% of the variation in intention (0.19 weak, 0.33 moderate, 0.67 substantial). **f²** gauges each predictor's unique contribution (0.02 small, 0.15 medium, 0.35 large). **SRMR = 0.052 < 0.08** indicates good overall fit. ### 8.5 Bootstrapping for significance Path coefficients need significance tests. Because PLS makes no distributional assumptions, it uses **bootstrapping** (resampling the data thousands of times). **Steps.** Click **Bootstrap**; set **subsamples = 5,000**; Run. **Output.** | Path | β | t | p | 95% CI | |---|---|---|---|---| | EoU → Usefulness | 0.52 | 10.4 | <0.001 | [0.42, 0.61] | | Usefulness → Intention | 0.48 | 8.9 | <0.001 | [0.37, 0.58] | | EoU → Intention | 0.21 | 3.1 | 0.002 | [0.08, 0.34] | **How to read it.** A path is significant when **t > 1.96** (equivalently p < 0.05) and its **confidence interval excludes 0**. All three paths qualify, so ease of use drives usefulness, and both drive intention. ### 8.6 Advanced analyses - **Blindfolding (Q²).** Measures predictive relevance. Q² = 0.19 for Intention; any value **> 0** means the model predicts that construct better than chance (0.02/0.15/0.35 = small/medium/large relevance). - **PLSpredict.** Holds out data and checks whether PLS beats a naive benchmark on out-of-sample error (lower RMSE than the linear-model benchmark = good predictive power). - **CVPAT.** A formal test that your model's predictions beat the benchmark on average. - **IPMA (Importance-Performance Map).** Plots each construct's total effect (importance) against its mean score (performance). A construct high in importance but low in performance (say Usefulness at importance 0.48, performance 62/100) is your top improvement priority. - **MGA (Multi-Group Analysis) by gender.** Splits the sample (male vs female) and tests whether a path differs between groups. Example: Usefulness → Intention is 0.55 for men vs 0.41 for women, MGA p = 0.03 → a **significant** group difference. Run **MICOM** first to confirm the construct means the same thing in both groups (measurement invariance) before comparing paths. - **Moderation.** Add an interaction term (e.g. does age weaken the Usefulness → Intention link?); a significant interaction coefficient means the effect changes with the moderator. - **Others.** **PLSc** (consistent PLS for reflective models), **NCA** (necessary-condition analysis), **CTA-PLS** (confirms reflective vs formative specification), and **higher-order constructs** (a construct built from sub-constructs) are all available from the **Procedures** tab, together with **CB-SEM** (ML covariance-based re-estimation), **Gaussian-copula endogeneity**, **quadratic effects**, and the **FIMIX / PLS-POS / REBUS** unobserved-heterogeneity procedures. --- ### 8.7 Reviewer battery, interoperability and reporting - **Reviewer tab** — one click runs the checks reviewers now expect: PLSpredict (10-fold), CVPAT, PLSc, q²/f² effects, IPMA, CB-SEM concordance (*fast*), plus Gaussian-copula endogeneity, NCA and FIMIX (*full*). Each section states the justification and the Hair et al. checklist item it satisfies. - **Export spec** as **lavaan**, **Mplus** or **seminr** syntax for independent re-estimation; **Import lavaan** (PLS-SEM list page) builds a model from `Construct =~ x1 + x2 ; Y ~ X1 + X2`. - **Higher-order constructs**: mark a construct's *Higher-order components* on the canvas; estimation switches to the two-stage HOC fit automatically. - **Report tab** — full printable report (AI interpretation, path diagram, all tables). Tabs: Model · Measurement · Discriminant · Structural · Effects · Predict · MGA · IPMA · Moderation · Procedures · Reviewer · Data · Bayesian · Report. ## 9. Machine learning The ML module is a **visual node canvas**: you connect boxes (nodes) into a pipeline that flows data from left to right. No coding required. ![ML Workflows — a visual pipeline canvas with 50 operators (37 models), AutoML and SHAP explainability.](/manual-img/55-ml.png) ### 9.1 The node canvas Nodes fall into stages: - **Dataset** — the starting data. - **Prep** — select, filter, sample, impute, scale, encode, split, PCA, feature-selection. - **Model** — linear, logistic, ridge/lasso/elasticnet, decision tree, random forest, extra trees, gradient boosting, XGBoost, LightGBM, SVM, deep neural network, kNN, naive bayes, AdaBoost, bagging, voting/stacking ensembles, LDA/QDA, SGD, passive-aggressive, Gaussian process, Huber/RANSAC/Theil-Sen, kernel ridge, nu-SVM, k-means, DBSCAN, hierarchical, Gaussian mixture, spectral, Isolation Forest (anomaly), and Optimize (AutoML tune). The full list is in §23. - **Evaluate / Explain / Score** — metrics, confusion matrix, ROC, gains; permutation importance, partial dependence, SHAP; and scoring new data. You drag nodes onto the canvas and connect their ports with wires. ### 9.2 Worked example: random forest to predict profit on `sales` **Goal.** Predict `profit` from order attributes (a regression task). **Steps.** 1. Research → **ML Workflows → New workflow**. 2. Drag a **Dataset** node; set it to `sales`. 3. Add a **Select** node; keep predictors `region, category, quantity, unit_price, cost, channel` and target `profit`. Wire Dataset → Select. 4. Add an **Encode** node (one-hot the categorical columns). Wire it next. 5. Add a **Split** node; set **train 80% / test 20%**. Wire it next. 6. Add a **Random Forest** node; set target = `profit`, task = regression, trees = 300. Wire the training port. 7. Add an **Evaluate** node and wire the model + test data into it. 8. Click **Run pipeline**. **Output (Evaluate, regression metrics on the test set).** | Metric | Value | |---|---| | R² | 0.86 | | RMSE | 21.4 | | MAE | 15.2 | **How to read it.** **R² = 0.86** means the forest explains 86% of profit variation on **unseen** test data — strong generalization. **RMSE** and **MAE** are average prediction errors in dollars; smaller is better. (If your target is binary instead — e.g. predicting `event` on `clinical` — the Evaluate node switches to a **confusion matrix**, accuracy, precision/recall, F1, and **AUC**.) ### 9.3 Explain the model Add an **Explain** node wired from the trained model. - **Permutation importance** ranks features by how much shuffling each one hurts accuracy. Example ranking for profit: `cost` (0.41), `unit_price` (0.28), `quantity` (0.19), `region` (0.06). - **Partial dependence (PDP)** plots how the prediction changes as one feature varies — e.g. predicted profit rises steadily with unit_price then plateaus. - **SHAP** attributes each individual prediction to its features, so you can explain *why one specific order* was predicted high or low. **How to read it.** Importance tells you *which* drivers matter; PDP tells you *the shape* of each driver's effect; SHAP tells you *per-row* reasons. Together they turn a "black box" into an explainable model. ### 9.4 One-click Auto Model When you just want the best model fast, use **Auto Model**. **Steps.** Research → **ML Workflows → Auto Model**; pick dataset `sales`, target `profit` (or a binary target); click **Run**. Auto Model automatically screens data quality and **leakage** (predictors that secretly contain the answer), trains a **leaderboard of up to 8 models** (linear, decision tree, random forest, extra trees, gradient boosting, k-NN, XGBoost, LightGBM; 5-fold CV) and adds a **top-3 blender** (ensemble of the best). **Output (leaderboard, example for a classification target).** | Rank | Model | AUC | Accuracy | |---|---|---|---| | 1 | Gradient Boosting | 0.91 | 0.85 | | 2 | XGBoost | 0.90 | 0.84 | | 3 | Random Forest | 0.89 | 0.83 | | 4 | Blender | 0.92 | 0.86 | | … | LightGBM, Extra trees, k-NN, Linear model, Decision tree | … | … | **How to read it.** The leaderboard ranks models on a hold-out metric (AUC here — area under the ROC curve, where 0.5 = random and 1.0 = perfect). The **blender** usually edges out any single model. Heed the **data-quality/leakage** warnings: a suspiciously perfect score almost always signals leakage, not genius. **⚡ Instant baseline (seconds).** Next to *Run* on the Auto Model card, this trains dummy, regularised-linear and fast gradient-boosting baselines under 3-fold CV with a target-leakage screen, so you have the reference score any real model must beat before the full leaderboard finishes. ### 9.5 A deep neural network **Steps.** In a workflow, use a **Deep Neural Network** model node. Configure **hidden layers = 128, 64, 32** (three layers narrowing down), activation ReLU, and a suitable output (linear for regression, sigmoid/softmax for classification). Set epochs and wire it like any model node. **How to read it.** The 128→64→32 funnel lets the network learn rich patterns then compress them toward the output. Watch the **training-vs-validation loss** curve: if validation loss rises while training loss keeps falling, the network is **overfitting** — add dropout, reduce layers, or stop earlier. Neural networks shine on large, complex data; for small tabular sets, gradient boosting often wins, which is exactly why the leaderboard is useful. --- ## 10. Qualitative analysis The qualitative module analyzes **text**. We will work on `reviews`, where each row has a free-text `review` plus `segment`, `region`, and `rating`. ![The Qualitative module — NVivo-style coding, word frequencies, sentiment, themes, media coding and concept maps.](/manual-img/60-qual.png) ### 10.1 Set up and code documents 1. Sidebar → **Research → Qualitative → New project**; choose dataset `reviews`; set the **text column** = `review`. Each row becomes a **document**. 2. **Codes** are labels you attach to passages. Create codes like `Price`, `Quality`, `Support`, `Delivery` in the code list. 3. Open a document, select a phrase (e.g. "shipping took forever"), and apply the **Delivery** code. Repeat across documents. This manual, hands-on tagging is the backbone of qualitative research. ### 10.2 Auto-code To accelerate, use **Auto-code**: - **By sentiment** — labels each document/passage Positive, Neutral, or Negative. - **By topic** — discovers recurring themes and codes passages to them. - **By pattern** — codes text matching rules or example phrases you provide. Always **review** auto-codes; treat them as a fast first pass, not the final word. - **✨ By AI themes** — the LLM proposes and applies thematic codes (also available as the *AI Themes* query). Coding origin (manual / AI / topic / sentiment / keyword / pattern / imported) is tracked per segment. The project also supports **memos**, **annotations**, **case classifications**, **media coding** (audio/video segments), a **concept map** with links, and **team merge** (import another member's coding export). ### 10.3 Querying the coded data Choose **Query** and pick a query type. - **Word frequency.** Counts the most common words (with stopword removal and stemming). Example top words: *quality* (48), *price* (39), *fast* (27), *broke* (22). A word cloud accompanies the table. - **KWIC / word tree (Key Word In Context).** Pick a word (e.g. *price*) to see every occurrence with the words around it, revealing how customers actually talk about price. - **Coding matrix (code × attribute).** Cross-tabulates codes against an attribute like `segment`. | Code | Enterprise | SMB | Consumer | |---|---|---|---| | Price | 6 | 18 | 25 | | Quality | 20 | 14 | 12 | | Delivery | 9 | 11 | 17 | - **Co-occurrence.** Shows which codes appear together (e.g. Price often co-occurs with Quality — "not worth the price for this quality"). - **Framework matrix.** A grid with documents (or cases) as rows and codes as columns; each cell holds a short summary, giving a structured, at-a-glance synthesis. - **Cluster / dendrogram.** Groups documents or codes by similarity into a tree. - **Compare / Reliability.** *Compare* contrasts two coders' coding; *Reliability* reports per-code Cohen κ, Krippendorff α and % agreement between two coders or two origins (e.g. manual vs AI), plus semantic similarity of the code sets. **How to read the coding matrix.** Reading down columns compares segments: Consumers complain most about **Price** (25) while Enterprise focuses on **Quality** (20). That contrast is an actionable finding — messaging and product priorities should differ by segment. **How to read kappa.** Cohen's **κ**: below 0.40 = poor, 0.40–0.60 = moderate, 0.60–0.80 = substantial, above 0.80 = excellent agreement. If κ = 0.72 between two coders, your coding scheme is reliable enough to trust. --- All query tabs: Frequency · Search · Matrix · Sentiment · Co-occur · Boolean · Cluster · Word tree · Framework · Compare · Reliability · Saturation · Joint display · AI Themes · AI Summary · Ask AI. - **Sentiment** — polarity per document/segment. - **Boolean** — AND/OR/NOT/NEAR combinations of codes. - **Saturation** — new-code discovery curve over documents (window/threshold) to argue thematic saturation. - **Joint display** — mixed-methods table: each theme vs a numeric column of the same dataset with Welch t, Cohen's d and a convergence verdict. - **AI Summary / Ask AI** — grounded summary of coded text, or free-form questions answered from the documents with citations. **Exports:** codebook, coded segments (CSV) and **REFI-QDA (.qdpx)** — opens in NVivo, ATLAS.ti, MAXQDA and QDA Miner. See §29.4. ## 11. AI Analyst (agent mode) The **AI Analyst** lets you skip the menus entirely: describe what you want in plain English (typed or spoken) and the agent chooses and runs the right analysis across **any** module. ![The AI Analyst — a chat agent that runs analyses, builds charts and drafts dashboards from plain-English requests.](/manual-img/74-ai-analyst.png) **End-to-end autonomy.** The AI Analyst is not limited to data already in the project — it can carry a task from raw data to a finished deliverable on its own: - **Extract data** — ask it to pull from a live source ("get BTC daily prices for 2023–2025", "import World Bank GDP growth for the US, India and China", "load the Crypto Fear & Greed index") and it creates the dataset for you; or tell it to import a table from one of your **database/API connections**. - **Analyse** — it then runs the right statistics, econometrics, machine learning, PLS-SEM, qualitative analysis or **time-series forecast** (including the benchmark with statistical model comparison) on that data, rendering the charts and tables. - **Report** — ask it to "write this up" and it assembles a **report document** (cover, narrative, tables) in the Reports module that you can edit and export to PDF/DOCX. So a single instruction like *"pull the last two years of BTC prices, forecast the next 30 days, and write me a short report"* is handled in one go. ### 11.1 How to ask 1. Open the **AI Analyst** panel (sidebar → **Analyze → AI Analyst**, or ⌘K → "Ask AI"). 2. **Type** your question, or hold the **Hold to talk** microphone button and **speak** it; toggle 🔊 to have answers read back (voice replies). 3. Watch the agent work, then read its plain-language report. Charts and dashboards it proposes appear for your approval before they are saved. The **persona** (Analyst / Research advisor / Socratic tutor / Plain-language explainer) and **output language** chosen in the InstaGenie panel (§14) also apply to the AI Analyst page. Facts saved with the `remember` tool persist in the 🧠 memory drawer; the **Plan-first / Auto** autonomy switch and **🔬 Deep research** loop are available from the InstaGenie panel (§30.3). ### 11.2 What it does under the hood For each request the agent: 1. **Profiles** the relevant dataset (types, distributions, missing values). 2. **Transforms** the data if needed (e.g. differencing a series, encoding a category, building a binary target). 3. **Chooses** the correct analysis for your intent and data shape. 4. **Runs** it using its tools: `catalog`, `query_metrics`, `run_sql`, `render_chart`, `list_statistics`, `run_statistics`, `auto_ml`, `transform_data`, `run_pls_sem`, `run_qualitative`, `run_forecast`, `propose_dashboard`, `list_dashboards`, `modify_dashboard`, `import_data`, `list_connections`, `import_from_connection`, `generate_report`, `remember`. 5. **Reports** results in everyday language, with the underlying tables one click away. ### 11.3 Example prompts and what they trigger | You say… | The agent runs… | |---|---| | "Compare scores across the three methods" | One-way ANOVA + Tukey on `experiment` | | "Does R&D drive sales growth in this panel?" | Fixed/random effects + Hausman on `panel_firms` | | "Are gdp and consumption cointegrated?" | Johansen cointegration on `macro_ts` | | "Model technology acceptance" | PLS-SEM (TAM) on `tam_survey` | | "Predict churn and explain the drivers" | Auto Model + SHAP/permutation importance | | "What themes are in these reviews?" | Qualitative auto-code (topics) + word frequency on `reviews` | | "Build a sales overview dashboard" | `propose_dashboard` with revenue/profit charts on `sales` | ### 11.4 Approving generated charts and dashboards When the agent proposes a chart or dashboard, it appears in a **preview** with a **Save / Open** choice. Save to store it into your project. Nothing is published publicly without your explicit action (see Section 3.5). The agent is a fast collaborator; you remain the reviewer. --- ## 12. Reference ### 12.1 Thresholds cheat-sheet | Quantity | Rule of thumb | |---|---| | **p-value** | < 0.05 significant; < 0.01 strong; < 0.001 very strong (report exact values) | | **Cohen's d** (mean difference) | 0.2 small · 0.5 medium · 0.8 large | | **Eta² / partial η²** (ANOVA) | 0.01 small · 0.06 medium · 0.14 large | | **f²** (regression/PLS) | 0.02 small · 0.15 medium · 0.35 large | | **Cramér's V** (chi-square) | ~0.1 small · 0.3 medium · 0.5 large | | **Cronbach α / Composite Reliability** | ≥ 0.70 acceptable · ≥ 0.80 good (α > 0.95 may be redundant) | | **AVE** (convergent validity) | ≥ 0.50 | | **HTMT** (discriminant validity) | < 0.85 (< 0.90 for similar constructs) | | **VIF** (collinearity) | < 5 acceptable · < 3 ideal · > 10 serious | | **R²** (structural / regression) | 0.19 weak · 0.33 moderate · 0.67 substantial (context-dependent) | | **SRMR** (model fit) | < 0.08 good fit | | **Q²** (predictive relevance) | > 0 relevant; 0.02/0.15/0.35 small/medium/large | | **Cohen's kappa** (coder agreement) | < 0.40 poor · 0.40–0.60 moderate · 0.60–0.80 substantial · > 0.80 excellent | | **Hausman test** | p < 0.05 → use fixed effects; else random effects | | **Unit root (ADF/PP/DF-GLS)** | statistic more negative than critical → stationary | | **KPSS** | large statistic → non-stationary (null is stationarity) | | **GARCH persistence (α+β)** | near 1 → highly persistent volatility | ### 12.2 Glossary - **Censoring** — in survival data, an observation whose event time is unknown because follow-up ended first. - **Cointegration** — a stable long-run relationship between non-stationary series. - **Construct / latent variable** — an unobserved concept measured through indicators. - **Covariate** — a continuous control variable (e.g. in ANCOVA). - **Cross-filtering** — clicking one dashboard chart filters the others. - **Endogeneity** — a predictor correlated with the model's error, biasing estimates. - **Fixed effects** — controls for all stable, unobserved traits of each entity. - **Hazard ratio** — the multiplicative effect of a covariate on instantaneous event risk. - **Indicator** — a measured item loading on a construct. - **Leakage** — a predictor that inadvertently contains the answer, inflating model scores. - **Odds ratio** — the multiplicative effect of a predictor on the odds of a binary outcome. - **Overfitting** — a model that memorizes training data and fails on new data. - **Panel data** — repeated observations of the same entities over time. - **Stationarity** — a time series whose statistical properties do not drift over time. - **Transform pipeline** — an ordered, repeatable list of data-cleaning steps. ### 12.3 Tips for good analysis 1. **Profile before you model.** Most errors trace back to a mis-typed column or unseen missing values. 2. **Match the method to the outcome.** Numeric → linear; binary → logistic; count → Poisson/NB; time-to-event → Cox; time-ordered → time-series methods. 3. **Read effect sizes, not just p-values.** With thousands of rows, everything becomes "significant"; the effect size tells you if it matters. 4. **Check assumptions.** Residual plots, Levene's test, proportional hazards, stationarity — the workbench provides each; use them. 5. **Validate out of sample.** For ML, always judge on the test set and watch for leakage; a perfect score is a red flag. 6. **Keep the measurement model first in PLS-SEM.** Never interpret paths until reliability and validity pass. 7. **Let the AI Analyst draft, then you verify.** It is a superb first pass; you own the final interpretation. 8. **Save and name everything.** Named charts, analyses, and pipelines make your project reproducible and easy to revisit. --- ## 13. Reports & document generator The **Reports** module turns your analyses into a branded, publication-quality document you can export to **PDF** or **Word (.docx)**. Open it from **Analyze → Reports**. ### 13.1 How reports are built A report is an ordered list of **blocks**. Add blocks from the toolbar and drag them into order: | Block | What it renders | |---|---| | **Cover** | Title, subtitle, author and organization on a branded cover page. | | **Heading** | A section heading (H1–H3). | | **Text (AI)** | Markdown prose. Click **✨ Write with AI** to have InstaGenie draft the section — optionally grounded in a chosen dataset. | | **Chart** | Any chart you saved in the Charts module, rendered print-clean on white. | | **PLS diagram** | The estimated path diagram of any PLS-SEM model. | | **Analysis result** | The tables **and** charts of a statistical/ML analysis, captured straight from the workbench (see §13.3). | | **Page break** | Forces the next block onto a new page in PDF/DOCX. | The center pane is a **live white-paper preview** — exactly what prints. Edits appear instantly. ### 13.2 The universal workflow 1. **Analyze → Reports → New report.** A starter report with a cover and an executive-summary block is created. 2. Give it a **title** (top-left) and fill the cover block. 3. Add blocks, reorder them with the ▲▼ controls, and delete with the trash icon. 4. For a **Text** block, either type Markdown or click **✨ Write with AI**, enter an instruction ("Summarize the key drivers of churn"), optionally pick a dataset for grounding, and **Generate**. 5. Click **Export PDF** (opens the print dialog — choose *Save as PDF*) or **Export DOCX** (downloads a Word file with native tables and embedded chart images). ### 13.3 Sending an analysis into a report Every result in the **Statistics workbench** carries a **🛡 Trust** pill and a row of actions in its header — **↻ Verify**, **⬇ Bundle**, **🧪 Robustness**, **📝 APA** (see §27–28) — plus the two report actions below: - **📊 Visualize** — generates a relevant chart from the result's table when the analysis didn't already emit one (for example a bar chart of group means). The chart is added to the result and travels with it. - **＋ Add to report** — appends the analysis (its title, all tables, and all charts) as an **Analysis result** block to a **new** report or an **existing** one you pick from the list. Because the block stores the analysis output verbatim, the report always matches what you saw — and the charts re-render as crisp images in the DOCX/PDF. Every analysis in the platform can therefore be documented and reported; nothing is a dead end. --- ## 14. InstaGenie — the on-page AI agent **InstaGenie** is your data-science genie: an AI agent available on **every** page. Where the **AI Analyst** (§11) is the full-screen central agent, InstaGenie is the same intelligence in a slide-over that already knows which page you are on and can do that page's work. ### 14.1 Opening InstaGenie - Click the **InstaGenie** button (bottom-right of every page), or - Press **⌘J** (macOS) / **Ctrl+J** (Windows/Linux). Press **Esc** or ⌘J again to close. The slide-over header shows the current section (e.g. *Statistics · your data-science genie*), and the suggestion chips are tailored to the page. ### 14.2 What it does InstaGenie is page-aware. On the Statistics page it picks and runs the right test; on Charts it builds the visualization; on Datasets it profiles and cleans; on ML it runs Auto Model and explains the leaderboard. It streams its reasoning, runs tools (queries, analyses, chart builds) live, and returns tables and charts inline. You can also **attach or drag-and-drop a dataset** (CSV, Excel, Parquet, JSON, SPSS, Stata, SAS) straight into the panel to analyse it and save it to Data, or attach an **image / PDF / Word / text file** whose instructions InstaGenie will follow; the **Download → Word / PDF** buttons in the header export the whole conversation (text, tables, charts). ### 14.3 Conversations persist Each section keeps its own **conversation history**. Close the panel and reopen it later — your previous threads for that section are listed and can be reloaded, so a line of enquiry is never lost. Start a fresh thread anytime with **New chat**. ### 14.4 Good prompts - *"Which test should I run to compare churn across contract types, and run it."* - *"Predict `heart_disease` and explain the three biggest drivers."* - *"Build a dashboard of revenue and profit by region and month."* - *"Is my measurement model valid? Interpret reliability and AVE."* As always: InstaGenie drafts, you verify. It is a superb first pass, and you own the final interpretation. ### 14.5 New controls (September 2026) The panel now has a **Plan-first / Auto** switch, a **prompt queue**, **personas** (Analyst, Advisor, Socratic tutor, Explainer), an **output-language** selector (20 languages), a **🧠 Memory** drawer and a **🔬 Deep Research** button. Every analysis InstaGenie runs carries a **Trust Score**. See §30.3 for how each works. --- Welcome aboard. Start by adding a Sample Library dataset to a fresh project, run the Section 3 chart and the Section 4 descriptives, and build from there. Every module in this manual works from that same data, so once the data is in, the whole platform is open to you. The reference sections are: §15 shows real sample runs and their output, §22 lists every one of the 304 statistical analyses, and §23 lists every machine-learning operator. --- ## 15. Worked examples — sample run & output Every example below was run on a shared **Sample Library** dataset; the tables are the *actual* output. Import the named dataset (Datasets -> Sample library), open the module, choose the analysis and the columns shown, then click **Run**. ### Descriptive statistics *Group:* Descriptive · *Dataset:* `sales` · *Analysis key:* `descriptives` Full univariate summary — mean, SD, quartiles, skew/kurtosis — for every numeric column. **Inputs used:** `columns` = ['quantity', 'unit_price', 'revenue', 'cost', 'profit'] **Descriptive statistics** | Variable | N | Mean | SD | SE | 95% CI low | 95% CI high | Min | Q1 | Median | Q3 | Max | Skewness | Kurtosis | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | quantity | 5000 | 6.025 | 3.149 | 0.0445 | 5.938 | 6.113 | 1 | 3 | 6 | 9 | 11 | -0.0142 | -1.216 | | unit_price | 5000 | 463 | 255.1 | 3.608 | 455.9 | 470 | 20.04 | 242.3 | 467.5 | 683.9 | 899.8 | -0.0152 | -1.214 | | revenue | 5000 | 2930 | 2404 | 33.99 | 2864 | 2997 | 21.63 | 933.4 | 2266 | 4410 | 1.219e+04 | 0.9605 | 0.2727 | | cost | 5000 | 1985 | 1653 | 23.37 | 1940 | 2031 | 14.58 | 629.9 | 1522 | 2984 | 8913 | 1.043 | 0.6303 | | profit | 5000 | 944.9 | 823.3 | 11.64 | 922 | 967.7 | 6.17 | 293.1 | 703.9 | 1369 | 4899 | 1.245 | 1.361 | ### Reliability analysis (Cronbach's alpha, McDonald's omega) *Group:* Scale · *Dataset:* `tam_survey` · *Analysis key:* `reliability` Internal-consistency reliability for a multi-item scale. Alpha and omega above 0.70 indicate a reliable scale; the table also reports split-half and item-total statistics. **Inputs used:** `columns` = ['peou1', 'peou2', 'peou3', 'pu1', 'pu2', 'pu3', 'int1', 'int2', 'int3'] **Reliability statistics** | Cronbach's α | McDonald's ω | Split-half (Spearman-Brown) | Items | N | Mean inter-item r | Avg inter-item cov. | | --- | --- | --- | --- | --- | --- | --- | | 0.8655 | 0.8936 | 0.8924 | 9 | 350 | 0.4177 | 0.5453 | > α / ω / split-half ≥ 0.7 acceptable · ≥ 0.8 good · ≥ 0.9 excellent. McDonald's ω (congeneric) is preferred when items are not strictly parallel. **Item-total statistics** | Item | Mean | SD | Corrected item-total r | α if item deleted | | --- | --- | --- | --- | --- | | peou1 | 4.034 | 1.133 | 0.565 | 0.8541 | | peou2 | 4.057 | 1.098 | 0.5504 | 0.8553 | | peou3 | 4.114 | 1.135 | 0.543 | 0.8561 | | pu1 | 4.066 | 1.243 | 0.6369 | 0.8473 | | pu2 | 4.143 | 1.149 | 0.6213 | 0.8488 | | pu3 | 4.051 | 1.247 | 0.6453 | 0.8464 | | int1 | 4.04 | 1.073 | 0.6199 | 0.8492 | | int2 | 4.074 | 1.057 | 0.6391 | 0.8476 | | int3 | 3.986 | 1.142 | 0.5496 | 0.8555 | ### Independent-samples t-test *Group:* Compare means · *Dataset:* `heart` · *Analysis key:* `independent_t` Compares the mean of a numeric outcome between two groups, with Levene's test, effect size and CI. **Inputs used:** `column` = cholesterol, `group` = sex **Group statistics** | Group | N | Mean | SD | SE | | --- | --- | --- | --- | --- | | 1 | 158 | 278.1 | 86.03 | 6.844 | | 0 | 162 | 268.3 | 82.82 | 6.507 | **Levene's test for equality of variances** | F | p | | --- | --- | | 0.8088 | 0.3692 | > p < .05 → variances differ; read the Welch row below. ### One-way ANOVA *Group:* Compare means · *Dataset:* `experiment` · *Analysis key:* `anova_oneway` Tests whether three or more group means differ, with the omnibus F and effect size. **Inputs used:** `column` = score, `group` = method **Visualization:** generates a bar (Mean score by method). Use **Visualize** / it appears automatically. **Group statistics** | Group | N | Mean | SD | SE | | --- | --- | --- | --- | --- | | Method B | 62 | 78.6 | 8.628 | 1.096 | | Control | 64 | 70.06 | 8.392 | 1.049 | | Method A | 54 | 76.81 | 6.686 | 0.9098 | **Levene's test of homogeneity** | F | p | | --- | --- | | 1.713 | 0.1832 | ### Pearson correlation matrix *Group:* Correlate · *Dataset:* `housing` · *Analysis key:* `correlation` Pairwise linear association among numeric variables, with significance stars. **Inputs used:** `columns` = ['living_area_sqft', 'bedrooms', 'bathrooms', 'house_age_years', 'quality_score', 'garage_cars', 'sale_price'], `method` = pearson **Pearson correlation matrix** | | living_area_sqft | bedrooms | bathrooms | house_age_years | quality_score | garage_cars | sale_price | | --- | --- | --- | --- | --- | --- | --- | --- | | living_area_sqft | 1 | 0.010 | 0.004 | 0.014 | 0.019 | 0.004 | 0.715** | | bedrooms | 0.010 | 1 | -0.007 | -0.020 | -0.037 | -0.016 | 0.125** | | bathrooms | 0.004 | -0.007 | 1 | -0.000 | -0.024 | 0.001 | 0.120** | | house_age_years | 0.014 | -0.020 | -0.000 | 1 | -0.023 | -0.001 | -0.223** | | quality_score | 0.019 | -0.037 | -0.024 | -0.023 | 1 | -0.054* | 0.469** | | garage_cars | 0.004 | -0.016 | 0.001 | -0.001 | -0.054* | 1 | 0.050* | | sale_price | 0.715** | 0.125** | 0.120** | -0.223** | 0.469** | 0.050* | 1 | > * p < .05 · ** p < .01 (pairwise deletion) ### Linear regression (OLS) *Group:* Regression · *Dataset:* `housing` · *Analysis key:* `linear_regression` Models a numeric outcome as a linear function of predictors — coefficients, R^2, F and diagnostics. **Inputs used:** `dependent` = sale_price, `predictors` = ['living_area_sqft', 'bedrooms', 'bathrooms', 'house_age_years', 'quality_score', 'garage_cars'] **Model summary** | R | R² | Adjusted R² | SE of estimate | F | df | p | Durbin-Watson | N | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 0.8978 | 0.806 | 0.8053 | 4.872e+04 | 1103 | 6, 1593 | < .001 | 2.065 | 1600 | **Coefficients** | Predictor | B | SE | Beta (std) | t | p | 95% CI low | 95% CI high | VIF | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | (Constant) | 8.912e+04 | 6578 | | 13.55 | < .001 | 7.622e+04 | 1.02e+05 | | | living_area_sqft | 121 | 1.891 | 0.7067 | 64.02 | < .001 | 117.3 | 124.8 | 1.001 | | bedrooms | 1.032e+04 | 857 | 0.133 | 12.04 | < .001 | 8637 | 1.2e+04 | 1.002 | | bathrooms | 1.667e+04 | 1431 | 0.1285 | 11.65 | < .001 | 1.386e+04 | 1.948e+04 | 1.001 | | house_age_years | -918.5 | 46.27 | -0.2192 | -19.85 | < .001 | -1009 | -827.7 | 1.001 | | quality_score | 1.775e+04 | 425 | 0.4621 | 41.76 | < .001 | 1.691e+04 | 1.858e+04 | 1.006 | | garage_cars | 1.015e+04 | 1508 | 0.0744 | 6.73 | < .001 | 7189 | 1.31e+04 | 1.003 | > VIF > 5 signals problematic collinearity. ### Logistic regression *Group:* Regression · *Dataset:* `heart` · *Analysis key:* `logistic_regression` Models a binary outcome; coefficients are log-odds — exponentiate for odds ratios. **Inputs used:** `dependent` = heart_disease, `predictors` = ['age', 'cholesterol', 'resting_bp', 'max_heart_rate', 'st_depression', 'exercise_angina'] **Logistic regression — P(heart_disease = 1)** | Predictor | B | SE | Wald | p | Odds ratio | OR CI low | OR CI high | | --- | --- | --- | --- | --- | --- | --- | --- | | (Constant) | -1.145 | 1.661 | 0.4752 | 0.4906 | 0.3182 | 0.0123 | 8.252 | | age | 0.0166 | 0.0169 | 0.9584 | 0.3276 | 1.017 | 0.9835 | 1.051 | | cholesterol | 0.009 | 0.0029 | 9.434 | 0.0021 | 1.009 | 1.003 | 1.015 | | resting_bp | 0.013 | 0.0076 | 2.933 | 0.0868 | 1.013 | 0.9981 | 1.028 | | max_heart_rate | -0.0238 | 0.0068 | 12.36 | < .001 | 0.9765 | 0.9637 | 0.9895 | | st_depression | 0.6878 | 0.1745 | 15.54 | < .001 | 1.989 | 1.413 | 2.801 | | exercise_angina | 1.016 | 0.5025 | 4.085 | 0.0433 | 2.761 | 1.031 | 7.392 | **Model fit** | -2 Log likelihood | McFadden R² | Nagelkerke R² | Accuracy | N | | --- | --- | --- | --- | --- | | 136.7 | 0.2619 | 0.3201 | 0.925 | 320 | ### Principal component analysis *Group:* Dimension reduction · *Dataset:* `wine` · *Analysis key:* `pca` Reduces correlated numeric variables to a few orthogonal components with an explained-variance table. **Inputs used:** `columns` = ['alcohol', 'volatile_acidity', 'sulphates', 'citric_acid', 'residual_sugar', 'ph', 'density'], `n_components` = 3, `rotation` = none **Visualization:** generates a line (Scree plot). Use **Visualize** / it appears automatically. **KMO and Bartlett's test** | Measure | Value | | --- | --- | | Kaiser-Meyer-Olkin (KMO) | 0.5003 | | Bartlett's chi-square | 10.58 | | df | 21 | | p | 0.9703 | > KMO ≥ 0.6 and significant Bartlett support factorability. **Total variance explained** | Component | Eigenvalue | % of variance | Cumulative % | | --- | --- | --- | --- | | 1 | 1.077 | 15.39 | 15.39 | | 2 | 1.042 | 14.89 | 30.28 | | 3 | 1.033 | 14.76 | 45.04 | | 4 | 1.002 | 14.31 | 59.35 | | 5 | 0.9696 | 13.85 | 73.21 | | 6 | 0.9461 | 13.52 | 86.72 | | 7 | 0.9296 | 13.28 | 100 | ### Process capability (Cp / Cpk) *Group:* Quality & SPC · *Dataset:* `wine` · *Analysis key:* `process_capability` Quality index comparing process spread to specification limits; Cpk >= 1.33 is capable. **Inputs used:** `column` = alcohol, `lsl` = 9, `usl` = 12 **Process capability — alcohol** | Cp | Cpk | Mean | SD | % out of spec | Expected PPM defective | | --- | --- | --- | --- | --- | --- | | 0.4601 | 0.4418 | 10.44 | 1.087 | 17.5 | 1.681e+05 | > Cp/Cpk ≥ 1.33 is generally capable; ≥ 1.67 is excellent. Cpk accounts for centering. ### Shewhart I-MR control chart *Group:* Quality & SPC · *Dataset:* `wine` · *Analysis key:* `control_chart_imr` Statistical process-control chart for individual measurements, with centre line and 3-sigma control limits plus out-of-control flags. **Inputs used:** `column` = ph **Visualization:** generates a line (I-chart — ph). Use **Visualize** / it appears automatically. **Individuals (I-MR) control chart — ph** | Center line | UCL | LCL | MR-bar | # out of control | N | | --- | --- | --- | --- | --- | --- | | 3.308 | 3.737 | 2.88 | 0.1612 | 5 | 1200 | > Points beyond the control limits signal a special-cause (out-of-control) process. ### Odds ratio (2x2) *Group:* Clinical trials · *Dataset:* `heart` · *Analysis key:* `odds_ratio` Association between an exposure and a binary clinical outcome, with a 95% CI. **Inputs used:** `exposure` = sex, `outcome` = heart_disease **Odds ratio — sex → heart_disease** | Odds ratio | 95% CI low | 95% CI high | p | | --- | --- | --- | --- | | 1.055 | 0.4794 | 2.322 | 0.894 | > OR > 1 = higher odds of the outcome with exposure; CI excluding 1 = significant. ### ROC / AUC *Group:* Clinical trials · *Dataset:* `heart` · *Analysis key:* `roc_auc` Discrimination of a continuous marker for a binary outcome; AUC of 0.5 is chance, 1.0 is perfect. **Inputs used:** `predictor` = st_depression, `outcome` = heart_disease **Visualization:** generates a line (ROC curve). Use **Visualize** / it appears automatically. **ROC / AUC — st_depression predicting heart_disease** | AUC | Discrimination | N | | --- | --- | --- | | 0.7457 | fair | 320 | > AUC: 0.5 = chance · 0.7 fair · 0.8 good · 0.9 excellent discrimination. ### Kaplan-Meier survival *Group:* Survival · *Dataset:* `clinical` · *Analysis key:* `kaplan_meier` Non-parametric survival curve from time-to-event data with censoring, with median survival. **Inputs used:** `time` = months, `event` = event **Visualization:** generates a line (Survival function S(t)). Use **Visualize** / it appears automatically. **Kaplan-Meier survival — months** | Metric | Value | | --- | --- | | 75% survival time | 8.8 | | 50% survival time | 21.3 | | 25% survival time | 49.4 | | Events | 178 | | Observations | 300 | > Median survival = 21.3. Step function of survival probability over time. ### Cox proportional-hazards regression *Group:* Survival · *Dataset:* `clinical` · *Analysis key:* `cox_regression` Semi-parametric survival model; exp(coef) is the hazard ratio per unit of a covariate. **Inputs used:** `time` = months, `event` = event, `predictors` = ['age', 'biomarker', 'treatment'] **Cox proportional-hazards regression — months** | Predictor | Coef. | SE | p | Hazard ratio | HR CI low | HR CI high | | --- | --- | --- | --- | --- | --- | --- | | age | 0.0278 | 0.0062 | < .001 | 1.028 | 1.016 | 1.041 | | biomarker | 0.4065 | 0.0826 | < .001 | 1.502 | 1.277 | 1.766 | | treatment | -0.8483 | 0.1591 | < .001 | 0.4281 | 0.3134 | 0.5848 | > n = 300. HR > 1 = higher hazard (shorter survival). Assumes proportional hazards over time. ### Augmented Dickey-Fuller unit-root test *Group:* Time series · *Dataset:* `macro_ts` · *Analysis key:* `adf_test` Tests a series for a unit root (non-stationarity) before ARIMA/VAR; p < 0.05 rejects a unit root. **Inputs used:** `column` = gdp, `date` = date, `frequency` = M **Augmented Dickey-Fuller — gdp** | ADF statistic | p | Lags used | 5% critical | | --- | --- | --- | --- | | -0.6692 | 0.8546 | 0 | -2.874 | > p < .05 → stationary (reject the unit-root null). ### GARCH(1,1) volatility model *Group:* Time series · *Dataset:* `macro_ts` · *Analysis key:* `garch_family` Models time-varying volatility (conditional heteroskedasticity) of a return series, with persistence and half-life. **Inputs used:** `column` = market_return, `model` = GARCH, `dist` = t, `date` = date, `as_returns` = yes, `order` = 1 **GARCH volatility model — market_return** | Parameter | Estimate | SE | p | | --- | --- | --- | --- | | mu | -0.0645 | 0.0902 | 0.4746 | | omega | 0.8894 | 0.3418 | 0.0093 | | alpha[1] | 0.267 | 0.1237 | 0.0309 | | beta[1] | 0.4117 | 0.1614 | 0.0107 | | nu | 9.567 | 5.293 | 0.0707 | > GARCH with t errors on returns. Log-likelihood = -446.7 · AIC = 903.5. Persistence α+β = 0.679; volatility half-life ≈ 1.8 periods. ### Exponential smoothing (Holt-Winters / ETS) *Group:* Forecasting · *Dataset:* `macro_ts` · *Analysis key:* `exp_smoothing` ETS forecasting with level and trend components and a forward forecast. **Inputs used:** `column` = gdp, `date` = date, `frequency` = M, `trend` = add, `seasonal` = none, `seasonal_periods` = 12, `horizon` = 12 **Visualization:** generates a line (gdp — actual vs forecast). Use **Visualize** / it appears automatically. **Holt (double) exponential smoothing — gdp** | Property | Value | | --- | --- | | Trend | add | | Seasonal | none | | Seasonal periods | — | | SSE | 204.5 | | AIC | -30.41 | | Observations | 240 | > Exponential smoothing weights recent observations more heavily; add trend (Holt) and seasonal (Holt-Winters) components as the data needs. **Smoothing parameters** | Parameter | Estimate | | --- | --- | | α (level) | 1 | | β (trend) | 0 | ## 16. Data sources and connectors Every analysis in InstaBizIntel starts from a **dataset** inside a project. The **Datasets** page is the single hub for bringing data in — from files, the web, databases, live market feeds, public research repositories, or documents. You never leave the browser and nothing is exported. ![The Datasets page — every import source is a tile; imported datasets appear as cards below.](/manual-img/02-datasets.png) ### 16.1 The simplest way in — upload a file 1. Open a project, click **Datasets** in the left sidebar. 2. Click **Upload a file** and choose your file. 3. Supported formats: **CSV, TSV, Parquet, JSON/JSONL, Excel (.xlsx/.xlsm/.xls/.ods, multi-sheet), SPSS (.sav/.zsav/.por), SAS (.sas7bdat/.xpt), Stata (.dta), XML, SQLite**. 4. The file is profiled automatically (row count, column types, distinct values) and appears as a card. **Expected output:** a dataset card showing rows, columns and the table name. *Tip: for a multi-sheet Excel workbook, each non-empty sheet becomes its own dataset.* Two more sources live on the same page: the **Sample library** tile (15 ready-made teaching datasets across BI, panel, time-series, PLS-SEM, qualitative, survival, compare-means and machine-learning categories — see §2.3), and a **Download CSV** / **Synthesize** action on every dataset page (§2.4). ### 16.2 From the web, cloud and databases - **From URL / Google Sheets** — paste a link to a CSV/Excel or a shared Google Sheet. - **Cloud storage** — AWS S3 and S3-compatible stores (Backblaze B2, Wasabi, MinIO — set the endpoint and path/virtual-hosted URL style) plus Azure Blob, with keys or public URLs. - **From a web page** — scrape HTML tables from any public page. - **Paste / enter data** — paste a range straight from Excel; the delimiter is auto-detected. - **Connections** (Data → Connections) — live database connectors: **Postgres, MySQL, SQL Server, generic ODBC, MongoDB, REST, OData**. Create a connection once, then import tables or run custom SQL. ![The Connections page — reusable live connectors to databases and APIs.](/manual-img/82-connections.png) ### 16.3 Live market data (crypto, stocks, indices, forex, commodities) Under the **Market data** tile: 1. Choose a source — **Crypto (Binance)** or **Stocks · Indices · Forex · Commodities (Yahoo)**. 2. Enter a symbol (e.g. `BTCUSDT`, `AAPL`, `^GSPC` for the S&P 500, `EURUSD=X`, `GC=F` for gold), an interval and a date range. 3. Click **Extract**. **Expected output:** a tidy OHLCV time series (date, open, high, low, close, volume), ready for the Forecasting Lab. The symbol box is a searchable picker — type *Apple*, *S&P 500*, *Gold* or *EUR/USD* (Yahoo) or *BTC*, *ETH* (Binance) and choose from the list. ### 16.4 Macro, sentiment and on-chain data The **Macro · sentiment · on-chain** tile provides keyless research feeds: - **FRED macro** — Treasury yields, VIX, CPI, credit spreads, money supply, and more. - **Crypto Fear & Greed Index** — the daily 0–100 sentiment series. - **Bitcoin on-chain** — hash-rate, transactions, difficulty, miners' revenue, market cap (from blockchain.info). ### 16.5 Public research data (economics, health, news, filings) The **Research data** tile connects to standard scholarly repositories, all keyless: | Source | What you get | |---|---| | World Bank | Country × year development/macro indicators (GDP, inflation, population, CO₂, …) | | Our World in Data | Any grapher series by its slug (life-expectancy, energy, health, …) | | WHO GHO | Global health indicators (life expectancy, mortality, obesity, …) | | ClinicalTrials.gov | Trial registry records (status, phase, enrolment, sponsor) | | SEC EDGAR | A company's reported financial concept over time (Revenues, NetIncomeLoss, Assets…) | | GDELT | Global news article metadata for a query | ### 16.6 Documents, media and references (for qualitative & reviews) - **Documents & transcripts** — upload PDF, DOCX, TXT/MD/RTF, VTT/SRT; audio/video (MP3, WAV, M4A, MP4, WebM, OGG, FLAC, AAC, MOV) is auto-transcribed. - **Web / YouTube / Reddit capture** — pull readable text, a video transcript, or a public thread. - **Reference library (RIS / BibTeX)** — import EndNote (.enw), Zotero/Mendeley (.ris, .bib) exports; each reference becomes a codeable document. ### 16.7 Feature engineering — technical indicators On any OHLCV dataset card, click **✨ Add technical indicators** to append (causally, no look-ahead) any of 25 indicators — **log / % returns, SMA(10/20/50), EMA(12/26), RSI(14), MACD (+signal, histogram), Bollinger bands (%B, bandwidth), ATR(14), OBV, stochastic %K/%D, realized volatility (10/20), rolling skew/kurtosis, momentum(10)** — as a new derived dataset you can use as exogenous predictors. --- ## 17. Forecasting Lab The **Forecasting Lab** (Research → Forecasting Lab) is a research-grade time-series forecaster built around the modern **decomposition–hybrid** approach and, crucially, the **statistical model-comparison tests** that Q1 journals expect. ![The Forecasting Lab — dataset, decomposition, model pool, evaluation mode and (for the Benchmark mode) a full model-comparison leaderboard with Model Confidence Set and Diebold–Mariano tests.](/manual-img/21-forecast-benchmark.png) ### 17.1 Run your first forecast (simple) 1. Pick a **Dataset** (a time series such as the market/BTC data from §16.3) and the **Target column** to forecast; optionally a **Date column**. 2. Leave **Decomposition** on **CEEMDAN (recommended)** and the default model pool. 3. Set a **Forecast horizon** (e.g. 14 steps). 4. Click **▶ Run hybrid forecast**. **Expected output:** a KPI strip (RMSE improvement vs ARIMA, components), a forecast-vs-actual chart, the IMF decomposition chart, and per-component model-selection + accuracy tables. ### 17.2 How the hybrid works The series is decomposed into components; each component's complexity is measured (permutation entropy, Lempel–Ziv); an **Adaptive Model Selection** step assigns the best forecaster per component from the pool; forecasts are recombined and benchmarked against ARIMA. The decomposition is fit on the training span only, so there is **no look-ahead leakage**. ### 17.3 The model pool Statistical & ML: naïve, mean, drift, linear, ridge, random forest, gradient boosting, SVR, k-NN, ARIMA, ETS, Theta. Light neural (CPU-fast): **DLinear, NLinear, N-BEATS, N-HiTS, TiDE**. Deep/foundation (GPU-badged, run slowly on CPU): **LSTM, BiLSTM, PE-BiLSTM, PatchTST, Chronos, TimesFM**. Decompositions: EMD, EEMD, **CEEMDAN**, VMD, Wavelet, STL, MSTL, SSA, EWT, Fourier low-pass. **Prophet (added 29 Sep 2026).** Meta's Prophet is in the pool: a piecewise-linear trend with automatically detected changepoints plus Fourier seasonality at the period you set, fitted with Stan. Its model card in the Lab states its scope: seasonal business series such as sales, demand, traffic, footfall and macro indicators. It is not suited to asset prices or returns because it ignores autocorrelation and volatility clustering; the ARIMA, GARCH-aware and neural models are the right choice there. The Python notebook export includes a **Prophet-style additive model** (the same trend-plus-seasonality structure fitted by ridge regression, no Stan install needed) in its hold-out race. ### 17.4 Evaluation modes - **One-shot (multi-step)** — a fixed-origin hold-out; fast; good default. - **Rolling one-step (paper protocol)** — expanding window, re-decomposed each step. - **Benchmark (compare all + MCS/DM)** — the flagship: every pooled model *plus* the hybrid is scored on a hold-out with the full metric suite (RMSE, MAE, MAPE, sMAPE, MASE, RMSSE, Theil's U2, directional accuracy, R²oos), then ranked into a **leaderboard** with **Diebold–Mariano** significance stars and the **Model Confidence Set** (the models statistically indistinguishable from the best). To run a benchmark: select **Benchmark** under Evaluation, tick a few models, and **Run**. **Expected output:** the leaderboard shown above (best model ★, DM vs best, ✓ in 90% MCS), a top-models forecast chart, and a per-model RMSE bar. ### 17.5 Multivariate, tuning and the publication report - Add **Exogenous inputs** (volume, on-chain, sentiment, indicators) for multivariate forecasting. - Tick **Hyperparameter tuning (MOTPE / Optuna)** to auto-tune the lag window. - Click **📄 Publication report (.docx)** (optionally **✨ + AI abstract**) to export a full manuscript — Abstract, Data, Methodology, Results tables, diagnostics, a reproducibility appendix and references — with your charts embedded. --- ### 17.6 Auto-ensemble with conformal prediction intervals Below the run form, the **🎯 Auto-ensemble + conformal prediction interval** card back-tests the models you ticked over rolling origins, builds an inverse-RMSE **top-3 ensemble** and wraps the forecast in a distribution-free **90 % conformal band** whose *empirical coverage* is quoted. Ticking a foundation model (Chronos / TimesFM) in the pool sets *include foundation models* (slower). Click **🎯 Run ensemble**; **Expected output:** ensemble RMSE (backtest), conformal half-width, the members and their weights, and a *History + ensemble forecast with conformal band* chart. See §28.5. ## 18. Literature Review and Meta-Analysis Open **Research → Lit Review + Meta**. This is an end-to-end systematic-review workbench: search free scholarly sources, screen with PRISMA, appraise quality, map the field, and draft a citation-safe manuscript — with a **full meta-analysis engine** alongside. ![The Literature Review + Meta-Analysis workbench — search, screen, and run a meta-analysis across eight scholarly sources.](/manual-img/30-literature.png) ### 18.1 Search & screening (tab 1) 1. Type a query and pick sources — **Crossref, OpenAlex, arXiv, Semantic Scholar, PubMed, Europe PMC, DOAJ, Google Scholar**. 2. Click **Search**. Results are de-duplicated by DOI. **Expected output:** a **PRISMA 2020 flow diagram** and a list of records with Include/Exclude toggles. 3. Mark records **Include**, then **Save included as dataset** to persist them for extraction or coding. **Reading a paper.** Click anywhere on a result card (a ↗ marks cards with a link) to **expand its full abstract** and **open the full paper in a new tab** — the open-access PDF where available, otherwise the publisher/DOI page; where no full text exists, the abstract is shown and a note explains it. The Include/Exclude buttons keep working independently. The screening result is drawn as a full **PRISMA 2020 flow diagram**: ![The PRISMA 2020 flow diagram in the Search & screening tab — Identification → Screening → Included, with the duplicate-removed and excluded side branches and per-source counts, above the screenable paper cards.](/manual-img/35-prisma-diagram.png) **How to read the PRISMA diagram.** It follows the standard three-stage layout (stage labels run down the left edge): - **Identification** — *Records identified from databases (n = …)*, with the per-source breakdown (Crossref, OpenAlex, arXiv, … counts) shown inside the box; the side branch reports *Duplicate records removed (n = …)*. - **Screening** — *Records screened (n = …)* (i.e. after de-duplication); the side branch reports *Records excluded (n = …)* and *Not yet screened (n = …)*. - **Included** — *Studies included in review (n = …)*. The diagram **updates live** as you click Include/Exclude on the record cards, and the same diagram appears in the **Review builder** tab. It is a genuine PRISMA 2020 flowchart (not just a summary strip), so it can be exported straight into a systematic-review manuscript. **Living reviews.** Under the results sits the **🔄 Living reviews** card — enter the query to monitor, choose a schedule (Weekly Mon 06:00 / Daily 06:00 / Monthly 1st / Twice a month), optionally an e-mail, and **+ Monitor**. Each run re-searches the chosen sources and surfaces only records not seen before; **▶ Run now** runs it immediately. See §29.2. ### 18.2 Review builder — the autonomous agent (tab 2) Type a topic and click **🤖 Run review agent**. In one step it searches every source, de-duplicates, clusters papers into themes and computes bibliometrics — running as a background job so large searches never time out. ![The Review builder output — PRISMA flow, a thematic cluster diagram (keyword co-occurrence), bibliometrics, appraisal tables and a grounded manuscript in your chosen citation style.](/manual-img/31-review-result.png) You then get, on one page: - A **PRISMA screening flow** and bibliometric summary (span, sources, authors). - A **thematic cluster diagram** — a VOSviewer-style keyword co-occurrence network coloured by theme — plus cluster and study-characteristics tables and bibliometric charts (annual production, top venues/authors, co-authorship network, three-fields Sankey). - **Quality appraisal**: choose an instrument — **MMAT 2018, Cochrane RoB 2, ROBINS-I, Newcastle-Ottawa, AMSTAR-2, GRADE** — and click **Appraisal table** for a robvis-style **traffic-light** matrix (🟢 low / 🟡 some concerns / 🔴 high / ⚪ unclear). Tick **AI-suggest** to have the AI propose ratings grounded on the abstracts. - **📄 Generate review manuscript (.docx)** — a grounded, **citation-safe** draft (the AI may only cite retrieved papers, never fabricated ones) in your chosen **citation style: APA 7, Harvard, MLA 9, Chicago, IEEE, Vancouver, AMA or ACS**. ### 18.3 Bring your own papers (SCOPUS / Web of Science) In the **Review builder** tab click **⬆ Upload paper PDFs (SCOPUS / WoS)** to add papers you exported from SCOPUS, Web of Science or elsewhere. Their metadata is extracted into the corpus. Uploaded PDFs are held **on local disk only (never Backblaze) and auto-delete after 24 hours**; a **Finish & delete PDFs** button purges them immediately. Uploaded PDFs can also be fed to the **🧪 Dual-LLM extraction** (§29.3). ### 18.4 Meta-analysis (tab 3) 1. Point it at a dataset whose rows are studies and whose columns hold effect sizes. 2. Choose an **effect type** — generic (effect + SE/variance/CI), **standardised mean difference (Hedges' g)**, **odds ratio** or **risk ratio** (2×2 counts), or **correlation** (r, n) — and a **model** (fixed or random effects). Map the columns. 3. Click **▶ Run meta-analysis**. **Expected output:** pooled effect with 95% CI and prediction interval, heterogeneity (Q, I², τ²), a **forest plot** and **funnel plot**, publication-bias diagnostics (**Egger's test, trim-and-fill**) and optional **meta-regression**. ![The Meta-analysis tab — pooled effect 0.53 [0.43, 0.62] with the forest plot (per-study effects + pooled diamond) and the funnel plot.](/manual-img/32-lit-meta.png) --- ## 19. Rigor Guard, reproducibility and preregistration Open **Research → Rigor Guard**. This page is the research-integrity hub. Its first card is the *reviewer pre-flight* described below; the same page also hosts the **Analysis runs** ledger (Trust Score, Inspector, Reproducibility Bundles, Zenodo), **Method advisor**, **Rigor Guard v2** (statcheck/GRIM, citation verifier, overclaim linter), **Reporting-standard coverage & Reviewer 2**, **Executable preregistration**, **Multiverse / specification-curve**, **🧭 Causal DAG**, **🏆 Hypothesis tournament** and **🚦 Publication gate** — see §19.6 and §27–§30. ![Rigor Guard — a red/amber/green submission-readiness report with a remedy for every issue, plus reporting checklists, a reproducibility manifest and preregistration.](/manual-img/41-rigor-result.png) ### 19.1 Run the pre-flight 1. Pick a **Dataset**, the **intended design** (regression, logistic, t-test, ANOVA, correlation, chi-square, general) and the **outcome**; for regression, tick the **predictors**. 2. Click **🛡 Run submission pre-flight**. **Expected output:** a **readiness score /100** (green ≥ 80, amber ≥ 60, red below) and a table of checks — statistical **power / sample size**, **missing data**, **normality**, **multicollinearity (VIF)**, **outliers**, **multiple comparisons**, **class balance**, **reliability**, and effect-size/CI reminders — each ✅/⚠️/❌ with a concrete remedy. ### 19.2 Reporting-standard checklists Choose **PRISMA 2020, PRISMA-ScR, CONSORT, STROBE** or **TRIPOD** and click **Load checklist** to get the fillable item list to attach as a supplementary file. ### 19.3 Reproducibility manifest Click **⬇ Reproducibility manifest (.json)** to export software versions, the fixed random seed, dataset metadata and your analysis configuration — a replication record for your supplementary materials. ### 19.4 Preregistration and the deviation report In the **Preregistration (free-text plan)** card, write your title, question, hypotheses, design and **planned analyses**, then **🔒 Save & lock**. After analysing, select the locked plan, enter the analyses you actually ran, and **Diff** to get a deviation report — ✅ confirmatory, ⚠️ planned-but-not-run, 🔶 exploratory — the disclosure reviewers now expect. ### 19.5 Grounded AI methodology critique Click **✨ AI methodology critique** to have the AI run the deterministic pre-flight and then critique the design, name threats to validity, and recommend tests and a robustness plan — using **only** the real findings and your data schema (it never invents variables, results or citations). --- ### 19.6 Everything else on the Rigor Guard page The pre-flight is only the first card. Scrolling down the same page you will find, in order: | Card | What it does | Manual | |---|---|---| | **🧭 Method advisor** | Assumption-checked, ranked, runnable test recommendations | §28.1 | | **🛡 Analysis runs** | The audit-run ledger — Trust Score, Inspector **Verify**, **Bundle** (RO-Crate), Zenodo draft, **🧪 Robustness**, **📝 APA** | §27.1–27.4, §28.2–28.3 | | **🔬 Rigor Guard v2** | statcheck + GRIM numeric re-computation, Crossref citation verifier, overclaim linter | §27.5 | | **📋 Reporting-standard coverage & Reviewer 2** | APA JARS-Quant / Hair PLS-SEM / BARG-WAMBS / GRAMMS / CONSORT coverage from a run's output; hostile-reviewer questions | §27.6 | | **🔒 Executable preregistration** | Lock a runnable plan, **▶ Run plan (confirmatory)**, **📒 Specification ledger**, **Δ Auto deviation report** | §27.7 | | **🌌 Multiverse / specification-curve** | Every reasonable specification, robust-vs-fragile verdict | §27.8 | | **🧭 Causal "why"** | DAG, backdoor-adjusted ATE, refutation tests, E-value, skeleton discovery | §29.1 | | **🏆 Hypothesis tournament** | generate → hostile critique → rank → refine, literature-verified | §30.1 | | **🚦 Publication gate** | Claim typing, reviewer gate, Methods from the execution log, APA Word export, AI-use / NIH DMS / availability statements | §30.2 | The free-text **Preregistration** card of §19.4 is kept for narrative plans; for anything you intend to run, prefer the executable preregistration. ## 20. Advanced research methods (causal inference & latent variables) These appear in the **Statistics** workbench (Research → Statistics), grouped and runnable exactly like every other analysis — pick the analysis, map the columns, run. ![The Statistics workbench groups 300+ analyses; the newest are under Causal inference, plus latent-variable and Bayesian methods.](/manual-img/10-statistics.png) ### 20.1 Causal inference (quasi-experiments) | Analysis | Use it for | |---|---| | **Difference-in-Differences** | Treatment effect from a treated×post design under parallel trends | | **Event study (dynamic DiD)** | Leads/lags around treatment — tests parallel trends and dynamic effects | | **Regression discontinuity** | The local effect at a cutoff (sharp RD, triangular kernel) | | **Propensity-score matching / IPW** | ATT + ATE with a covariate-balance table (observational causal effects) | | **Synthetic control** | A weighted "synthetic" comparison unit for case studies | | **Mediation analysis (bootstrap)** | The indirect effect a·b with a bootstrap CI (Hayes model 4) | | **Moderation analysis** | Interaction X×W with simple slopes | | **Specification-curve (multiverse)** | Robustness of a focal effect across every covariate specification | | **ARDL / Markov-switching / Local projections** | Long-run relationships, regime-switching, and impulse responses | Two causal tools live outside the Statistics catalogue: the **🧭 Causal DAG estimator** (state or discover a graph, backdoor-adjusted ATE, refutation tests, E-value, what-if) in Rigor Guard — §29.1 — and the **🌌 Multiverse / specification-curve** card in Rigor Guard (§27.8), which supersedes the single-analysis *Specification-curve* entry above for disclosure purposes. Every estimate from either becomes an audit run with a Trust Score (§27). ### 20.2 Latent-variable and Bayesian methods - **Confirmatory factor analysis (CFA)** — test a measurement model (CFI/TLI/RMSEA, loadings, AVE & CR) by typing `F1=~x1,x2,x3; F2=~x4,x5,x6`. - **Latent class / profile analysis (LCA / LPA)** — person-centred typologies (BIC, entropy, class profiles). - **Bayesian t-test & correlation** — JZS **Bayes factors** (evidence for H1 vs H0). - **Network analysis (SNA)** — centrality + community detection on an edge list, with a network diagram. - **Conformal prediction** — distribution-free prediction intervals with a coverage guarantee. - **Embedding (UMAP / t-SNE / PCA)** — a 2-D map of high-dimensional data for visualization and cluster discovery (Dimension reduction group). - **Item Response Theory (2PL)** — item discrimination and difficulty with characteristic curves, for scale validation (Scale group). - **Market-basket (association rules)** — apriori rules (support/confidence/lift) from transaction data (Classify group). ### 20.3 Prediction Profiler (interactive what-if & optimization) Open **Research → Prediction Profiler**. This is the JMP-style interactive profiler: fit a model of a response on several predictors, then see how the prediction moves as you change each predictor — and optimize. ![The Prediction Profiler — per-predictor response curves with editable reference values, model R², and a desirability optimizer.](/manual-img/90-profiler.png) 1. Pick a **Dataset**, the **Response** to predict, a **Model** (gradient boosting / random forest / linear) and the **Predictors**. 2. Click **▶ Fit & profile**. **Expected output:** the predicted response and model **R²**, and one panel per predictor showing the **response curve** as that predictor varies while the others stay fixed, with the current reference value marked. 3. **Change a reference value** in any panel to update the prediction live (what-if analysis). 4. Choose a **Goal** — maximise, minimise or hit a target — and click **🎯 Optimize** to jump the reference point to the predictor settings that best achieve it (**desirability optimization**). ### 20.4 Design of Experiments and choice modeling These run from **Research → Statistics** (they appear under **Design & Quality** and **Consumer & Market Research**). The design generators build a plan from parameters — pick any dataset, choose the analysis, set the factors, and **Run**. ![Running a D-optimal custom design in the Statistics workbench — the generated run table is shown with its D-efficiency.](/manual-img/92-doe.png) - **D-optimal custom design** — a model-driven optimal design (main-effects, interaction or quadratic) for a chosen number of factors and runs, when a classical factorial/RSM design is impractical. Output: a coded run table (−1/+1) with its relative D-efficiency. - **Taguchi orthogonal array** — robust-design screening arrays (L4/L8/L16/L32 for two levels; L9/L27 for three) for a chosen number of factors. Output: the run table to map to your real settings. - **Conjoint analysis (part-worths)** — ratings-based conjoint on a dataset with a preference/rating column and attribute columns. Output: **part-worth utilities** per attribute level and an **attribute-importance** table + chart (which attributes drive choice). - **Discrete choice (conditional logit)** — choice-based conjoint on choice data (a chosen 0/1 flag, a choice-set ID and alternative attributes). Output: utility weights per attribute with significance. - **MaxDiff (best-worst scaling)** — preference scores from best/worst choices. Output: an item-score ranking + chart. *(These, like every one of the 300+ analyses, are also runnable by the AI Analyst — just ask, e.g. "run a conjoint analysis of the survey with rating on brand, price and size".)* ### 20.5 Reliability engineering, spatial regression & double-LASSO Three more JMP-class methods run from **Research → Statistics**. Reliability lives under a dedicated **Reliability Engineering** family in the sidebar; spatial regression sits under **Regression** and double-LASSO under **Causal Inference**. Pick a dataset, choose the analysis, set the columns, and **Run**. ![Weibull reliability (life data) in the Statistics workbench — shape β, scale η, MTTF, B10 life and the reliability curve R(t).](/manual-img/94-weibull.png) - **Weibull reliability (life data)** — fits a Weibull life distribution by maximum likelihood to time-to-failure data, with **right-censoring** supported (an *event* column: 1 = failed, 0 = still running/censored). Output: shape **β** (β<1 = infant mortality, β≈1 = random, β>1 = wear-out), scale/characteristic life **η**, **MTTF** (mean life), **B10** life (time by which 10 % fail), the failure-pattern verdict, an optional reliability-at-time value, and the **R(t) reliability curve**. Use it for warranty analysis, component life, MTBF/MTTF, and maintenance planning. - **Accelerated life test (ALT)** — a Weibull AFT model that relates life to a stress variable (Arrhenius for temperature, inverse-power for voltage/load, or linear) so you can **extrapolate median life at a use-condition stress** from data collected at higher stresses. Output: the stress coefficient and the extrapolated life at your specified use stress. - **Degradation analysis** — fits per-unit degradation paths, projects each unit's crossing of a failure threshold into a pseudo-failure time, then fits a Weibull to those times. Output: β, η, MTTF and B10 from degradation data when few or no outright failures were observed. - **Spatial regression (lag / error)** — spatial-econometric models (spatial-lag **GM_Lag** with autoregressive coefficient ρ, or spatial-error **GM_Error** with λ) on data with coordinate columns, using a k-nearest-neighbour, row-standardised spatial weights matrix. Use it when observations are geographically linked and OLS residuals are spatially autocorrelated (regional economics, epidemiology, real-estate pricing). - **Double-LASSO (causal effect)** (post-double-selection) — Belloni–Chernozhukov–Hansen causal estimation: LASSO selects controls that predict the outcome **and** controls that predict the treatment, takes their union, then runs OLS of the outcome on the treatment plus selected controls with **HC1 robust** errors. Output: the **treatment effect**, its robust SE, confidence interval and p-value, plus which controls survived selection. Use it for a credible treatment effect when you have many candidate controls. *(All five are also runnable by the AI Analyst — e.g. "fit a Weibull to the failure_hours column using event as the censoring indicator" or "run a double-LASSO of sales on the promo treatment controlling for all the store variables".)* ### 20.6 Running an analysis end-to-end through the AI Analyst Every one of the 300+ analyses can be run in plain English by the **AI Analyst** (`Analyze → AI Analyst`) — you don't have to open the Statistics workbench or know the exact parameters. The agent inspects your datasets, picks the right analysis, runs it on the server, shows the result tables and charts, and writes the interpretation. Worked example — a Weibull reliability study run entirely by the agent: 1. Open **AI Analyst** and type a request naming the dataset, the analysis and the columns — e.g. *"Using the Component life test (demo) dataset, run a Weibull reliability (life data) analysis with failure_hours as the time-to-failure and event as the censoring indicator. Then explain the shape β, characteristic life, MTTF, B10 and whether the components show wear-out or infant-mortality."* 2. The agent lists your datasets (**catalog**), looks up the analysis and its parameters (**list_statistics**), then calls **run_statistics** — it maps your wording ("time-to-failure", "censoring indicator") onto the analysis's real parameters automatically, so you can describe columns in natural language. 3. It runs the fit on the server, streams the **result table and the R(t) reliability curve** into the chat, and then explains the numbers in plain English. ![The AI Analyst running a Weibull reliability analysis end-to-end — it ran the analysis on the Component life test dataset and reported the fitted shape β = 1.82, characteristic life η = 6,144 h, MTTF = 5,461 h and B10 = 1,786 h, with a wear-out interpretation.](/manual-img/95-agent-weibull.png) You can then ask follow-ups in the same chat ("what is the reliability at 3,000 hours?", "now fit an accelerated life model using temp_c as the stress"), export the whole thing to **Word or PDF** from the top of the AI Analyst, or add any chart to a report. The same pattern works for any analysis — regression, SEM, forecasting, clustering, DOE, causal inference and the rest — just describe what you want. ### 20.7 Bayesian statistics with Bayes factors Under a new **Bayesian** family in `Research → Statistics`, the platform reports **Bayes factors** — the weight of evidence for the effect (H₁) versus the null (H₀) — alongside the classical statistics. A Bayes factor answers a question p-values cannot: it can provide evidence *for* the null, and it quantifies *how much* the data favour one hypothesis over the other on the Jeffreys scale (BF > 3 moderate, > 10 strong, > 30 very strong, > 100 extreme). ![Bayesian one-way ANOVA — BF₁₀ with evidence category, the classical F and p for reference, and the group means.](/manual-img/97-bayes.png) - **Bayesian one-way ANOVA** — evidence for a group-mean difference. Output: BF₁₀, BF₀₁, evidence category, classical F/p, group means. - **Bayesian repeated-measures ANOVA** — evidence for a within-subject condition effect (subject held as a blocking factor). - **Bayesian linear regression** — evidence that the predictors jointly beat an intercept-only model; reports BF₁₀ with R²/adj R². - **Bayesian correlation (JZS)** — the exact JZS Bayes factor for a non-zero Pearson correlation. - **Bayesian contingency table** — evidence for association vs. independence in a two-way table. Bayes factors use the exact JZS prior where a closed form exists (correlation, and the existing Bayesian t-test) and the Schwarz (BIC) approximation, BF₁₀ ≈ exp((BIC_null − BIC_alt)/2), for the model-comparison procedures — the standard approach for reporting Bayes factors in applied research. This closes the SPSS Bayesian-statistics gap. ### 20.8 Life Distribution suite (multi-distribution reliability) The **Life distribution (multi-fit)** analysis (Reliability Engineering family) extends the single Weibull fit toward JMP's Life Distribution platform: it fits **several life distributions at once** — Weibull, lognormal, exponential, log-logistic — and ranks them so you pick the best-supported model rather than assuming Weibull. ![Life distribution multi-fit — AIC comparison across distributions, best-fit parameters with 95% profile-likelihood CIs, the B1/B10/B50/B90 distribution profiler, and the reliability curve.](/manual-img/96-life-dist.png) Outputs: an **AIC comparison table** (ΔAIC < 2 ≈ comparable fits), the **best-fit parameters with 95% profile-likelihood confidence intervals**, a **distribution profiler** giving the B1/B10/B50/B90 life quantiles (the time by which 1/10/50/90 % of units fail), and the fitted **reliability curve R(t)**. Right-censoring is supported through an event column (1 = failed, 0 = still running). A companion **Parametric survival regression (AFT)** fits an accelerated-failure-time model with a choice of distribution (Weibull / lognormal / log-logistic), reporting each covariate's time-ratio effect on survival and the model C-index. Both are runnable by the AI Analyst — e.g. *"compare life distributions for failure_hours using event as the censoring flag"*. ### 20.9 Definitive Screening Designs (DSD) Under **Design & Quality (DOE / SPC)**, the **Definitive screening design (DSD)** generator builds a three-level screening design in only ~2·(factors)+1 runs — JMP's signature design. Its key property: main effects are orthogonal to each other *and* to two-factor interactions, and it can even estimate quadratic (curvature) effects — something a classical fractional factorial cannot. ![A 6-factor Definitive Screening Design — the 13-run coded table (−1/0/+1) with a note confirming main effects are orthogonal (max off-diagonal correlation 0.000).](/manual-img/98-dsd.png) - **Definitive screening design (DSD)** — enter the number of factors (3–24); the output is the coded run table (−1 = low, 0 = middle, +1 = high) plus a design-quality note (max off-diagonal main-effect correlation) and a "how to use" guide. Map the coded levels to your real factor settings and run the experiment. - **Fit definitive screening** — after collecting responses, this fits the orthogonal main effects (with proper inference, since a DSD leaves residual degrees of freedom) and then scans each factor's quadratic term for curvature — the DSD-aware analysis that avoids saturating the model. ### 20.10 CHAID decision tree Under **Multivariate & Classification**, **CHAID decision tree** (Chi-square Automatic Interaction Detection) builds a statistical, multi-way classification tree — the SPSS Decision Trees deliverable that tree *ensembles* (random forest / boosting) do not provide. ![A CHAID tree predicting churn — the tree structure with each node's split rule, size, majority class and purity, chosen by chi-square significance.](/manual-img/99-chaid.png) At each node it bins numeric predictors into quantile groups, **merges predictor categories** while their split is non-significant, then splits on the predictor with the smallest Bonferroni-adjusted chi-square p-value — growing a readable tree. Outputs: the **tree structure** (each node's rule, size, majority class and purity), a **gains-per-terminal-node table** (which segments are enriched for the target class, with an index vs. the overall rate) and a **classification table** with accuracy. Ideal for segmentation, churn/response profiling and driver analysis. Both DSD and CHAID are runnable by the AI Analyst. ### 20.11 Complex Samples (survey design) **Complex samples (survey design)** (in the Descriptive & Exploratory family) produces **design-based estimates** for data collected through a complex survey — with stratification, clustering (PSUs) and sampling weights — using Taylor-series linearization with the "ultimate cluster" variance estimator (the method behind SPSS Complex Samples and R's *survey* package). Ordinary statistics that ignore the design under-state the true standard errors; this closes that gap. **Audit sampling — Horvitz-Thompson estimates** (same family) is the companion of the **Bernoulli / Poisson sample** transform step (§2.6). Auditors and internal-control teams draw a sample of transactions in which every item is included independently with probability π (equal for all items, or proportional to the item's value), then estimate the population total, the total misstatement, the number of erroneous items and the error rate with the Horvitz-Thompson estimator Σ y/π and its exact Poisson-design variance Σ(1−π)/π² y². The analysis reports each estimate with its SE and 95% CI, a one-sided 95% upper bound on misstatement, the misstatement as a percentage of the estimated total, and the largest sampled misstatements. Supply the known population size or book value to see the estimate against it. ![Complex-samples estimate — a design-based weighted mean with linearized SE, DEFF, coefficient of variation, and the effective sample size.](/manual-img/100-survey.png) Pick the variable, the **sampling weight**, the estimand (mean / total / proportion) and optionally the **strata** and **cluster/PSU** columns. Output: the weighted estimate, the **linearized standard error**, a design-based 95% CI, the **design effect (DEFF)** (how much the design inflates variance vs. simple random sampling), the **coefficient of variation** and the **effective sample size** (n ÷ DEFF). Essential for public-health, social-science and market-research survey data. ### 20.12 Exact Tests The **Exact Tests** family gives exact or Monte-Carlo p-values for small, sparse or unbalanced data, where the usual large-sample approximations are unreliable — the SPSS Exact Tests add-on. ![Exact contingency test — Pearson χ², Cramér's V, and a Monte-Carlo exact p-value from 3,000 fixed-margin permutations.](/manual-img/101-exact.png) - **Exact contingency test (r×c)** — Fisher's exact test for a 2×2 table, or a **Monte-Carlo exact p-value** (3,000 fixed-margin permutations) for larger tables, alongside Pearson χ² and Cramér's V. - **Exact Mann-Whitney U** — the exact-conditional rank-sum test for two small, untied groups (falls back to the normal approximation otherwise). - **Exact Wilcoxon signed-rank** — the exact paired rank test for small samples. All of these procedures are also runnable by the AI Analyst — e.g. *"estimate the design-based mean of income weighting by w, stratified by region"* or *"run an exact test of contract by churn"*. ### 20.13 Meta-analysis (random effects) & meta-regression The **Meta-analysis (random effects)** procedure (Survival & Clinical family) upgrades pooling from fixed-effect to a full evidence-synthesis workflow. ![Random-effects meta-analysis — pooled effect with τ²/Q/I², a per-study forest table with weights, and publication-bias diagnostics.](/manual-img/102-meta.png) Provide the study **effect size** and its **standard error** (and optionally a study label). Output: the **DerSimonian-Laird pooled effect** with 95% CI, between-study variance **τ²**, Cochran's **Q** and **I²** heterogeneity; a **forest-plot table** (each study's effect, CI and weight, ending with the pooled diamond); a **funnel plot**; and **publication-bias diagnostics** — Egger's regression test for asymmetry and a Duval-Tweedie **trim-and-fill** estimate of missing studies with the bias-adjusted effect. A companion **Meta-regression** regresses the effects on a moderator (random-effects WLS) to explain heterogeneity. ### 20.14 GLM, survival & missing-data upgrades Four upgrades close the remaining SPSS/JMP output gaps: ![Cox diagnostics — hazard ratios, the Schoenfeld proportional-hazards test, and the baseline survival curve.](/manual-img/103-cox.png) - **Cox diagnostics (PH test + baseline)** — a Cox model with the **Schoenfeld-residual proportional-hazards test** (per covariate and global) that flags when a hazard ratio changes over time, plus the **baseline survival curve**. - **Little's MCAR test** — tests whether missing data are Missing Completely At Random, using EM estimates of the mean/covariance; a small p rejects MCAR and points you toward MAR-appropriate imputation. - **RM-ANOVA (sphericity + power)** — repeated-measures ANOVA with **Mauchly's test of sphericity**, the **Greenhouse-Geisser** corrected p, generalized η² and **observed power** — the corrections SPSS reports and a plain RM-ANOVA omits. - **One-way ANOVA power** — observed (post-hoc) power at the sample effect size and the total N needed for 80% power. These are all AI-Analyst runnable — e.g. *"run a random-effects meta-analysis of effect_logOR with se, labelled by study"* or *"check the proportional-hazards assumption for the Cox model of failure_hours on temp_c"*. ### 20.15 Specialised methods: forecasting, functional data, reliability growth & marketing This batch closes several JMP/SPSS specialty engines. ![Expert modeler — the auto-selected forecasting model, an AIC leaderboard across ARIMA and exponential-smoothing families, and the forecast with confidence bands.](/manual-img/105-expert.png) - **Expert modeler (auto-forecast)** *(Forecasting)* — one-click automatic forecasting: it fits several ARIMA orders and exponential-smoothing variants and picks the best by AIC, reporting the chosen model, an AIC leaderboard, RMSE/MAPE fit statistics and a forecast with 95% bands. Mirrors the SPSS Expert Modeler. - **Functional PCA (curve data)** *(Dimension reduction)* — smooths a set of curves (each row a curve over ordered measurement points) and extracts **functional principal components** — the dominant modes of variation across curves — with a variance-explained table and mean/FPC1 curves. A first slice of JMP's Functional Data Explorer. - **Reliability growth (Crow-AMSAA)** and **Recurrent events (MCF)** *(Reliability)* — the repairable-system side of reliability: Crow-AMSAA fits the NHPP power-law growth model (shape β, instantaneous MTBF, Duane plot; β<1 = improving), and the MCF gives the mean cumulative number of events per unit over time for warranty / recurrent-failure analysis. ![RFM analysis — customers scored into value tiers (Champions → Hibernating) with size, share, average spend and frequency.](/manual-img/104-rfm.png) - **RFM analysis (direct marketing)** *(Consumer & Market Research)* — scores customers on Recency, Frequency and Monetary value (quintiles, 5 = best) and rolls them into value tiers (Champions, Loyal, Potential, At risk, Hibernating) with per-tier size, share and average spend. Mirrors the SPSS Direct Marketing RFM module. - **Conjoint design generator** *(Consumer & Market Research)* — builds a **D-efficient orthogonal profile plan** from your attribute levels (e.g. `3,3,2,4`) to present to respondents, complementing the existing conjoint *analysis*. All six are AI-Analyst runnable — e.g. *"auto-forecast the close series 14 steps ahead"* or *"run an RFM analysis using recency_days, frequency and monetary"*. ### 20.16 Computer experiments, uplift & multivariate SPC The final batch closes JMP's computer-experiment stack and two more advanced methods. ![Model-driven multivariate SPC — PCA-based Hotelling T² and SPE limits with out-of-control counts, plus the T² and SPE control charts.](/manual-img/106-mspc.png) - **Multivariate SPC (PCA T²/SPE)** *(Quality & SPC)* — model-driven multivariate control charting: it builds a PCA model of the process variables and plots **Hotelling T²** (variation inside the model) and **SPE/Q** (residual variation outside it) with 99% limits, flagging multivariate out-of-control points that univariate charts miss. - **Space-filling design (Latin Hypercube)** and **Mixture design (simplex-lattice)** *(Design of experiments)* — the computer-experiment and formulation designs: a Latin Hypercube spreads runs evenly through a continuous factor space (with a discrepancy measure), and a simplex-lattice places runs where component proportions sum to 1. - **Gaussian-process regression (kriging)** *(Regression)* — a GP emulator (RBF kernel + noise) for smooth or expensive-to-evaluate functions, reporting fit R², log-marginal-likelihood and per-predictor length-scales — the surrogate model behind computer-experiment analysis. - **Uplift modeling (treatment effect)** *(Causal Inference)* — two-model (T-learner) uplift that estimates each unit's **incremental** response to a treatment and ranks segments by predicted uplift (deciles), so you target those the treatment actually moves. All are AI-Analyst runnable. ### 20.17 Presentation tables, model deployment & the control-chart builder These close the remaining presentation and deployment gaps. ![Custom table — a churn × contract banner with counts, column percentages and APA column-proportion significance letters.](/manual-img/108-ctables.png) - **Custom tables (banner + sig.)** *(Descriptive)* — the SPSS Custom Tables (CTABLES) deliverable: a presentation crosstab of a row variable against a banner (column) variable showing counts and **column percentages**, with **APA-style column-proportion significance letters** (a superscript letter marks a column whose proportion is significantly greater than the lettered column) and the overall χ². Optionally show **cell means** of a measure instead of counts. ![Score-code export — the fitted model's coefficients plus ready-to-paste Python, SQL and JavaScript scoring formulas.](/manual-img/109-scorecode.png) - **Score-code export (formula depot)** *(Regression)* — JMP's Formula Depot idea: fit a linear or logistic model and export a self-contained **scoring formula as Python, SQL and JavaScript** to deploy the model in any pipeline, alongside the coefficient table. - **Control chart builder (auto/phased)** *(Quality & SPC)* — one entry point that **auto-selects** the right control chart (I-MR / X-bar-R / p / c) from the data, with an optional **baseline phase** so limits are computed from a stable period and applied forward — the flexible builder behaviour, complementing the individual SPC charts. > **Coverage note.** With these batches the platform closes the statistical- and output-level gaps identified against **JMP 18/Pro** and **IBM SPSS Statistics 31**. Exploratory graphing is already provided by the interactive **Chart Builder** (25 chart types with dimensions/measures/filters and drill-through); the only residual vs. JMP's Graph Builder is live cross-chart *brushing*, tracked on the roadmap. ### 20.18 Output parity with JMP & SPSS (verification) Because these procedures are built on the same canonical estimators the reference tools use (lifelines Weibull MLE, `scipy` exact tests, `statsmodels` OLS/logistic/ANOVA/ARIMA, `pingouin` JZS Bayes factors, DerSimonian-Laird meta-analysis), outputs match to numerical precision. Verified against published/hand-computed benchmarks: | Analysis | Benchmark | Reference value | InstaBizIntel | |---|---|---|---| | Weibull life fit | Lieblein-Zelen ball-bearing data | β ≈ 2.10, η ≈ 81.9 (JMP Life Distribution) | **β = 2.102, η = 81.88** | | Fisher exact (2×2) | Lady-tasting-tea [[3,1],[1,3]] | p = 0.4857 (R / SPSS Exact) | **p = 0.4857** | | One-way ANOVA | A[1,2,3] B[3,4,5] C[5,6,7] | F = 12.0, p = 0.008 (hand) | **F = 12.0, p = 0.008** | | Random-effects meta | 3 studies, hand-computed DL | pooled 0.50, τ² 0.08, Q 18, I² 88.9% | **0.50, 0.08, 18.0, 88.9%** | | Logistic coefficients | simulated logit 0.5 + 1.2·x₁ | 0.5 / 1.2 / 0 | **0.57 / 1.27 / −0.03** | | Bayesian correlation | JZS engine | pingouin = JASP engine | **same engine** | --- ### 20.19 High-value method gap-closers (GMM, fsQCA, NMA, pharmacometrics) Following a review of the methods used in recent Q1/Q2 journal papers (Management/Finance/Commerce/HR/Marketing/Pharmacy), five frequently-used methods were added to close the largest coverage gaps. All are validated against known inputs and runnable by the AI Analyst. **Dynamic-panel GMM (Arellano-Bond)** *(Panel data / Time Series & Forecasting family)* — the standard corporate-finance and commerce estimator for dynamic panels where the outcome depends on its own lag (which biases classical fixed-effects). One-step difference GMM with GMM-style lagged-level instruments, robust standard errors, the **Sargan over-identification test** and the **AR(2) serial-correlation test**. ![Dynamic-panel GMM — coefficient table (lagged dependent + covariates with robust SE) and specification diagnostics (Sargan, AR(2)).](/manual-img/120-gmm.png) **fuzzy-set QCA (fsQCA)** *(Multivariate family)* — configurational analysis (fast-rising in management/marketing) that finds which *combinations* of conditions are sufficient for an outcome. It auto-calibrates variables to fuzzy membership, runs an **analysis of necessity** (consistency/coverage per condition), and a **sufficiency truth table** with consistency and coverage, then reports the solution paths. ![fsQCA — necessity analysis and the sufficiency truth table with consistency and coverage per configuration.](/manual-img/121-fsqca.png) **Network meta-analysis** *(Survival & Clinical family)* — contrast-based frequentist NMA that pools direct and indirect evidence across ≥3 treatments. Output: network summary (τ², Q, I²), basic estimates vs. a reference, a full **league table** of pooled relative effects, and a **P-score treatment ranking**. ![Network meta-analysis — summary, basic estimates vs. reference, and the league table of pooled effects.](/manual-img/122-nma.png) **PK non-compartmental analysis** *(Survival & Clinical family)* — pharmacokinetics from a concentration-time profile: **Cmax, Tmax, AUC(0–t) & AUC(0–∞)** (trapezoidal), terminal rate constant and **half-life**, MRT, and **clearance/volume** (given the dose). ![PK non-compartmental analysis — Cmax/Tmax/AUC, terminal half-life, clearance and volume with the concentration-time curve.](/manual-img/123-pknca.png) **Dissolution / release kinetics** *(Survival & Clinical family)* — fits the standard drug-release models (**zero-order, first-order, Higuchi, Korsmeyer-Peppas, Hixson-Crowell**), reports each model's R², identifies the best fit and the Korsmeyer-Peppas release exponent *n* (mechanism), and computes the **f2 similarity factor** against a reference profile. ![Dissolution kinetics — model-fit table with R² for each release model, the best fit, and the release-exponent interpretation.](/manual-img/124-dissolution.png) ### 20.20 Advanced econometrics, efficiency, text & real-world-evidence methods Eight further methods were added to close the remaining Q1/Q2 gaps across finance/econometrics, operations, marketing text analytics and pharmacoepidemiology. All validated against known inputs and AI-Analyst runnable. **Heckman selection model** *(Regression)* — corrects for **sample-selection bias** when the outcome is only observed for a self-selected subset (e.g. wages only for workers). A two-step estimator: a probit selection equation yields the inverse Mills ratio, which is added to the outcome regression; a significant λ signals selection bias. ![Heckman selection model — outcome coefficients corrected for selection, with the inverse-Mills-ratio (λ) selection-bias test.](/manual-img/125-heckman.png) **Stochastic frontier analysis (SFA)** and **Data Envelopment Analysis (DEA)** *(Regression / Multivariate)* — efficiency measurement. SFA fits a normal-halfnormal production/cost frontier separating noise from inefficiency (σ_u, σ_v, γ, technical efficiency); DEA computes non-parametric CCR input-oriented efficiency scores for each decision-making unit. ![Stochastic frontier analysis — frontier coefficients, variance decomposition (σ_u/σ_v/γ) and mean technical efficiency.](/manual-img/126-sfa.png) ![Data Envelopment Analysis — per-DMU efficiency scores and the count of efficient units.](/manual-img/127-dea.png) **Topic modelling (LDA / NMF)** *(Multivariate)* — discovers latent themes in a text column (marketing/management text analytics), reporting each topic's top terms and how many documents it dominates. ![Topic modelling — latent topics with their most characteristic terms and document shares.](/manual-img/128-topic.png) **Value at Risk & Expected Shortfall** and **Connectedness (Diebold-Yilmaz)** *(Time series)* — market-risk analytics. VaR reports Historical, Gaussian and Cornish-Fisher (skew/kurtosis-adjusted) VaR plus CVaR at a chosen confidence; connectedness measures return/volatility **spillovers** across series via a VAR generalized FEVD (FROM/TO/NET directional and a total connectedness index). ![Value at Risk — Historical, Gaussian and Cornish-Fisher VaR and Expected Shortfall.](/manual-img/129-var.png) ![Diebold-Yilmaz connectedness — FROM/TO/NET spillover table, total connectedness index, and net directional chart.](/manual-img/130-connectedness.png) **Bayesian SEM / mediation** *(Bayesian)* — a path model reporting the a, b and c′ paths and a **Bayesian posterior for the indirect effect (a·b)** with a 95% credible interval and P(effect > 0) — the Bayesian-mediation output common in OB/psychology. ![Bayesian SEM / mediation — path estimates and the indirect-effect posterior with a 95% credible interval.](/manual-img/131-bayes-sem.png) **Target-trial emulation (RWE / IPTW)** *(Causal Inference)* — estimates a treatment effect from observational real-world data via **inverse-probability-of-treatment weighting**, reporting the ATE, risk ratio, effective sample size, and a **covariate-balance table** (standardised mean differences before vs. after weighting). ![Target-trial emulation — IPTW treatment effect and the covariate-balance table showing confounders balanced after weighting.](/manual-img/132-target-trial.png) ### 20.21 Research-completeness methods (frontier econometrics, text, Bayesian & scaling) A final set of eight methods brings the platform to research completeness across the frontier techniques appearing in top-tier journals. **Staggered DiD (Callaway-Sant'Anna)** *(Causal Inference)* — the modern, **heterogeneity-robust** difference-in-differences for staggered treatment adoption. It estimates group-time ATTs against never-treated (or not-yet-treated) controls, aggregates to an overall effect (bootstrap SE), and reports an **event study** of dynamic effects — avoiding the negative-weighting bias of two-way fixed-effects DiD. ![Staggered DiD (Callaway-Sant'Anna) — overall ATT, the event-study table and the dynamic-effects chart.](/manual-img/133-did-staggered.png) **Fama-MacBeth regression** *(Panel data)* — the two-pass asset-pricing estimator: period-by-period cross-sectional regressions averaged over time, giving each factor's priced premium and a Fama-MacBeth t-statistic. ![Fama-MacBeth — average factor premia with Fama-MacBeth standard errors and t-statistics.](/manual-img/134-fama-macbeth.png) **Structural VAR (SVAR)** *(Time series)* — a Cholesky-identified structural VAR with **orthogonalised impulse responses** and a forecast-error variance decomposition — the workhorse of structural macro-econometrics. ![Structural VAR — orthogonalised impulse responses and variance decomposition.](/manual-img/135-svar.png) **Wavelet coherence** *(Time series)* — time-frequency **co-movement** of two series across short/medium/long horizons via the continuous (Morlet) wavelet transform — widely used in energy-finance and macro co-movement studies. ![Wavelet coherence — mean coherence by frequency band with the coherence chart.](/manual-img/136-wavelet.png) **Topic model (BERTopic-style)** *(Multivariate)* — an embedding-based topic model: documents are embedded (TF-IDF → UMAP), clustered by density (**HDBSCAN**, which auto-selects the number of topics and isolates outliers), and each topic is described by class-based TF-IDF (c-TF-IDF) terms — complementing the LDA/NMF topic modeller. ![BERTopic-style topic model — density-clustered topics with c-TF-IDF top terms.](/manual-img/137-bertopic.png) **Bayesian SEM (MCMC)** *(Bayesian)* — a **full Gibbs-MCMC** path/mediation model reporting posterior means, SDs and 95% **credible intervals** for every path and the indirect effect (a·b) — the fully Bayesian complement to the analytic Bayesian mediation. ![Bayesian SEM (MCMC) — posterior means, SDs and 95% credible intervals for all paths and the indirect effect.](/manual-img/138-bayes-mcmc.png) **OVERALS (multiple-set canonical)** and **Preference scaling / unfolding (PREFSCAL)** *(Dimension reduction)* — the SPSS Categories optimal-scaling family: OVERALS finds a common latent dimension across two-to-three sets of variables (generalized canonical correlation), while PREFSCAL places objects in a common perceptual map from preference/rating data. ![OVERALS — fit per dimension and component loadings across the variable sets.](/manual-img/139-overals.png) ![PREFSCAL — a 2-D perceptual map of objects from preference data.](/manual-img/140-prefscal.png) > **Interactive graphing note.** JMP's Graph-Builder-style linked **brushing** is already provided on Dashboards (see §3.4) — click to select and toggle Filter vs. Highlight to see a selection in context across every linked chart. With these additions the platform is research-complete across the methods used in recent Q1/Q2 journals. --- ## 21. Research applications by discipline Statistics underpins research across every field. The table below maps common research needs, by discipline, to the InstaBizIntel tools that address them — a guide for students and researchers choosing what to run. (Section numbers/sources in brackets.) | Discipline | Typical research questions & analyses | Where in InstaBizIntel | |---|---|---| | **Management & organisational research** | Construct validity and theory testing (CFA, **PLS-SEM/CB-SEM**), mediation/moderation, multi-group comparison, common-method-bias checks, reliability (α, ω); PLS-SEM reviewer battery & lavaan/Mplus export (§28.4); validated model templates TAM/UTAUT/TPB (§31.1) | PLS-SEM (§8); CFA, mediation, moderation (§20); Statistics reliability | | **Marketing & consumer research** | Segmentation (cluster/LCA), conjoint-style regression, choice models (logit/probit), A/B testing, uplift & causal ML, driver analysis (key influencers); causal DAG (§29.1) | ML Workflows (§9); LCA (§20); regression variants (§7); DiD/PSM (§20) | | **Finance & investment** | Return/volatility modelling (**GARCH/EGARCH/GJR**, HAR), factor & event studies, portfolio risk, cointegration/pairs (VECM/Johansen), **price forecasting with statistical model comparison (MCS/DM)** | Time-series econometrics (§6); Forecasting Lab (§17); market data (§16.3) | | **Investment banking & valuation** | Company fundamentals from filings, comparable analysis, scenario forecasting, credit-risk scoring, stress/robustness (multiverse) | SEC EDGAR (§16.5); Forecasting Lab (§17); logistic regression & ML (§7, §9); spec-curve (§20) | | **Economics & econometrics** | Panel models (FE/RE, dynamic GMM), IV/2SLS for endogeneity, causal inference (**DiD, RDD, synthetic control**), local projections, macro forecasting; causal DAG & refutation (§29.1); multiverse (§27.8) | Panel data (§5); causal inference (§20); World Bank/FRED (§16.4–16.5) | | **Commerce, accounting & business analytics** | KPI dashboards, demand/sales forecasting, ratio & trend analysis, fraud/anomaly detection, survey analysis | Business intelligence (§3); Forecasting Lab (§17); ML anomaly (§9); Statistics (§4) | | **Pharmaceutical, clinical & biostatistics** | Trial analysis (RR/OR/NNT, CMH, ROC/AUC), survival (Kaplan–Meier, Cox), dose–response/nonlinear regression, GEE/GLMM for repeated measures, **meta-analysis** and **RoB2/ROBINS-I appraisal**, ClinicalTrials.gov evidence; dual-LLM extraction with adjudication (§29.3) | Statistics clinical-trials & survival groups (§4); Meta-analysis & appraisal (§18); ClinicalTrials.gov (§16.5) | | **Public health & epidemiology** | Incidence/prevalence, risk models, WHO indicators, systematic reviews with **PRISMA + MMAT/GRADE** | WHO GHO (§16.5); Literature Review suite (§18); Statistics (§4) | | **Psychology, education & social science** | Scale validation (**CFA, measurement invariance, IRT**), latent class/profile & growth, multilevel/mixed models, mediation/moderation, **Bayes factors**, power analysis; careless-response screen & PLS sample-size calculator (§31.2–31.3) | CFA/LCA/LPA/Bayes (§20); PLS-SEM (§8); Rigor Guard power (§19); Statistics (§4) | | **Operations, engineering & quality** | SPC control charts (I-MR/Xbar-R/p/c/EWMA/CUSUM), process capability (Cp/Cpk), **Design of Experiments** (factorial, CCD, Box–Behnken, RSM), Gage R&R, reliability | Statistics Quality & SPC and DOE groups (§4) | | **Energy, environment & climate** | Load/price forecasting with decomposition hybrids, spatial econometrics, OWID climate indicators, trend/seasonality decomposition | Forecasting Lab (§17); OWID (§16.5); spatial/spec methods (§20) | | **Political science, communication & media** | Event/text analysis (GDELT), sentiment & topic modelling, network/co-occurrence analysis, survey experiments | GDELT (§16.5); Qualitative (§10); network analysis (§20) | | **Information systems & data science** | Predictive modelling with **35+ ML models + AutoML**, explainability (SHAP), conformal uncertainty, model comparison | ML Workflows (§9); conformal (§20) | | **Any field doing a literature/systematic review** | End-to-end review: search 8 sources → PRISMA → cluster/thematic map → appraisal tables → meta-analysis → **citation-safe drafted manuscript** in any citation style; living reviews (§29.2) | Literature Review + Meta suite (§18) | | **Every field — publishing** | Audit trail, Trust Score, reproducibility bundle, statcheck/citation/overclaim checks, publication gate, APA Word export, Methods from the execution log | §27, §30 | Whatever the field, the workflow is the same: **bring data in (§16) → analyse (§4–§20) → check rigour and reproducibility (§19, §27–§30) → report (§13, §17.5, §18.2, §30.2)** — all in one browser, on your own server. --- ## 22. Complete technique reference (304 analyses) Every analysis available under **Research → Statistics**, grouped by family. Each is selectable in the Statistics workbench and runnable by the AI Analyst by name. "Inputs" lists the parameters each expects. > 📘 **Downloadable guide:** the **[Complete Analysis Guide (PDF)](/InstaBizIntel_301_Analyses_Guide.pdf)** documents **all 301 analyses** end-to-end, organised by sector. For every method it gives: a short *"what this does"* explainer, a **real, meaningful sample dataset** (telecom churn, a technology-acceptance survey, Bitcoin prices, asset returns, an accelerated component life-test, clinical/pharma trials, firm panels, and more), the exact **steps** to run it, the **live output** (tables + chart), and a **detailed interpretation** of the result. Coverage includes the newest additions — staggered difference-in-differences (Callaway–Sant'Anna), Fama–MacBeth, structural VAR, wavelet coherence, BERTopic-style topic clustering, MCMC Bayesian SEM, OVERALS/PREFSCAL, dynamic-panel GMM, fsQCA, network meta-analysis, PK/dissolution, Heckman selection, stochastic frontier/DEA, VaR/connectedness, and target-trial emulation. ### 22.1 Descriptive & Exploratory (38) | Technique | What it does | Inputs | |---|---|---| | **Anderson-Darling normality** | Powerful EDF normality test | column | | **Bimodality coefficient** | Detect multimodal distributions | column | | **Bootstrap CI** | Distribution-free bootstrap confidence interval for a mean/median | column, statistic | | **Chi-square goodness-of-fit** | Category counts vs uniform | column | | **Complex samples (survey design)** | Design-based mean/total/proportion with strata, clusters & weights (DEFF, CV, CI) | variable, weight, estimand, strata, cluster | | **Audit sampling — Horvitz-Thompson estimates** | Design-unbiased totals, misstatement and error rate from a Bernoulli / Poisson (PPS) sample with CIs | inclusion_prob, amount, error, error_flag, population_n, population_total | | **Cramér-von Mises normality** | EDF normality test | column | | **Crosstab with row %** | Contingency table with percentages | row, column | | **Crosstabs & chi-square** | Contingency table with chi-square test and Cramér's V | row, column | | **Custom tables (banner + sig.)** | Presentation crosstab with column % and column-proportion significance letters | row, column, measure | | **D'Agostino K² normality** | Omnibus skew+kurtosis normality test | column | | **Descriptives** | Mean, SD, CI, quartiles, skewness, kurtosis per variable | columns | | **Descriptives (all variables)** | Summary of every numeric column | | | **Descriptives by group** | Group-wise summary statistics | column, group | | **Distribution fitting** | Fit & compare Normal/Log-normal/Exp/Gamma/Weibull by AIC | column | | **Diversity indices** | Shannon/Simpson diversity | column | | **Exact binomial test** | Test a success proportion | column, prob | | **Frequencies** | Counts, percentages and cumulative % of a categorical variable | column | | **Frequency distribution (histogram)** | Binned counts + histogram | column, bins | | **Frequency table** | Counts of a categorical variable | column | | **Geometric & harmonic means** | Geo/harmonic mean, CV, MAD | column | | **Gini coefficient** | Inequality/concentration index | column | | **Grubbs' outlier test** | Detect the single most extreme outlier | column | | **IQR outlier screen** | Tukey 1.5×IQR outlier detection + boxplot | column | | **Jarque-Bera normality** | Normality via skewness & kurtosis | column | | **Little's MCAR test** | Test whether missing data are Missing Completely At Random (EM) | columns | | **Missing-data summary** | Missingness per column | | | **Normal tolerance interval** | Population coverage interval | column, coverage, confidence | | **Normality test battery** | Shapiro+JB+D'Agostino+KS combined | column | | **Normality tests** | Shapiro-Wilk and Kolmogorov-Smirnov with histograms | columns | | **One-proportion test** | Test a proportion against an expected value (z + exact binomial) | column, expected | | **Percentile summary** | Deciles & tail percentiles | column | | **Proportion confidence interval** | Wilson & Clopper-Pearson CIs | column | | **Shapiro-Wilk normality** | Most powerful normality test | column | | **Skewness & kurtosis tests** | Test shape departures from normal | column | | **Stem-and-leaf plot** | Exploratory stem-and-leaf display of one numeric variable | column | | **Trimmed & winsorized means** | Robust central tendency | column | | **Z-score outlier screen** | Flag values by /z/ | column, threshold | ### 22.2 Compare Groups (52) | Technique | What it does | Inputs | |---|---|---| | **Bartlett's test** | Equal-variance test | column, group | | **Bayesian t-test (Bayes factor)** | JZS Bayes factor for a mean difference (evidence for H1 vs H0) | group1, group2, paired | | **F-test for equal variances** | Ratio-of-variances test | column, group | | **Games-Howell post-hoc** | Post-hoc for unequal variances | column, group | | **Homogeneity of variance** | Levene, Bartlett and Fligner-Killeen equal-variance tests | column, group | | **Independent-samples t-test** | Compare two group means (Student + Welch, Levene's test, Cohen's d) | column, group | | **One-sample t-test** | Compare a mean against a fixed test value | column, test_value | | **One-sample z-test** | Large-sample mean test | column, value | | **One-way ANOVA** | Compare 3+ group means with Levene, eta² and Tukey HSD post-hoc | column, group | | **Paired-samples t-test** | Compare two related measurements | column1, column2 | | **Planned contrasts** | A priori linear contrast of means | column, group, weights | | **Tukey HSD post-hoc** | Family-wise pairwise comparisons | column, group | | **Two-proportion z-test** | Compare success rates across groups | column, group | | **Two-way ANOVA** | Factorial ANOVA with interaction (Type II SS) | column, factor1, factor2 | | **Welch's ANOVA** | One-way ANOVA for unequal variances | column, group | | **Yuen's trimmed t-test** | Robust two-group mean comparison | column, group, trim | | **Ansari-Bradley scale test** | Rank test for equal spread | column, group | | **Barnard's exact test** | Unconditional exact 2×2 test | row, column | | **Brunner-Munzel test** | Robust 2-sample rank test | column, group | | **Cochran's Q** | Repeated-measures test for 3+ related binary variables | columns | | **Dunn's post-hoc** | Post-hoc after Kruskal-Wallis | column, group | | **Epps-Singleton test** | Equal-distribution test | column, group | | **Exact Mann-Whitney U** | Exact-conditional rank-sum test for two small groups | column, group | | **Exact Wilcoxon signed-rank** | Exact paired rank test for small samples | column1, column2 | | **Exact contingency test (r×c)** | Fisher (2×2) or Monte-Carlo exact p-value for small/sparse tables | row, column | | **Fisher's exact test** | Exact 2×2 association | row, column | | **Fligner-Killeen test** | Robust equal-variance test | column, group | | **Friedman test** | Nonparametric repeated-measures comparison of 3+ variables | columns | | **Jonckheere-Terpstra trend** | Ordered-alternatives trend test | column, group | | **Kruskal-Wallis + effect size** | Rank ANOVA with ε²/η² | column, group | | **Kruskal-Wallis H** | Nonparametric alternative to one-way ANOVA | column, group | | **Loglinear analysis** | Poisson model of a multiway contingency table; tests independence | factors | | **Mann-Whitney U** | Nonparametric alternative to the independent t-test | column, group | | **McNemar's test** | Paired binary test (before/after on the same subjects) | column1, column2 | | **Mood's median test** | Nonparametric equality-of-medians across groups | column, group | | **Mood's scale test** | Equal-dispersion test (2 groups) | column, group | | **One-sample Wilcoxon** | Signed-rank test of the median | column, median | | **Quade test** | Repeated-measures (Friedman alternative) | columns | | **Runs test (randomness)** | Wald-Wolfowitz test for randomness of a sequence | column | | **Sign test** | Distribution-free test of the median | column, median | | **Two-sample KS test** | Compare two distributions | column, group | | **Wilcoxon signed-rank** | Nonparametric alternative to the paired t-test | column1, column2 | | **k-sample Anderson-Darling** | Same-distribution test across groups | column, group | | **ANOVA effect sizes (η²/ω²)** | Eta², omega², epsilon² | column, group | | **Association (Cramér's V)** | Chi-square, Cramér's V, contingency C, phi | row, column | | **Cliff's delta** | Nonparametric effect size (2 groups) | column, group | | **Cohen's f² (regression)** | Effect size for a regression model | dependent, predictors | | **Correlation ratio (η)** | Categorical → numeric association | column, group | | **Effect size (Cohen's d)** | d, Hedges' g, Glass' Δ and CLES for two groups | column, group | | **Paired effect size (dz)** | Standardized paired difference | column1, column2 | | **Point-biserial correlation** | Correlation between continuous and binary variable | column, binary | | **Rank-biserial (Mann-Whitney)** | Nonparametric effect size | column, group | ### 22.3 Correlation & Association (13) | Technique | What it does | Inputs | |---|---|---| | **Bayesian correlation (Bayes factor)** | Bayes factor for a Pearson correlation | x, y | | **Biweight midcorrelation** | Robust (outlier-resistant) correlation | column1, column2 | | **Correlation matrix** | Pearson, Spearman or Kendall correlations with significance | columns, method | | **Correlation matrix + heatmap** | Correlation matrix with heatmap | columns, method | | **Correlation with CI** | Pearson r + Fisher-z 95% CI | column1, column2 | | **Covariance matrix** | Variance-covariance table | columns | | **Distance correlation** | Detects any (nonlinear) dependence | column1, column2 | | **Kendall's τ-b correlation** | Rank correlation robust to ties | column1, column2 | | **Multiple correlation (R)** | Multiple R of outcome on predictors | dependent, predictors | | **Mutual information** | Nonlinear categorical dependence | column1, column2 | | **Ordinal association (γ, τ-c)** | Goodman-Kruskal γ, Kendall τ-c | row, column | | **Partial correlation** | Correlation between two variables controlling for others | x, y, controls | | **Spearman ρ with CI** | Rank correlation + confidence interval | column1, column2 | ### 22.4 Regression & Modeling (46) | Technique | What it does | Inputs | |---|---|---| | **Breusch-Pagan test** | Heteroskedasticity diagnostic | dependent, predictors | | **Categorical regression (CATREG)** | Optimal-scaling regression with nominal/ordinal predictors (ALS) | dependent, predictors | | **Conformal prediction intervals** | Distribution-free prediction intervals with coverage guarantee | target, predictors, alpha | | **Cook's distance (influence)** | Influential-observation diagnostics | dependent, predictors | | **Curve estimation** | Fit & rank linear/quadratic/cubic/log/inverse/power/exponential/S models | dependent, independent | | **Durbin-Watson test** | Residual autocorrelation | dependent, predictors | | **Gaussian-process regression (kriging)** | GP emulator for smooth/expensive functions with uncertainty | dependent, predictors | | **Generalized estimating equations (GEE)** | Population-averaged model for clustered data with robust SEs | dependent, predictors, group, family | | **Generalized linear mixed model (GLMM)** | Random-intercept logistic/Poisson mixed model | dependent, predictors, group | | **Generalized linear model (GLM)** | GLM with selectable family/link | dependent, predictors, family | | **IV / two-stage least squares** | Instrumental-variables regression for endogenous regressors | dependent, endogenous, instruments, exogenous | | **Linear regression** | OLS with standardized betas, CIs, VIF and Durbin-Watson | dependent, predictors | | **Logistic diagnostics (H-L)** | Hosmer-Lemeshow calibration | dependent, predictors | | **Logistic regression** | Binary logit with odds ratios, pseudo-R² and classification table | dependent, predictors | | **Multinomial logistic** | Categorical outcome with 3+ levels; relative risk ratios | dependent, predictors | | **Multiple imputation (MICE)** | Impute missing data m times and pool with Rubin's rules | columns, m | | **Negative binomial regression** | Overdispersed count-data regression | dependent, predictors | | **Nonlinear regression (NLS)** | Fit logistic/Gompertz/Michaelis-Menten/exponential-plateau growth models | dependent, independent, model | | **Ordinal regression** | Proportional-odds model for ordered outcomes (e.g. Likert) | dependent, predictors | | **Partial least squares regression** | For many/collinear predictors | dependent, predictors, components | | **Poisson regression** | Count-data regression with incidence-rate ratios | dependent, predictors | | **Principal-component regression** | Regress on orthogonal PCs | dependent, predictors, components | | **Probit regression** | Binary model with probit link | dependent, predictors | | **Quantile regression** | Model a conditional quantile (robust to outliers) | dependent, predictors, quantile | | **Ramsey RESET test** | Functional-form/mis-specification test | dependent, predictors | | **Ridge / Lasso regression** | Penalized regression (CV-tuned) | dependent, predictors, penalty | | **Robust regression (Huber)** | Outlier-resistant regression | dependent, predictors | | **Score-code export (formula depot)** | Export a fitted linear/logistic model as Python, SQL & JavaScript scoring code | dependent, predictors, model_type | | **Spatial regression (lag / error)** | Spatial-lag or spatial-error model with k-nearest-neighbour weights | dependent, predictors, x_coord, y_coord, model, k | | **Specification-curve (multiverse)** | Robustness: the focal effect across every covariate specification | outcome, focal, covariates | | **Spline regression** | Flexible nonlinear B-spline fit | dependent, predictor, df | | **Stepwise regression (AIC)** | Forward selection by AIC | dependent, predictors | | **Theil-Sen robust slope** | Median-slope robust regression | column1, column2 | | **Variance inflation factors** | Collinearity diagnostics | predictors | | **Weighted least squares** | Regression with observation weights | dependent, predictors, weight | | **White's test** | General heteroskedasticity test | dependent, predictors | | **Zero-inflated Poisson** | Counts with excess zeros | dependent, predictors | | **ANCOVA** | ANOVA with covariates and adjusted (marginal) means | column, factor, covariates | | **Box's M test** | Equality of covariance matrices | columns, group | | **Hotelling's T²** | Multivariate two-group mean comparison | columns, group | | **Linear mixed model** | Multilevel regression with a random intercept (ICC reported) | dependent, predictors, group | | **MANOVA** | Multivariate ANOVA — several DVs by one factor (Wilks, Pillai…) | columns, group | | **RM-ANOVA (sphericity + power)** | Repeated-measures ANOVA with Mauchly, Greenhouse-Geisser & observed power | columns | | **Repeated-measures ANOVA** | Within-subject comparison of 2+ repeated measurements | columns | ### 22.5 Multivariate & Classification (39) | Technique | What it does | Inputs | |---|---|---| | **Canonical correlation** | Correlate two variable sets | set1, set2 | | **Confirmatory factor analysis (CFA)** | Test a measurement model with fit indices (CFI/TLI/RMSEA), loadings, AVE & CR | model | | **Correspondence analysis** | Map categorical associations | row, column | | **Embedding (UMAP / t-SNE / PCA)** | 2-D map of high-dimensional data for visualization & cluster discovery | features, method, label | | **Factor analysis / PCA** | Principal components with varimax rotation, KMO, Bartlett, scree | columns, n_components, rotation | | **Functional PCA (curve data)** | Smooth curves and extract functional principal components (FDA) | columns | | **ML factor analysis** | Latent common-factor model | columns, n_factors | | **Multidimensional scaling** | 2D distance-preserving map | columns | | **Parallel analysis** | How many factors to retain | columns | | **CHAID decision tree** | Chi-square automatic interaction detection tree with gains & classification | target, predictors, max_depth, min_node | | **DBSCAN clustering** | Density-based clustering with automatic noise detection | columns, eps, min_samples | | **Discriminant analysis (LDA)** | Linear discriminant functions with cross-validated classification | columns, group | | **Gaussian mixture clustering** | Probabilistic (soft) clustering | columns | | **Hierarchical clustering** | Agglomerative clustering with agglomeration schedule | columns, k, method | | **K-means clustering** | Cluster observations with center profiles and separation ANOVA | columns, k | | **Latent class analysis (LCA)** | Person-centred typology from categorical/binary indicators (BIC, entropy) | indicators, classes | | **Latent profile analysis (LPA)** | Person-centred profiles from continuous indicators (BIC, entropy) | indicators, classes | | **Mahalanobis outliers** | Multivariate outlier detection | columns | | **Market-basket (association rules)** | Apriori association rules (support/confidence/lift) from transaction data | transaction, item, min_support, min_confidence | | **Quadratic discriminant analysis** | Curved-boundary classifier | columns, group | | **Silhouette (optimal k)** | Choose the number of clusters | columns | | **Bland-Altman agreement** | Agreement between two methods + plot | method1, method2 | | **Cohen's kappa (agreement)** | Inter-rater agreement for two raters | rater1, rater2 | | **Fleiss' kappa** | Agreement among 3+ raters | columns | | **Intraclass correlation (ICC)** | ICC(2,1) and ICC(2,k) reliability across raters/measures | columns | | **Item Response Theory (2PL)** | Item discrimination & difficulty with characteristic curves (psychometrics) | items | | **Kendall's W (concordance)** | Agreement among rankers | columns | | **Lin's concordance (CCC)** | Agreement between two continuous measures | column1, column2 | | **Reliability (Cronbach's α)** | Scale reliability with item-total statistics and α-if-deleted | columns | | **Categorical PCA (CATPCA)** | Nonlinear PCA on optimally-scaled categorical variables | columns | | **Key influencers** | Rank which factors most increase or decrease a target (Power BI-style) | target, factors | | **Network analysis (SNA)** | Centrality + community detection on an edge list (co-authorship, SNA, etc.) | source, target, weight, directed | | **TwoStep cluster** | Auto-selects cluster count by BIC, then profiles the clusters | columns, max_k | ### 22.6 Causal Inference (11) | Technique | What it does | Inputs | |---|---|---| | **Difference-in-Differences** | Causal effect from treated×post interaction (parallel-trends design) | outcome, treated, post, covariates, cluster | | **Double-LASSO (causal effect)** | Post-double-selection LASSO — unbiased treatment effect with high-dimensional controls | outcome, treatment, controls | | **Event study (dynamic DiD)** | Leads/lags around treatment — tests parallel trends & dynamic effects | outcome, unit, time, cohort, window | | **Mediation analysis (bootstrap)** | Hayes model 4 — indirect effect a·b with bootstrap CI | x, mediator, y, covariates, bootstrap | | **Moderation analysis** | Interaction X×W with simple slopes (mean-centred) | x, moderator, y, covariates | | **Propensity-score matching / IPW** | ATT via nearest-neighbour matching + IPW ATE + covariate balance | outcome, treatment, covariates | | **Regression discontinuity** | Local-linear LATE at a cutoff (sharp RD, triangular kernel) | outcome, running, cutoff, bandwidth | | **Synthetic control** | Comparative case study: weighted donor pool vs treated unit | outcome, unit, time, treated_unit, treatment_time | | **Uplift modeling (treatment effect)** | Two-model uplift: incremental response of a treatment per unit | outcome, treatment, predictors | ### 22.7 Consumer & Market Research (5) | Technique | What it does | Inputs | |---|---|---| | **Conjoint analysis (part-worths)** | Ratings-based conjoint — attribute utilities & importance | rating, attributes | | **Conjoint design generator** | D-efficient orthogonal profile plan for a conjoint study | levels, num_profiles | | **Discrete choice (conditional logit)** | Choice-based conjoint — utility weights from choice data | choice, choice_set, attributes | | **MaxDiff (best-worst scaling)** | Preference scores from best/worst choices | best, worst | | **RFM analysis (direct marketing)** | Recency/Frequency/Monetary scoring into customer value tiers | recency, frequency, monetary | ### 22.8 Time Series & Forecasting (38) | Technique | What it does | Inputs | |---|---|---| | **ACF / PACF** | Autocorrelation & partial autocorrelation with chart | column, lags | | **ARCH / EGARCH / GJR-GARCH** | Asymmetric & exponential volatility models with a choice of error distribution | column, model, dist, date, as_returns, order | | **ARCH-LM test** | Test for ARCH heteroskedasticity (volatility clustering) | column, lags | | **ARDL model** | Autoregressive distributed-lag model for mixed I(0)/I(1) series | dependent, predictors, lags, exog_order | | **Augmented Dickey-Fuller** | Unit-root / stationarity test | column, date, frequency | | **Differencing + ADF** | Difference a series to stationarity | column, date, frequency, order | | **Engle-Granger cointegration** | Two-series cointegration test | column1, column2 | | **GARCH volatility** | Model volatility clustering (ARCH/GARCH) | column, date, as_returns, order | | **Granger causality** | Test whether one series helps predict another | dependent, cause, date, maxlag | | **Hodrick-Prescott filter** | Trend/cycle decomposition | column, date, frequency, lambda | | **Johansen cointegration** | Trace + max-eigenvalue cointegration test | columns, date, lags | | **KPSS test** | Stationarity test (reversed null) | column, date, frequency | | **Ljung-Box test** | Test for remaining autocorrelation (white noise) | column, lags | | **Local projections (IRF)** | Jordà impulse responses — response of outcome to a shock by horizon | outcome, shock, controls, horizon | | **Mann-Kendall trend test** | Monotonic-trend test | column | | **Markov-switching regression** | Regime-switching model (bull/bear, high/low volatility states) | series, regimes, switching_variance | | **Moving averages (SMA/EWMA)** | Smoothing with SMA & EWMA | column, date, frequency, window | | **Rolling correlation** | Time-varying correlation + chart | column1, column2, window | | **Spectral periodogram** | Dominant cycles / power spectrum | column | | **Unit-root tests** | ADF, Phillips-Perron, DF-GLS and KPSS stationarity tests | column, date | | **VAR (vector autoregression)** | Multivariate VAR with variance decomposition + Granger causality | columns, date, maxlags, horizon | | **VECM (error correction)** | Vector error-correction model for cointegrated series | columns, date, lags | | **Classical decomposition** | Trend+seasonal+residual split | column, date, frequency, model, seasonal_periods | | **Expert modeler (auto-forecast)** | Auto-select ARIMA vs. exponential smoothing by AIC, with forecast | column, date, seasonal_periods, horizon | | **Exponential smoothing (Holt-Winters/ETS)** | Simple, Holt (trend) and Holt-Winters (seasonal) smoothing with forecast | column, date, frequency, trend, seasonal, seasonal_periods, horizon | | **STL decomposition** | Robust seasonal-trend decomposition with strength-of-seasonality metrics | column, date, frequency, seasonal_periods | | **Seasonal ARIMA (SARIMA/SARIMAX)** | ARIMA with a seasonal component (P,D,Q,s) and forecast with 95% CI | column, date, frequency, ar, diff, ma, sar, sdiff, sma, seasonal_periods, horizon | | **Seasonal naïve forecast** | Baseline seasonal benchmark forecast | column, date, frequency, seasonal_periods, horizon | | **Time series (ARIMA)** | Stationarity test, seasonal decomposition, ARIMA forecast with CI | column, date, frequency, horizon | | **Panel: Hausman test** | Fixed vs random effects specification test | entity, time, dependent, predictors | | **Panel: fixed effects** | Within (fixed-effects) regression with entity/time effects (Stata xtreg,fe) | entity, time, dependent, predictors, time_effects | | **Panel: random effects** | Random-effects GLS panel regression (Stata xtreg,re) | entity, time, dependent, predictors | ### 22.9 Design & Quality (DOE / SPC) (27) | Technique | What it does | Inputs | |---|---|---| | **Box-Behnken design** | 3-level RSM design (3 factors) | n_factors | | **Central composite design** | Response-surface (RSM) design | n_factors, center_points | | **D-optimal custom design** | Model-driven optimal design when a classical design is impractical | n_factors, model, runs | | **Definitive screening design (DSD)** | 3-level screening design (Jones-Nachtsheim), ~2m+1 runs, estimable curvature | num_factors | | **Factorial ANOVA** | Analyze a factorial experiment | response, factors | | **Fit definitive screening** | Fit main effects + quadratic curvature from a DSD run's response | response, factors | | **Fractional factorial design** | Half-fraction screening design | n_factors | | **Full factorial design** | Generate a full factorial DOE | n_factors, levels | | **Mixture design (simplex-lattice)** | Simplex-lattice design for formulation/mixture experiments (proportions sum to 1) | num_components, degree | | **Response surface (RSM)** | Fit a quadratic optimization model | response, factors | | **Space-filling design (Latin Hypercube)** | Latin Hypercube design for computer experiments / simulation | num_factors, num_runs | | **Taguchi orthogonal array** | Robust-design screening arrays (L4/L8/L16/L32, L9/L27) | n_factors, levels | | **One-way ANOVA power** | Observed (post-hoc) power and N for 80% power at the sample effect size | column, group | | **Power analysis** | A-priori sample size for t-tests and ANOVA | kind, effect_size, alpha, power, groups | | **Power — correlation** | Sample size to detect a correlation | r, alpha, power | | **Power — two proportions** | Sample size for two proportions | prop1, prop2, alpha, power | | **CUSUM control chart** | Cumulative-sum shift detection | column | | **Control chart builder (auto/phased)** | Auto-select I-MR/X-bar-R/p/c chart with optional baseline-phase limits | column, chart_type, subgroup_size, phase | | **EWMA control chart** | Detects small sustained shifts | column, lambda | | **Gage R&R (MSA)** | Measurement systems analysis | part, operator, measurement | | **I-MR control chart** | Individuals & moving-range chart | column | | **Multivariate SPC (PCA T²/SPE)** | Model-driven multivariate control charts (Hotelling T² and SPE/Q) | columns, variance | | **Pareto analysis** | Vital-few frequency chart | column | | **Process capability (Cp/Cpk)** | Capability indices vs spec limits | column, lsl, usl | | **X-bar & R chart** | Subgroup mean/range control chart | column, subgroup_size | | **c-chart (defect count)** | Defects-per-unit control chart | column | | **p-chart (proportion)** | Fraction-defective control chart | column, subgroup_size | ### 22.10 Survival & Clinical (21) | Technique | What it does | Inputs | |---|---|---| | **Cox diagnostics (PH test + baseline)** | Cox model with Schoenfeld proportional-hazards test and baseline survival curve | time, event, predictors | | **Cox proportional hazards** | Semi-parametric survival regression with hazard ratios | time, event, predictors | | **Life table (actuarial)** | Interval survival, conditional death probability & survival curve | time, event, intervals | | **Log-rank test** | Compare survival curves across groups | time, event, group | | **Nelson-Aalen cumulative hazard** | Nonparametric hazard estimate + chart | time, event | | **Survival (Kaplan-Meier)** | Life tables, median survival, survival curves and log-rank test | time, event, group | | **Cochran-Mantel-Haenszel** | Stratified association test | exposure, outcome, stratum | | **Diagnostic test accuracy** | Sensitivity/specificity/PPV/NPV/LR | test, truth | | **Incidence rate** | Events per person-time with CI | events, person_time | | **Meta-analysis (fixed-effect)** | Pool study effects + heterogeneity | effect, se | | **Meta-analysis (random effects)** | DerSimonian-Laird pooling with forest, funnel, Egger & trim-and-fill | effect, se, label | | **Meta-regression** | Regress study effect sizes on a moderator (random-effects WLS) | effect, se, moderator | | **Non-inferiority test (means)** | NI test with a margin | column, group, margin | | **Number needed to treat** | ARR and NNT | exposure, outcome | | **Odds ratio** | Odds ratio with 95% CI (2×2) | exposure, outcome | | **ROC curve / AUC** | Discrimination of a predictor | predictor, outcome | | **Relative risk** | Risk ratio with 95% CI (2×2) | exposure, outcome | ### 22.11 Reliability Engineering (7) | Technique | What it does | Inputs | |---|---|---| | **Accelerated life test (ALT)** | Weibull AFT of life vs stress (Arrhenius/inverse-power) — extrapolate to use conditions | time, stress, event, model, use_stress | | **Degradation analysis** | Extrapolate degradation paths to a failure threshold, then fit a Weibull life model | unit, time, measurement, threshold | | **Life distribution (multi-fit)** | Fit & compare Weibull/lognormal/exponential/log-logistic by AIC, with profiler | time, event | | **Parametric survival regression (AFT)** | Accelerated-failure-time regression with a choice of life distribution | time, event, predictors, distribution | | **Recurrent events (MCF)** | Mean cumulative function for repairable-system / warranty events | unit, time | | **Reliability growth (Crow-AMSAA)** | NHPP power-law growth model with instantaneous MTBF and Duane plot | time, total_time | | **Weibull reliability (life data)** | Weibull fit with shape/scale, MTTF, B10 and reliability curve (censoring supported) | time, event, reliability_at | ### 22.12 Bayesian (7) | Technique | What it does | Inputs | |---|---|---| | **Bayesian contingency table** | Bayes factor for association vs. independence in a two-way table | row, column | | **Bayesian linear regression** | Bayes factor that predictors improve on the null model | dependent, predictors | | **Bayesian one-way ANOVA** | Bayes factor for a group-mean difference (BIC/JZS approximation) | column, group | | **Bayesian repeated-measures ANOVA** | Bayes factor for a within-subject condition effect | columns | ## 23. Machine-learning operator reference The ML Workflows canvas offers **50 operators**. Drag them onto the canvas, wire Data -> Prep -> Model -> Evaluate, then Run. ### Data (4) | Operator | What it does | Parameters | | --- | --- | --- | | Dataset | Source table — loaded by the server when the workflow runs | Dataset | | Select columns | Keep only the listed columns | Columns to keep | | Filter rows | Keep rows matching a condition | Column, Operator, Value | | Sample | Random sample by fraction or row count | Fraction (0-1), Rows, Seed | ### Prep (6) | Operator | What it does | Parameters | | --- | --- | --- | | Impute missing | Fill missing values (fit on train when a split exists) | Strategy, Columns (blank = auto), Constant value | | Scale | Standardize (z-score) or min-max scale numeric columns | Method, Columns (blank = all numeric) | | One-hot encode | Dummy-code categorical columns (max 20 levels per column) | Columns (blank = all categorical) | | Train/test split | Hold out a test partition; downstream prep/models fit on train | Test fraction (0.1-0.5), Stratify, Stratify column, Seed | | Select features | Rank features vs the target and keep the top-k | Target, Keep top k, Method | | PCA | Reduce numeric columns to principal components | Components, Target to keep (optional) | ### Model (37) | Operator | What it does | Parameters | | --- | --- | --- | | Linear / Logistic | LinearRegression or LogisticRegression depending on the target | Target | | Decision tree | CART decision tree | Target, Max depth | | Random forest | Bagged tree ensemble | Target, Trees, Max depth | | Gradient boosting | Histogram gradient boosting | Target, Max depth | | k-nearest neighbours | Distance-based prediction | Target, Neighbours (k) | | Naive Bayes | GaussianNB (classification only) | Target | | Extra trees | Extremely randomized tree ensemble | Target, Trees, Max depth | | Gradient boosting (exact) | Classic gradient boosting (staged trees) | Target, Stages, Max depth | | XGBoost | Extreme gradient boosting (leaderboard workhorse) | Target, Rounds, Max depth | | LightGBM | Fast histogram gradient boosting | Target, Rounds, Max depth | | Support vector machine | Kernel SVM (SVC/SVR) | Target, C (regularization), Kernel | | Neural network (deep MLP) | Configurable feed-forward neural network (deep learning for tabular data) | Target, Hidden layers (e.g. 128,64,32), Activation, Solver, L2 regularization (alpha), Max epochs, Early stopping | | Ridge / L2 | L2-penalized linear/logistic | Target, Alpha, C (classification) | | Lasso / L1 | L1-penalized linear/logistic | Target, Alpha, C (classification) | | Elastic net | L1+L2 penalized linear/logistic | Target, Alpha, C (classification) | | AdaBoost | Adaptive boosting ensemble | Target, Estimators | | Bagging | Bootstrap-aggregated base learners | Target, Estimators | | Voting ensemble | Soft-vote blend of RF + GBM + linear | Target | | Stacking ensemble | Meta-learner over RF + GBM + linear | Target | | Linear discriminant (LDA) | Linear discriminant classifier | Target | | Quadratic discriminant (QDA) | Curved-boundary discriminant classifier | Target | | SGD linear | Stochastic-gradient linear model (fast on large data) | Target | | Passive-Aggressive | Online large-margin linear model | Target | | Gaussian process | Kernel probabilistic model (best on smaller data) | Target | | Bernoulli Naive Bayes | Naive Bayes for binary/boolean features | Target | | Huber regression | Outlier-robust linear regression | Target | | RANSAC regression | Robust regression by inlier consensus | Target | | Theil-Sen regression | Median-slope robust regression | Target | | Kernel ridge | Kernelized ridge regression | Target, Alpha | | Nu-SVM | Nu-parameterized kernel SVM | Target | | K-means | Unsupervised clustering (no target) | Clusters (k), Columns (blank = all numeric) | | DBSCAN | Density-based clustering (auto noise detection) | Epsilon, Min samples, Columns (blank = all numeric) | | Hierarchical clustering | Agglomerative (Ward) clustering | Clusters (k), Columns (blank = all numeric) | | Gaussian mixture (GMM) | Soft probabilistic clustering | Components (k), Columns (blank = all numeric) | | Spectral clustering | Graph-based non-convex clustering | Clusters (k), Columns (blank = all numeric) | | Isolation Forest (anomaly) | Unsupervised anomaly / outlier detection | Contamination, Columns (blank = all numeric) | | Optimize (AutoML tune) | Hyperparameter search over a base algorithm | Target, Algorithm, Search, Iterations (random/Bayesian) | ### Evaluate (3) | Operator | What it does | Parameters | | --- | --- | --- | | Evaluate | Metrics + ROC/lift/gains on the holdout (5-fold CV when no split) | — | | Explain | Permutation importance, partial dependence, SHAP attributions | — | | Score | Apply the trained model to the full dataset | — | --- *(For a one-click score any pipeline must beat, use **⚡ Instant baseline** on the ML Workflows list page — §31.6.)* ## 24. Data tools — SQL Editor & Alerts ### 24.1 SQL Editor Run DuckDB SQL directly against any dataset in the workspace, for power users who prefer queries to the visual tools. ![The SQL Editor — write a query, pick tables from the right-hand list, Run, and export the result to CSV or Excel.](/manual-img/83-sql.png) 1. Open **Data → SQL Editor**. 2. The right panel lists every queryable **table** (each dataset's physical table name) and your **Saved queries**. Expand a table to see its columns; clicking a table name when your SQL ends in `FROM ` inserts `"table" LIMIT 100`. 3. Type SQL in the editor — e.g. `SELECT contract, COUNT(*) AS customers, ROUND(AVG(monetary),0) AS avg_spend FROM customer_churn_… GROUP BY contract ORDER BY customers DESC`. 4. Click **Run (⌘⏎)**. Results appear as a table below. 5. Export with **CSV** or **Excel**, or name and **Save** the query to reuse it later. Supports full DuckDB SQL — CTEs, window functions, `corr()`, `regr_slope()`, joins across datasets, and aggregate/statistical functions beyond the point-and-click builders. ### 24.2 Alerts Get notified when a metric crosses a threshold or moves abnormally. ![The Alerts module — threshold and anomaly rules with an activity feed, delivered in-app, by webhook and by email.](/manual-img/84-alerts.png) 1. Open **Data → Alerts** and click **New alert**. 2. Choose a **threshold** alert (fires when a value crosses a fixed number) or an **anomaly** alert (learns a baseline and fires on unusual moves). 3. Set the metric (aggregation + column, and a date column for anomaly alerts), the schedule (**every 15 min**, **hourly** or **daily 08:00**), and delivery — **in-app**, a **Slack-compatible webhook URL**, and/or **email**. Click **Create**. 4. Fired alerts appear in the **Activity feed** on the right. --- ### 24.3 Morning briefs Below the alert list, **Morning briefs** are scheduled narrative reports: pick a dataset, a date column, one or more metric columns, optional dimensions and a period (day/week/month). Each run compares the latest period with the previous one and an 8-period baseline (z-score anomaly flag), decomposes the change by each dimension to name the key drivers, and writes a plain-English narrative — delivered in-app, by Slack-compatible webhook and by email on a cron schedule (default 07:00 daily). **👁 Preview now** shows the brief immediately, **💾 Save & schedule** registers it, and 🔊 **Listen** reads it aloud. See §28.6. ## 25. Research workbench process reference (complete) This section documents **every process** in each research workbench — the steps to run it, all available options/techniques, and the outputs — so any user can operate them end to end. (Statistics-workbench techniques are catalogued in §22; the AI Analyst in §11.) ### 25.1 PLS-SEM (structural equation modelling) A visual partial-least-squares SEM workbench comparable to SmartPLS: draw a measurement model (latent constructs + indicators) and a structural model (paths) on a canvas, estimate with the PLS algorithm, bootstrap for inference, then run a large battery of advanced procedures. Produces publication-ready validity/reliability tables, a path diagram, and an AI-written report. (Premium.) ![PLS-SEM editor — the path diagram with constructs, indicators and structural paths.](/manual-img/50-pls.png) After **▶ Estimate**, the **Measurement** tab reports the reflective measurement model — an automated pass/fail assessment plus construct reliability & validity (Cronbach's α, rho_A, CR, AVE) and outer loadings/weights with VIF: ![PLS-SEM Measurement tab — measurement-model assessment (all criteria met), construct reliability/validity (α, rho_A, CR, AVE), and outer loadings with VIF.](/manual-img/52-pls-measurement.png) The **Structural** tab reports the hypothesis tests — path coefficients with f² and inner VIF, R²/adjusted R² for each endogenous construct, SRMR model fit, and a plain-English interpretation (run **Bootstrap** to add t-values, p-values and confidence intervals): ![PLS-SEM Structural tab — path coefficients (PEOU→PU 0.429, PU→INT 0.418), R² for PU and INT, SRMR fit, and interpretation.](/manual-img/53-pls-structural.png) Click **Bootstrap** to add inferential statistics — the path table gains **SE, t-values, p-values, percentile confidence intervals (2.5–97.5 %) and significance stars**: ![PLS-SEM after Bootstrap — path coefficients with SE, t, p, 95% CI and significance (3/3 paths significant).](/manual-img/54-pls-bootstrap.png) The **IPMA** tab (importance-performance map analysis) plots each construct's total-effect **importance** against its **performance** (0–100) for a chosen target, highlighting where to focus improvement: ![PLS-SEM IPMA — importance vs. performance map for the target construct.](/manual-img/56-pls-ipma.png) **How to run it.** 1. On the **Research** page, enter a **Model name**, pick the **dataset** with your indicator/survey data, and click **New model**. 2. In the **Model** tab, add latent variables with **+ Construct** (name it; choose **Reflective (Mode A)** or **Formative (Mode B)**; tick indicator columns) — or click **✦ Auto-detect constructs** to build the whole measurement model from item-name prefixes (`peou1/peou2/…` → "Peou"). 3. Select a construct to open the inspector: switch reflective/formative, position indicators, or make it a **higher-order component**. 4. Toggle **→ Connect paths** and click source → target to draw hypotheses; use **✦ Moderation** / **⤳ Mediation** to add those effects. Set **Show on paths** (β / p / t / none), **Loadings**, **R²**, **Sig. stars**; **⤢ Tidy layout**; **Download PNG/PDF**. 5. Pick the **inner weighting scheme** (Path / Factor / Centroid) and click **▶ Estimate**. 6. Choose bootstrap resamples (500 / 1000 / 2000 / 5000) and click **Bootstrap** (runs on a background worker with live progress). 7. Review the tabs; **✦ Interpret with AI** writes a narrative; **Report → 🖨 Print / PDF** exports the full document. **Every process/option (by tab).** Model, Measurement, Discriminant, Structural, Effects, Predict, MGA, IPMA, Moderation, Procedures, **Reviewer**, Data, **Bayesian** (shown after a Bayesian estimation), Report. - **Specification:** reflective (Mode A) / formative (Mode B); higher-order components; auto-detected constructs; mediation & moderation tagging. - **Estimation:** PLS with Path/Factor/Centroid weighting; handles missing values; scales to ~1,000,000 rows. - **Measurement model:** Cronbach's α, rho_A, CR/rho_c, AVE; outer loadings & weights with outer VIF; cross-loadings; automated pass/warn/fail assessment (CR/α ≥ 0.70, AVE ≥ 0.50, loadings ≥ 0.708, HTMT < 0.90). - **Discriminant validity:** Fornell-Larcker; HTMT with bootstrap CIs. - **Structural model:** path coefficients with SE, t, p, bootstrap CIs, stars, f², inner VIF; R² & adjusted R²; SRMR; automated structural assessment. - **Effects:** direct/indirect/total & specific indirect effects (mediation chains). - **Predict:** blindfolding **Q²**, **PLSpredict** (out-of-sample vs LM benchmark), **CVPAT**. - **MGA:** multigroup analysis (permutations) + **MICOM** invariance. - **IPMA:** importance-performance map (scatter + table). - **Moderation:** two-stage, product-indicator, quadratic/nonlinear, endogeneity (Gaussian copula). - **Procedures:** **PLSc** (consistent PLS), **NCA**, **CTA-PLS**, **q²** effect sizes, **CB-SEM** (χ²/CFI/TLI/RMSEA), and latent-class segmentation via **FIMIX-PLS**, **PLS-POS**, **REBUS-PLS**. - **Report:** sample & model-fit tables (SRMR, d_ULS, d_G, NFI, RMS_theta, χ²), embedded diagram, AI interpretation, all tables. **Outputs.** Interactive path diagram (PNG/PDF); validity/reliability, loadings, cross-loadings, Fornell-Larcker, HTMT, path, R², effects tables (Copy/CSV); pass/warn/fail assessment cards; IPMA chart; AI narrative; print-to-PDF report. Every run is audited. - **Reviewer:** **Run battery** — *Fast* (PLSpredict, CVPAT, PLSc, q²/f², IPMA, CB-SEM concordance) or *Full* (+ Gaussian-copula endogeneity, NCA, FIMIX-PLS) with a Hair et al. coverage score (§28.4). - **Export / import:** **Export spec → lavaan / Mplus / seminr → ⬇ Download**; on the PLS-SEM list page paste **lavaan syntax** and **Create model from syntax** (§28.4). ### 25.2 Forecasting Lab A hybrid decomposition + adaptive-model-selection time-series forecaster: decompose a series, pick the best model per component from a pool, recombine, and benchmark against ARIMA — with classical, ML, light-neural and deep/foundation models, multivariate inputs, three evaluation protocols, optional tuning, and a one-click Word report. (Premium; heavy runs are background jobs.) ![Forecasting Lab — the run form: dataset, target, decomposition, horizon, model pool and evaluation mode.](/manual-img/20-forecast-form.png) **How to run it.** 1. Select a **Dataset**, **Target column**, and optionally a **Date column**. 2. Choose a **Decomposition** (default CEEMDAN) and **Forecast horizon** (5–365). 3. Toggle models in the **Model pool** (deep models show a GPU badge; light-neural an NN badge). 4. Optionally toggle **Exogenous inputs** (multivariate predictors). 5. Pick **Evaluation**: One-shot (multi-step), Rolling one-step (paper protocol), or Benchmark (compare all + MCS/DM); set a **Test window** for the latter two. 6. Optionally enable **Hyperparameter tuning (MOTPE / Optuna)**. 7. Click **▶ Run hybrid forecast**; then **📄 Publication report (.docx)** or **✨ + AI abstract**. ![Forecasting Lab — benchmark comparison of every model with MCS membership and Diebold-Mariano significance.](/manual-img/21-forecast-benchmark.png) **Every option.** - **Decompositions:** None, EMD, EEMD, CEEMDAN, VMD, Wavelet, STL, MSTL, SSA, EWT, Fourier low-pass. - **Model pool:** Naïve/Mean/Drift; Linear, Ridge, Random Forest, Gradient Boosting, SVR, k-NN; ARIMA, ETS, Theta; light-neural DLinear, NLinear, N-BEATS, N-HiTS, TiDE; deep LSTM, BiLSTM, PE-BiLSTM, PatchTST, Chronos, TimesFM. - **Evaluation:** one-shot multi-step; rolling one-step (re-decomposes each step); benchmark hold-out with **Model Confidence Set** (Hansen–Lunde–Nason 90%), pairwise **Diebold–Mariano** (HLN-corrected), SPA vs naïve. - **Metrics:** RMSE, MAE, MAPE, sMAPE, MASE, RMSSE, Theil U2, DirAcc %, out-of-sample R², % improvement vs ARIMA, Pesaran–Timmermann directional test, DM stars, MCS membership, SPA vs naïve. **Outputs.** Summary stat tiles; forecast-vs-actual, per-model RMSE, residual and decomposition charts; metric tables (Copy/CSV); downloadable **.docx** publication report (optional AI abstract) with embedded figures. - **Auto-ensemble:** click **🎯 Run ensemble** in the *Auto-ensemble + conformal prediction interval* card for a rolling-origin back-tested top-3 ensemble with a 90 % conformal band (§17.6, §28.5). ### 25.3 Literature Review + Meta-Analysis An end-to-end systematic-review workbench in three tabs — **Search & screening** (with a **🔄 Living reviews** card), **Review builder (agent)** (with **🧪 Dual-LLM data extraction**), and **Meta-analysis** — that searches free scholarly APIs, screens with a PRISMA flow, monitors the literature on a schedule, extracts study data from PDFs with adjudication, drafts a manuscript, and runs a full meta-analysis. Uploaded PDFs are stored on local disk only (never cloud), under a 24-hour TTL with explicit purge. The module has **three tabs**, used in sequence — you can use any on its own. **Tab 1 · Search & screening** — find and triage the literature. Type a query, pick which **Sources** to query (Crossref, OpenAlex, arXiv, Semantic Scholar, PubMed, Europe PMC, DOAJ, Google Scholar), set **Per source** (5–50), and **🔍 Search**. It queries all chosen APIs, merges and **de-duplicates** the hits, and shows a **PRISMA screening flow** (Records identified → After de-duplication → Included → Excluded → Unscreened) with per-source hit counts. Each result card shows title, authors, year, venue, citation count and an open-access badge; click **✓ Include / ✕ Exclude** to screen, then **Save N included as a dataset** (it appears in Datasets, ready for the Meta-analysis tab). *Output:* the PRISMA flow, screened record cards, and an included-studies dataset. ![Tab 1 · Search & screening — the PRISMA 2020 flow diagram (Identification → Screening → Included with duplicate/excluded branches and per-source counts) above the screenable paper cards.](/manual-img/35-prisma-diagram.png) The screening results are drawn as a proper **PRISMA 2020 flow diagram**: an *Identification* box (records identified from databases, with the per-source breakdown) branching to *duplicate records removed*, a *Screening* box (records screened) branching to *records excluded / not yet screened*, and an *Included* box (studies included) — the diagram updates live as you include/exclude records, and appears in both the Search & screening and Review-builder tabs. **Tab 2 · Review builder (agent)** — turn a topic into a structured review automatically. Enter a review question, choose sources and **Per source** limit, and **🤖 Run review agent** (a background job). The agent searches across the sources, de-duplicates, **clusters the papers into themes** (shown as a keyword co-occurrence network + cluster-size chart), computes **bibliometrics** (year span, source and author counts), and builds a **PRISMA flow**. You can **⬆ Upload your own PDFs (SCOPUS/WoS)** — they are analysed and merged, kept on local disk only for 24 h, and purged with **Finish & delete PDFs**. Then produce a **quality-appraisal table** (choose an instrument — MMAT 2018, Cochrane RoB 2, ROBINS-I, Newcastle-Ottawa, AMSTAR-2, GRADE — with optional AI-suggested ratings) and **📄 Generate review manuscript (.docx)** in a chosen **citation style** (APA 7, Harvard, MLA 9, Chicago, IEEE, Vancouver, AMA, ACS), optionally AI-drafted and grounded in the found papers. *Output:* theme cluster diagram, bibliometrics, PRISMA flow, appraisal table, and a downloadable Word manuscript. ![Tab 2 · Review builder — the agent's PRISMA flow, thematic keyword-co-occurrence cluster diagram, appraisal-instrument picker and manuscript generator.](/manual-img/34-lit-review.png) **Tab 3 · Meta-analysis** — quantitatively pool study results. Pick a **dataset** of studies (rows) × effect columns, choose an **Effect size type** (Generic effect+SE/variance/CI; Standardised mean difference / Hedges' g; Odds ratio or Risk ratio from 2×2 counts; Correlation r,n), a **Model** (Random effects — DerSimonian–Laird, or Fixed effect — inverse variance), **map your columns** to the required fields (auto-mapped by name), optionally set a numeric **Moderator** for meta-regression, and **▶ Run**. *Output:* summary tiles (pooled effect, 95% CI, p-value, I², k studies), a **forest plot** (per-study effects + pooled diamond), a **funnel plot**, a per-study weights table, and heterogeneity (Q, I², τ², H², prediction interval) with publication-bias diagnostics (Egger, trim-and-fill). ![Tab 3 · Meta-analysis — pooled effect 0.53 [0.43, 0.62] with the forest plot, funnel plot, heterogeneity and per-study weights.](/manual-img/32-lit-meta.png) **Every option.** - **Sources:** Crossref, OpenAlex, arXiv, Semantic Scholar, PubMed, Europe PMC, DOAJ, Google Scholar (best-effort). - **Screening:** DOI/title de-dup, PRISMA counts, manual include/exclude, save-to-dataset, open-access links. - **Review agent:** thematic clustering (network diagram), bibliometrics, PDF ingestion (24-h local TTL), study-characteristics table. - **Appraisal instruments:** MMAT 2018 (5 categories), Cochrane RoB 2, ROBINS-I, Newcastle-Ottawa, AMSTAR-2, GRADE SoF; optional AI ratings. - **Citation styles:** APA 7, Harvard, MLA 9, Chicago, IEEE, Vancouver, AMA, ACS. - **Meta-analysis:** generic (effect+SE/variance/CI), SMD (Hedges' g), odds ratio, risk ratio, correlation; random-effects (DerSimonian–Laird) or fixed-effect; heterogeneity (Q, I², τ², H²); prediction interval; publication bias (Egger, trim-and-fill); subgroup; meta-regression. **Outputs.** Record cards; PRISMA cards; theme diagrams & bibliometric charts; appraisal/characteristics tables; meta-analysis tiles (pooled effect, CI, p, I², k), forest & funnel charts, weights, heterogeneity & bias tables; downloadable review **manuscript (.docx)**; included-studies dataset. - **Living reviews:** scheduled re-search (weekly / daily / monthly / twice a month), new-records-only diff, optional e-mail, **▶ Run now** (§29.2). - **Dual-LLM extraction:** two extractor passes per uploaded PDF, agreement auto-accepted, disagreements to an adjudication queue with page anchors, AI-use disclosure sentence (§29.3). ### 25.4 Rigor Guard The research-integrity hub. Card 1 is a "reviewer pre-flight" for quantitative studies: given a dataset and intended design, it runs the checks Q1 reviewers reject on and returns a red/amber/green submission-readiness score, plus an AI methodology critique, reporting-standard checklists, a reproducibility manifest and a free-text preregistration/deviation workflow. The page also hosts: **🧭 Method advisor** (§28.1); **🛡 Analysis runs** — Trust Score, Inspector, Reproducibility Bundles, Zenodo, Robustness, APA (§27); **🔬 Rigor Guard v2** — statcheck/GRIM, citation verifier, overclaim linter (§27.5); **📋 Reporting-standard coverage & Reviewer 2** (§27.6); **🔒 Executable preregistration** with specification ledger and auto deviation report (§27.7); **🌌 Multiverse / specification-curve** (§27.8); **🧭 Causal DAG** (§29.1); **🏆 Hypothesis tournament** (§30.1); **🚦 Publication gate** (§30.2). ![Rigor Guard — the pre-flight form: dataset, intended design, outcome and predictors.](/manual-img/40-rigor-form.png) **How to run it.** 1. Pick a **Dataset**, **Intended design** (Linear/Logistic regression, t-test, ANOVA, Correlation, Chi-square, General), and **Outcome/target**. 2. For t-test/ANOVA choose a **Grouping variable**; for regression pick **Predictors**. 3. **🛡 Run submission pre-flight** for the scored report; **✨ AI methodology critique**; **⬇ Reproducibility manifest (.json)**. 4. Under **Reporting-standard checklist**, choose a standard and **Load checklist**. 5. Under **Preregistration (free-text plan)**, fill the plan and **🔒 Save & lock**; later **Diff** the analyses actually run to produce a deviation report. ![Rigor Guard — the scored submission-readiness report with per-check status and remedies.](/manual-img/41-rigor-result.png) **Every check.** Sample size & statistical/achieved power (per design); missing data (%, MCAR/MAR/MNAR reminder) with a missingness chart; normality of outcome (Shapiro–Wilk); multicollinearity (max VIF); outliers (IQR); multiple comparisons (Holm/BH reminder); class balance (logistic); scale reliability (Cronbach's α); always-on effect-size+CI and assumptions/robustness reminders; AI methodology critique. **Reporting checklists:** PRISMA 2020, PRISMA-ScR, CONSORT 2010, STROBE, TRIPOD. **Reproducibility manifest:** timestamp, Python/OS, seed, library versions, dataset metadata, config. **Preregistration & deviation diff.** **Outputs.** A readiness **score /100** with color verdict and fail/warn counts; a check table (status glyph, detail, remedy); a missingness chart; AI critique; printable checklist; preregistration list & deviation diff; downloadable **reproducibility_manifest.json**. ### 25.5 Prediction Profiler A JMP-style interactive what-if profiler: fit a model of a response on several predictors, then slide each predictor to see how the prediction moves (others held fixed), and optimize the settings to maximise/minimise/hit a target (desirability optimization). ![Prediction Profiler — per-predictor response curves with a live reference point and desirability optimization.](/manual-img/90-profiler.png) ![Prediction Profiler after Optimize — the predicted response and model R², with each predictor jumped to the settings that maximise the response.](/manual-img/91-profiler-optimize.png) **How to run it.** 1. Pick a **Dataset**, a **Response**, and a **Model** (Gradient boosting / Random forest / Linear). 2. Select **Predictors (factors to profile)**; click **▶ Fit & profile**. 3. Edit any predictor's value to re-profile live (a reference line marks the current point). 4. Set a **Goal** (Maximise / Minimise / Hit target) and click **🎯 Optimize** to jump to the best settings. **Options.** Linear / Random forest (200 trees) / Gradient boosting; 30-point profile grid over each predictor's observed range; debounced live re-profiling; desirability optimization via SciPy `differential_evolution`. **Outputs.** Predicted-response and model-**R²** tiles; a grid of per-predictor line charts with a "current" reference line and value boxes; optimizer returns the best settings and predicted optimum. ### 25.6 ML Workflows A RapidMiner-style visual machine-learning pipeline builder: drag operators (Data / Prep / Model / Evaluate) onto a canvas, wire them into a DAG, set parameters, and run — each node's output is inspectable. A one-click **Auto Model** builds a cross-validated leaderboard from a dataset + target; explainability includes permutation importance, partial dependence and SHAP. ![ML Workflows — the visual pipeline canvas with operators wired into a flow.](/manual-img/55-ml.png) ![ML Auto Model — the data-quality screen and a 5-fold cross-validated leaderboard ranking every algorithm.](/manual-img/85-ml-automodel.png) **How to run it.** - **Auto Model:** on the list page, pick a dataset + **Target column**, **Run**, read the leaderboard. **⚡ Instant baseline (seconds)** on the same page gives the dummy / regularised-linear / fast-GBM score any real model must beat, with a target-leakage screen (§31.6). - **Visual pipeline:** **New workflow** → from the **Operators** panel add nodes (**+ operator**); toggle **Connect** and click source → target to wire edges; select a node to set parameters; **Save**, then **▶ Run**; click a node (green dot) to view its result; red dots = errors. **Every operator.** - **Data:** Dataset, Select columns, Filter rows, Sample. - **Prep:** Impute missing, Scale (standard/minmax), One-hot encode, Train/test split (optional stratify), Select features (mutual-info / F-test), PCA. - **Model (supervised):** Linear/Logistic, Decision tree, Random forest, Gradient boosting (hist & exact), k-NN, Naive Bayes (Gaussian/Bernoulli), Extra trees, XGBoost, LightGBM, SVM/Nu-SVM, Neural network (deep MLP), Ridge, Lasso, Elastic net, AdaBoost, Bagging, Voting & Stacking ensembles, LDA, QDA, SGD, Passive-Aggressive, Gaussian process, Huber, RANSAC, Theil-Sen, Kernel ridge. - **Model (unsupervised):** K-means, DBSCAN, Hierarchical, Gaussian mixture, Spectral clustering, Isolation Forest (anomaly). - **Tuning (Optimize node):** grid / random / **Bayesian (Optuna/TPE)** search, CV-scored. - **Evaluate:** Evaluate (metrics + ROC/lift/gains, or 5-fold CV), Explain (permutation importance, partial dependence, SHAP), Score (apply model to the dataset). - **Auto Model:** data-quality screen, 5-fold CV leaderboard over 6+ algorithms plus a top-3 soft-vote Blender, with feature importance; task auto-detected. **Outputs.** Metric tables (accuracy/precision/recall/F1/ROC-AUC/confusion, or RMSE/MAE/R²), ROC & gains charts, feature-importance & SHAP tables/charts, partial-dependence, scoring summaries, clustering size tables (BIC/AIC for GMM), anomaly counts, leaderboard bars. ### 25.7 Qualitative An NVivo-style qualitative-coding workbench: each row of a text column becomes a document; highlight passages and apply hierarchical codes, then run word/matrix/sentiment/AI queries, auto-code, build concept maps, code media, manage case classifications, and export. AI covers code suggestion, thematic analysis, summarization and grounded Q&A. ![Qualitative workbench — codebook, documents with inline highlights, and the query panel.](/manual-img/60-qual.png) ![Qualitative sentiment query — auto-coded documents (green = positive, red = negative), the sentiment-distribution table & bar chart, and a word-frequency table.](/manual-img/61-qual-sentiment.png) **How to run it.** 1. On the list page, name a **Project**, pick a **Dataset** + **text column**, optionally tick **Case attributes**, **Create**. 2. **Workbench (3 panes):** left = **Codebook** (hierarchical codes, reference counts); center = **Documents** (attribute chips); right = **Queries / References**. 3. **Coding:** select text → floating bar → choose a code and **Apply**, **+ New**, **Suggest** (AI), **Note**, or **🔗 Link** two passages. 4. **Auto-coding:** topics, sentiment, pattern-spread, **✨ AI themes**. 5. **Tools:** Concept map, Media coding, Classification sheet, Merge (import another coder), Export (JSON / segments CSV / **REFI-QDA .qdpx** for NVivo, ATLAS.ti, MAXQDA, QDA Miner — §29.4). 6. **Queries:** pick a tab, set input, **▶ Run**. **Every query.** Frequency (word/bigram/trigram), Search (keyword-in-context), Matrix (code × attribute), Sentiment (VADER), Co-occur, Boolean (AND/OR/NOT/NEAR), Cluster (k-means docs), Word tree, Framework matrix, Compare (inter-coder Cohen's κ & Krippendorff's α), **Reliability** (two coders/origins: κ, Krippendorff's α, semantic/Jaccard agreement), **Saturation** (cumulative-codes curve), **Joint display** (mixed-methods theme × measure table) — see §29.4 — AI Themes, AI Summary, Ask AI (grounded Q&A with citations). **Outputs.** Highlighted documents; codebook with counts; query envelopes rendering prose (AI summaries/answers), tables (frequencies, matrices, agreement) and charts (bar, network, dendrogram, wordcloud, word tree); auto-coding cards; JSON/CSV export; merge import — all AI results grounded in the project's documents with citations. --- ## 26. What's new (September 2026) — the reviewer-proof research stack - **Map visualisations (1 Oct 2026)** — the Map chart gains an **India states map**, **point/bubble maps** from latitude/longitude columns and animated **flow maps** (origin → destination), with a Map section in the Format pane. See §3.1d. - **Data Stories (30 Sep 2026)** — a new *Analyze → Data Stories* module: sequenced scenes (title, dashboard with saved filter/spotlight state, chart, side-by-side compare, KPI strip, text, image), AI captions drafted from the scene's live data, presenter mode with keyboard/autoplay/fullscreen/notes, a scrollytelling reading page, public share links, version history, and PDF / PNG / PPTX export. See §3.6a. - **Executive dashboard polish (30 Sep 2026)** — `columnLabels` now renames series/legend entries, radar axes and bubble axes; pies with outside labels keep their labels readable (no more truncated “25…”); waterfall axes honour the Number format; shared-dashboard tiles hide the duplicate tile name when the chart has its own title. Sort & limit controls in the builder stay inside the sidebar. - **Tableau / Power BI filter set (30 Sep 2026)** — chart filters now cover pick-values (include/exclude), compare, range, wildcard/regex, null/blank, **Top N** (count or %), condition-by-field, relative dates, AND/OR and post-aggregation measure filters; dashboards gain Top N, relative-date and text filter kinds. See §3.1c. - **Data model with schema diagram (30 Sep 2026)** — Power BI-style relationships between datasets with auto-detection, validation and cardinality; related fields (`Dataset › column`) in the chart builder; dashboard filters that flow across tables; multi-rule Replace transform; smart Excel header detection for banner + two-row headers. See §3.6. - **Finance Lab result actions (29 Sep 2026)** — every Finance Lab result now has ✨ AI interpretation, PDF / Word / Excel / CSV / PNG downloads and a report builder that assembles the tabs into one document or an editable report in the Reports module; ticker search picker with live suggestions and a grouped benchmark list (§34). - **Prophet, audit sampling and credit risk (29 Sep 2026)** — Meta's Prophet joins the Forecasting Lab pool with a model card (seasonal business series) and a Prophet-style additive model in the notebook export (§17.3); a **Bernoulli / Poisson sample** transform step plus **Audit sampling — Horvitz-Thompson estimates** in Statistics (§2.6, §20.11); a **Credit risk** tab in Finance Lab with correlated-Bernoulli default simulation, credit VaR, expected shortfall and economic capital (§34.9). - **Interactive shared dashboards (29 Sep 2026)** — public share links now support the same cross-filtering, multi-select, Filter/Highlight brushing and filter controls as the editor. See §3.5. - **Finance Lab (29 Sep 2026)** — a new Research module for investment analysis: multi-asset performance and risk analytics, mean-variance / risk-parity portfolio construction with efficient frontier and back-test, technical signal board, Google News + GDELT headlines scored with a finance lexicon or the AI, a FinGPT-style analyst brief grounded in prices and news, SEC EDGAR fundamentals with a DCF valuer and sensitivity grid, retrieval-augmented Q&A over 10-K/10-Q filings, market-model event studies and a one-click equity research note (Word). InstaGenie has a matching `finance_lab` tool. See §34. - **Chart Format pane (28 Sep 2026)** — every chart and table now has Power BI-style formatting: titles/subtitles/captions, 15 palettes plus custom and per-series colours, axis titles and scales, legend position, data-label control, series styling (rounded/gradient/step/markers), pie centre text, KPI prefix/suffix, table totals/striping/header renames. See §3.1. InstaBizIntel gained a large set of research-integrity, AI-assistance and teaching features between 16 and 18 September 2026. This section is the map; §§27–31 explain each feature and how to use it. | Area | New capability | Where | |---|---|---| | Every analysis | **Audit run** with data & results hashes, **Trust Score**, **Inspector** (independent re-derivation), **Reproducibility Bundle** (RO-Crate, CITATION.cff, `reproduce.py`, Zenodo deposit) | Statistics workbench result bar · Rigor Guard → Runs | | Rigor Guard v2 | statcheck/GRIM numeric re-computation, Crossref **citation verifier**, **overclaim linter**, reporting-checklist **coverage**, **Reviewer 2** questions, **executable preregistration** + specification ledger + automatic deviation report, **multiverse / specification-curve** | Rigor Guard | | Method choice | **Method advisor** (assumption-checked, ranked, runnable), **Robustness** battery, **APA-7 write-up** per run | Rigor Guard · Statistics result bar | | PLS-SEM | **Reviewer battery** (PLSpredict, CVPAT, PLSc, q²/f², IPMA, CB-SEM concordance; full: copula endogeneity, NCA, FIMIX), **lavaan / Mplus / seminr export**, **lavaan import** | PLS-SEM → model → Reviewer tab · list page | | Forecasting | **Auto-ensemble** with rolling-origin backtests and **conformal prediction intervals** | Forecasting Lab | | Monitoring | **Morning briefs** — "what changed and why", scheduled, in-app / email / webhook, with 🔊 Listen | Data → Alerts | | Causal | **Causal DAG estimator** — backdoor adjustment, refutation tests, E-value, what-if; **skeleton discovery** | Rigor Guard | | Evidence | **Living reviews** (scheduled searches that surface only new records), **dual-LLM extraction** with adjudication queue and page anchors | Lit Review + Meta | | Qualitative v2 | **Dual reliability** (κ, Krippendorff α, semantic agreement), **saturation curve**, **joint display** (mixed methods), **REFI-QDA (.qdpx) export** | Qualitative workbench | | Publishing | **Hypothesis tournament**, **claim typing**, **reviewer gate**, **Methods from the execution log**, **APA Word export**, **AI-use disclosure / NIH DMS plan / availability statements** | Rigor Guard → Publication gate | | InstaGenie | **Plan-first autonomy** with Approve & run, **prompt queue**, **personas** (Analyst / Advisor / Socratic tutor / Explainer), **20 output languages**, **persistent memory**, **Deep Research** with a relevance filter | InstaGenie panel | | Teaching & Thesis | Validated **model templates** (TAM, UTAUT, UTAUT2, TPB, SERVQUAL, ECM, D&M IS Success) → questionnaire + PLS model, **careless-response screen**, **sample-size calculator**, **instructor process log**, **thesis pipeline** (proposal → prereg → instrument → analysis → Chapter 4 → viva) | Research → Teaching & Thesis | | Data | **Synthetic twin** of any dataset (Gaussian copula, fidelity + privacy report), **OSI semantic-model import**, **Instant baseline** for ML | Dataset page · ML Workflows | | Dashboards | **Governed version history** with diff and restore | Dashboard → History | | Admin | **AI-provider key vault** (14 presets incl. **InstaRoute** auto-routing, per-provider keys, active provider), **Visitors today** with day-wise per-user activity and anonymous sessions, **accuracy evaluation harness** with a public badge | Admin | | Integrations | **MCP server** (35 tools) for ChatGPT, Claude, Claude Code and any MCP client; CLI; agent skill; A2A card | §32 | | Notebook export | **📓 Notebook + PDF** on every analysis: an executed Jupyter notebook (clean-room open-source Python), the data, the platform results and a verification cell, plus HTML and PDF renderings — hand-written, engine-verified reference code for all 304 analyses plus PLS-SEM, Forecasting Lab and Auto Model | Result bars · Rigor Guard → Runs · PLS-SEM · Forecasting Lab · ML Workflows (§27.9) | | Appearance | Light theme fully readable (tables, badges, status colours); QA page at `/theme-check` | Light/Dark toggle | --- ## 27. Audit trail, Trust Score and reproducibility (every analysis) Every statistical run — whether you click **▶ Run** in the workbench, InstaGenie runs it for you, or an MCP client calls it — is now recorded as an **audit run**: the exact analysis key and parameters, a SHA-256 hash of the data used, the results, a hash of the results, who/what ran it (workbench, agent, plan), the LLM model if an agent was involved, and the duration. ![After every run the result bar shows the Trust pill and the Verify / Bundle / Robustness / APA / Visualize / Add-to-report actions.](/manual-img/41-rigor-result.png) ### 27.1 The Trust Score pill Next to every result you will see **🛡 Trust NN · label**. Click it for the five-factor breakdown: | Factor | What earns points | |---|---| | Deterministic re-run | The **Inspector** re-executed the analysis independently and the results hash matched | | Data integrity | The dataset hash still matches (nothing changed under the analysis) | | Preregistration | The run matches a locked plan (confirmatory) | | Assumptions | Assumption checks passed / were reported | | Provenance | Full parameters and environment captured | Scores ≥ 80 are green, 60–79 amber, below 60 red. A red score is not "wrong" — it usually means *not yet verified* or *exploratory*; use the actions below to raise it. ### 27.2 Verify (Inspector) Click **↻ Verify** on a result (or **Verify** in Rigor Guard → Runs). The Inspector reloads the data, re-runs the analysis from the recorded parameters and compares hashes. Verdicts: **✓ independently re-derived**, **✗ re-run differs** (investigate — usually the data changed), **⚠ data changed**. Runs shorter than 4 s are verified automatically the moment they finish. ### 27.3 Reproducibility Bundle Click **⬇ Bundle**. You get a ZIP that is a valid **RO-Crate 1.1** research object: - `ro-crate-metadata.json` (schema.org provenance), `CITATION.cff`, `codemeta.json` - `data_manifest.json` (hash, rows, columns), `analysis.json` (exact spec), `results.json` - `reproduce.py` — re-runs the analysis and verifies the hash on any machine with Python - `environment.json` (library versions, seed) **Deposit to Zenodo.** In Rigor Guard → Runs, paste a Zenodo personal token and click **Zenodo draft**: a draft deposition with the bundle and metadata is created; you publish it from Zenodo to mint the DOI for your Data/Code Availability statement. ### 27.4 The Runs ledger (Rigor Guard) **Research → Rigor Guard → Analysis runs** lists every run with dataset, source (workbench / agent / plan), tag (confirmatory / exploratory), Trust and Inspector status. From here you can **Verify**, **Bundle**, run **🧪 Robustness**, generate **📝 APA**, and select runs for the Publication gate (§30). ### 27.5 Rigor Guard v2 — what journals now check at submission In the **Rigor Guard v2** card: 1. **Numeric consistency (statcheck + GRIM).** Paste APA results (`t(28) = 2.31, p = .028`, `F(2, 57) = 4.10, p = .022`, χ², r) — each is recomputed; mismatches and **decision errors** (reported significance contradicts the statistic) are flagged. Add `mean, N, items` lines for the **GRIM** test of impossible means. You can also point it at a stored run to check the run's own tables. 2. **Citation verifier.** One reference per line (DOI or full reference). Each is resolved against **Crossref**: ✅ verified, ⚠️ suspect (DOI resolves but details differ), ❌ NOT FOUND (likely fabricated). 3. **Overclaim linter.** Paste results/discussion text; 12 pattern families (causal language on correlational designs, "proves", generalisation beyond the sample, p-value misstatements, …) are flagged with rewrites, tuned to the design you selected at the top. ### 27.6 Checklist coverage and Reviewer 2 Choose a reporting standard — **APA JARS-Quant, Hair et al. PLS-SEM, BARG/WAMBS (Bayesian), GRAMMS (mixed methods), CONSORT** — and a run. **Coverage** auto-checks the items that can be evidenced from the run's output (e.g. effect sizes, CIs, assumption tests) and lists what you still have to write. **Reviewer 2** produces the hostile-reviewer questions for that run and standard (tick *AI* for model-generated questions grounded only on the run). ### 27.7 Executable preregistration, specification ledger and automatic deviation report The **Executable preregistration** card turns a plan into something the platform can run: 1. Give the plan a title and hypotheses, then add entries — hypothesis label, dataset, analysis (from the catalogue), parameters as JSON. 2. **🔒 Lock plan.** The plan is hashed (`plan_hash`) and time-stamped. 3. Later, select the plan and click **▶ Run plan (confirmatory)**. Every entry runs exactly as registered and is tagged **confirmatory**; any other run on those datasets is automatically tagged **exploratory**. 4. **📒 Specification ledger** lists every run against the plan; **Δ Auto deviation report** compares what was registered with what was actually run — no manual typing. ### 27.8 Multiverse / specification-curve analysis Pick outcome, focal predictor, candidate covariates, outlier rules and transforms. **🌌 Run multiverse** estimates *every* reasonable specification and draws the specification curve (sorted effects with CIs; filled points are significant). The verdict tells you whether the effect is **robust** (most specifications agree) or **fragile** — the disclosure that pre-empts p-hacking concerns. --- ### 27.9 Notebook + PDF export — show the code behind every result Every result can be downloaded as a **Jupyter notebook that reproduces it with open-source Python** — for a supervisor, a reviewer's Code Availability request, a viva, or your own learning. Click **📓 Notebook + PDF** on a Statistics result bar (or the Rigor Guard Runs ledger), on an estimated PLS-SEM model, in the Forecasting Lab after a run, or on the ML Workflows page for Auto Model. Generation takes a few seconds (up to a minute or two for forecasting and Auto Model) and downloads a ZIP containing: | File | What it is | |---|---| | `analysis.ipynb` | The **executed** notebook: provenance header (dataset, data hash, run id), data loading, the analysis in pandas / statsmodels / scipy / scikit-learn / lifelines / arch / linearmodels with explanatory markdown, the outputs (tables, charts) already filled in, and a **verification cell** | | `analysis.pdf` / `analysis.html` | The same notebook rendered as a document — code, outputs and charts together, with the *Generated with InstaBizIntel* footer on every page | | `data.csv` | The exact data the analysis used, so the notebook runs anywhere (Jupyter, VS Code, Colab) | | `platform_results.json` | InstaBizIntel's own result tables and the list of verification checks | | `README.txt` | What is inside and how to re-run (`pip install …`, `jupyter notebook analysis.ipynb`) | **Clean-room, not a code dump.** The notebook does not export the platform's internal engine; it is a reference re-implementation with standard libraries. Its last cell recomputes the key quantities (test statistics, coefficients, effect sizes, loadings, path coefficients, R²…) and compares them with the platform's stored results, printing **VERIFIED ✓** when they agree — so the notebook doubles as an independent check of the result. Bootstrap- or seed-dependent quantities (e.g. bootstrap CIs) may differ slightly and are flagged rather than failed. **Coverage.** Every one of the 304 catalogue analyses has a **hand-written reference implementation** (statistics, econometrics, non-parametrics, effect sizes, regression family and diagnostics, design of experiments, SPC/quality, clinical and survival, reliability engineering, multivariate and classification, Bayesian, causal inference, consumer research, panel data, meta-analysis, text/topic models), plus PLS-SEM (Wold's algorithm with Mode A and path weighting, reliability/AVE/HTMT, bootstrap), the Forecasting Lab race and Auto Model. Each template was executed on sample data and compared with the InstaBizIntel engine; about nine in ten reproduce the platform's key statistics exactly, and the rest differ only by documented convention (group ordering, sign, positive-class definition, matching rules) — the verification cell shows you which. No AI model is involved in producing a notebook: generation is deterministic code, so the same run always yields the same notebook. **Integrity note.** Notebooks carry a visible *Generated with InstaBizIntel* attribution and the run id; combined with the instructor process log (§31.4) they document how a result was produced without pretending the student wrote the code by hand. ## 28. Choosing, hardening and writing up an analysis ### 28.1 Method advisor In **Rigor Guard → Method advisor**: pick the dataset, a **question type** (compare groups, repeated measures, relationship, predict, association of categoricals, time-to-event, …, or *auto*), the outcome / group / predictors. **Advise** runs the assumption tests on your actual columns (normality, variance homogeneity, sample sizes, scale types) and returns **ranked, runnable recommendations** with the exact parameters — e.g. *Kruskal-Wallis (non-normal revenue) → Welch ANOVA → one-way ANOVA*. Click **▶ Run** on a recommendation to execute it as an audit run. ### 28.2 Robustness battery **🧪 Robustness** on any run: bootstrap CIs, HC3 robust standard errors, outlier sensitivity (with/without IQR outliers), and the non-parametric counterpart, ending with a one-line **stability verdict**. ### 28.3 APA-7 write-up **📝 APA** turns the run's *tables* into an APA-7 results paragraph (never from chat). Tick *polish with AI* to improve flow while every number is locked. ### 28.4 PLS-SEM reviewer battery and interoperability Open an estimated PLS model → **Reviewer** tab → **Run battery**. *Fast* mode runs PLSpredict, CVPAT, PLSc, q²/f² effects, IPMA and CB-SEM concordance in seconds; *Full* adds Gaussian-copula endogeneity checks, NCA and FIMIX-PLS. Each procedure is reported with the reviewer-expected thresholds and a coverage score against the Hair et al. checklist. **Export / import.** From the model page choose **Export → lavaan / Mplus / seminr** to hand the model to R, Mplus or a co-author; on the PLS-SEM list page paste **lavaan syntax** (`PU =~ PU1 + PU2 + PU3`, `BI ~ PU + ATT`) and **Create model from syntax**. ### 28.5 Forecast auto-ensemble with conformal intervals **Forecasting Lab → 🎯 Ensemble**: the model pool is back-tested over rolling origins, an inverse-RMSE top-3 ensemble is built, and a **split-conformal** prediction interval is reported with its *empirical coverage* (e.g. 0.92 for a nominal 0.90). Tick *include foundation models* to add Chronos/TimesFM (slower). ### 28.6 Morning briefs **Data → Alerts → Briefs**: pick a dataset, date column, metrics, dimensions, period (day/week/month) and schedule. Each run compares the latest period with the previous one and an 8-period baseline (z-score anomaly flags), decomposes the change by each dimension (key drivers), and writes a narrative — delivered in-app, by email and by webhook. **👁 Preview now** shows the brief immediately; **🔊 Listen** reads it aloud (voice service, or the browser's speech engine as fallback) — handy on a phone. ### 28.7 Admin: AI providers, visitors and the accuracy harness - **Admin → AI providers**: add API keys for any provider (**InstaRoute**, Anthropic, OpenAI, Azure OpenAI, Google Gemini, Groq, Mistral, DeepSeek, Together, OpenRouter, xAI, Perplexity, OpusMax, local Ollama…) — keys are encrypted at rest; set one as **Active** for the whole platform and **Test** it. The active platform provider overrides any older per-user setting. - **InstaRoute (auto-routing · best value)** is an OpenAI-compatible routing gateway: instead of pinning a fixed model you choose a **routing goal** as the model — `auto` (best value), `cheapest`, `quality` or `local_first` — and InstaRoute picks the most efficient, most economical model for each request across every connected provider (you bring your own keys, billed at list price with no markup). Integration is a one-line change: point the base URL at the gateway, use an `sk-inf-…` key, and request `model="auto"`. Set it as the Active platform provider (or choose it in **Settings → AI model provider** for a single workspace) to make auto-routing the default; all other fixed-model providers remain available. - **Admin → Visitors today**: click the number for a day-wise, per-user activity table (visualisations, AI analyses, statistics, uploads…) including **anonymous** visitors (session, pages, device). - **Admin → Evaluations**: **Run benchmark** builds a set of questions with known answers from a dataset, asks the AI analyst, and scores it. Results are public at `/api/eval-badge` for a README badge. --- ## 29. Causal, evidence synthesis and qualitative v2 ### 29.1 Causal "why" with a DAG (Rigor Guard → 🧭 Causal) Mainstream "why did X change" tools rank correlations. Here you state (or discover) a causal graph and the platform estimates the effect properly, then tries to break it. 1. Choose **Treatment / cause** and **Outcome**. 2. Add **edges** (cause → effect) — e.g. `quantity → discount`, `quantity → profit`, `discount → revenue`, `revenue → profit`. Or open **Discover a skeleton**, tick variables in causal/temporal order and click **🔍 Discover edges** (partial-correlation / Fisher-z tests) — then confirm the edges. 3. **⚖️ Estimate & refute.** **Expected output:** - *Causal effect table* — naive vs **backdoor-adjusted ATE** with 95 % CI and p; the adjustment set (parents of the treatment) is derived from the DAG; **mediators are excluded automatically** (adjusting for them would block the effect). - *Refutation tests* — placebo treatment (effect must vanish), random common cause (must be unchanged), subsample stability, **E-value** (how strong an unmeasured confounder would need to be), naive-vs-adjusted shift — with a **Robust / Partially robust / Fragile** verdict. - *What-if contrasts* — predicted outcome at low vs high treatment. A cyclic graph is rejected with a clear message; the estimate is stored as an audit run. ### 29.2 Living reviews (Lit Review + Meta → Search tab) Type the query you want to monitor, choose a schedule (weekly / daily / monthly / twice a month), optionally an e-mail, and **+ Monitor**. Each run re-searches the sources, **diffs against everything already seen** (DOI/title keys) and surfaces **only new records**, raising an alert in the Alerts feed. **▶ Run now** shows the new set with PRISMA counts; the list shows records tracked, runs, last/next run. ### 29.3 Dual-LLM extraction with adjudication (Review tab) After **⬆ Upload paper PDFs**, click **🧪 Extract from uploaded PDFs**. Two independent extractor personas read each paper (design, N, population, intervention, comparator, outcome, effect type, effect, CI, p). Cells they agree on are accepted; **disagreements go to an adjudication queue** where you pick A or B — with the **page number** where the value occurs. Agreed effects feed the **Meta-analysis** tab; an **AI-use disclosure** sentence is generated for your Methods. ### 29.4 Qualitative v2 (qual workbench → Queries) - **Reliability** — choose two coders or two origins (e.g. *Manual coding* vs *All AI-assisted*, or *Topic model* vs *Sentiment*). **Expected output:** per-code % agreement, **Cohen's κ**, **Krippendorff's α**, Landis-Koch label, plus **semantic similarity / Jaccard** of the code sets — report both, because κ punishes near-synonymous AI codes. - **Saturation** — cumulative distinct codes per document in coding order, with the document at which ≤ 1 new code appeared over 3 consecutive documents (or "not yet saturated"). - **Joint display** — pick a numeric column of the same dataset (rating, spend, score). **Expected output:** a GRAMMS-style table — each theme's mean on the measure vs the rest, Welch t, Cohen's d, and a **Convergent / Divergent** integration verdict — plus a meta-inference sentence. - **Export REFI-QDA (.qdpx)** (left panel) — the project exchange format that opens in **NVivo, ATLAS.ti, MAXQDA and QDA Miner**, with codebook, documents and every coded selection. --- ## 30. From hypotheses to a submission-ready manuscript All of these live in **Rigor Guard** (🏆 Hypothesis tournament and 🚦 Publication gate cards) and in **Teaching & Thesis**. ### 30.1 Hypothesis tournament Type a topic (a dataset selected at the top supplies real variable names), choose how many hypotheses, keep *search literature* on, **🏆 Run tournament** (~1 minute). The platform **generates → critiques (hostile reviewer) → ranks → refines**. Each hypothesis shows IV/DV, the operationalisation, the suggested analysis mapped to a **catalogue key** you can run directly, novelty / feasibility / rigor scores, reviewer weaknesses, controls to add, and citations **limited to the retrieved literature and verified against Crossref**. ### 30.2 Publication gate 1. Paste manuscript text and references; tick the analysis runs that back the paper. 2. **🏷 Type claims** — every sentence is classified *data-derived / literature-derived / interpretation*; interpretations (the least reliable class of AI-written claims) are called out with an action. 3. **🚦 Reviewer gate** — one report: citation verification, numeric consistency, overclaims, claim balance, checklist coverage → **✅ READY / ⚠️ READY WITH REVISIONS / ❌ NOT READY** with the blockers listed (e.g. *a reference could not be found — possible fabrication*; *a reported p-value contradicts the test statistic*). 4. **📄 Methods section** — generated from the **execution log** of the selected runs (data, preregistration status, each procedure with its parameters and verification), never from chat; optional AI polish keeps every fact. 5. **⬇ APA Word (.docx)** — Method + Results with APA paragraphs and numbered tables (Table 1, 2 … with *Note.* lines) for the selected runs. 6. **Compliance statements** — **🤖 AI-use disclosure** (the 9 GAMER items, a UGC-India band and a CRediT-style contribution line, built from the workspace's actual AI-use log), **🗂 NIH DMS plan** (the 2026 structured six-element format, filled from your dataset metadata; tick *data contain PII* for the restricted variant), **🔗 Availability statements** (data + code, citing the bundle ids and repository/DOI). ### 30.3 InstaGenie: plan-first autonomy, queue, personas, languages, memory, Deep Research - **⚡ Auto / 🗺 Plan-first** (bottom bar). In plan-first mode InstaGenie shows its tool plan and waits; click **✅ Approve & run** or **Cancel**. Your choice is remembered. - **Prompt queue.** While a run is in progress, type the next request and press Enter or **Queue** — it runs automatically when the current one finishes (the counter shows how many are queued). - **Persona** (top bar): **Analyst** (runs and interprets), **Advisor** (states the research question, justifies the method, discusses assumptions, limitations and reviewer objections), **Socratic tutor** (runs the analysis, then guides the student with questions and hints before confirming), **Explainer** (plain language with analogies, statistics in a "for the record" line). - **Language** (top bar): Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Bahasa Indonesia/Melayu, Arabic, Spanish, Portuguese, French, German, Chinese, Japanese, Turkish, Vietnamese, English. All prose is written in the chosen language; **numbers, symbols and column names stay exactly as produced**. The same setting applies to the full-screen AI Analyst and the thesis tools. - **🧠 Memory.** Facts, preferences and glossary entries that every future session respects ("fiscal year starts in April", "always report Hedges' g", "CI = continuance intention"). Add, click-to-edit or delete entries; InstaGenie can also save entries itself when you say *remember that …*. - **🔬 Deep** (next to the input). Type a goal ("find what drives profit so we can decide where to cut discounts") and click **🔬 Deep**. InstaGenie proposes questions, the deterministic advisor runs each as an audit run, then each finding is rated for **relevance to the goal** and **practical significance**. The memo reports **only** findings that are statistically supported *and* relevant; "significant but irrelevant / trivial" findings are listed as excluded — the trap autonomous research agents fall into. ### 30.4 Dashboard version history Every **save** of a dashboard snapshots the previous state (last 25). Click **History** in the dashboard toolbar to see each version with its timestamp and a diff against the current layout (widgets added / removed / moved), and **Restore** any of them — restoring is itself versioned, so nothing is lost. --- ## 31. Teaching & Thesis Open **Research → Teaching & Thesis**. Select a dataset (for mapping/screening) and an output language at the top. ### 31.1 Validated research-model templates Choose a template — **TAM, UTAUT, UTAUT2, TPB, SERVQUAL, ECM, DeLone & McLean IS Success** — and enter the `[system]` / `[firm]` name. Expand the template to read the constructs, definitions, cited item banks and hypothesised paths. - **⬇ Questionnaire (.docx)** — a ready survey instrument with instructions, demographics, an attention check, the numbered items on the template's scale and a **codebook** (`PU1`, `PU2` …) telling you exactly how to name the columns. - **🔍 Check column mapping** — shows which constructs were found in the dataset (items are matched by construct-code prefix: `PU1`, `pu_2`, `PEOU3`) and which are missing. - **✨ Build PLS-SEM model** — creates the measurement + structural model with the template's paths, ready to estimate; a link opens it in PLS-SEM. Reverse-scored items (marked *(R)*) must be recoded in Transforms first. ### 31.2 Careless-response screen **Run screen** auto-detects Likert item columns and computes, per respondent: **longstring** (longest run of identical answers), **IRV** (intra-individual variability), **Mahalanobis D²** (multivariate outlier, χ² p < .001), **even–odd consistency** (per construct halves; needs ≥ 3 constructs) and, if a completion-time column is given, **speeding**. Respondents with ≥ 2 flags are recommended for exclusion, 1 flag for review; the note gives the exact thresholds to cite in Methods (Meade & Craig, 2012; Curran, 2016). ### 31.3 PLS-SEM sample-size calculator Enter the smallest path coefficient you expect, α, power and the maximum arrows into a construct (optionally a target R² and predictor count). **Expected output:** the **inverse-square-root** and **gamma-exponential** minimums (Kock & Hadaya, 2018), the 10-times rule (flagged as a heuristic reviewers no longer accept alone), regression power via the noncentral F, and a recommended target with a 20 % buffer for careless responses. ### 31.4 Instructor process log Every upload, transform, AI prompt, analysis, verification and locked preregistration by a member — in order, with sittings, span, **AI-initiated share** and integrity notes (e.g. "more than 80 % of actions were AI-initiated"; "all work in a single sitting"). Students **View log** / **⬇ Word** their own to submit with coursework; workspace **admins** can select any member. Optional *since* date. ### 31.5 Thesis pipeline 1. **Proposal** — type the topic and context; with a template selected, the proposal (background, gap, RQs, directional hypotheses, conceptual model, design, sampling & sample size, instrument, analysis plan, ethics, timeline) is written in your chosen language, citing **only** literature retrieved live. 2. **Preregister** — opens Rigor Guard (§27.7). 3. **Instrument** — the template questionnaire (§31.1). 4. **Analyse** — PLS-SEM / Statistics; every run becomes an audit run. 5. **Chapter 4 draft** — tick the runs, paste your hypotheses; the chapter (introduction, screening, descriptives, hypothesis testing with a supported/not-supported decision per hypothesis, summary table) is assembled from the runs' APA paragraphs and tables — numbers are copied, never invented. **⬇ Chapter 4 (.docx)** exports it. 6. **Viva Q&A** — 12 likely examiner questions with **model answers that quote the student's actual numbers** and a follow-up probe for weak answers (generic questions if no runs are selected). ### 31.6 Data tools for teaching and privacy - **Synthesize** (dataset page): creates a **synthetic twin** with a Gaussian copula — same marginals and correlations, no real rows. The report gives per-column fidelity (KS / TVD), pairwise-correlation fidelity and **privacy checks** (exact-match count, distance-to-closest-record ratio). Use it for classes, demos and sharing. - **Import OSI** (dataset → Semantic tab): paste an **Open Semantic Interchange** / dbt-metrics / LookML-style YAML or JSON; measures are validated, dimension descriptions and synonyms merged, unknown columns reported. **Preview** before **Import & merge**. - **⚡ Instant baseline** (ML Workflows): in seconds, dummy vs regularised linear vs fast gradient boosting under 3-fold CV — the score any real model must beat — with a **target-leakage screen** (near-perfect single-column associations, deterministic targets). --- ## 32. Using InstaBizIntel from ChatGPT, Claude and other AI clients (MCP) InstaBizIntel ships an **MCP server** so any Model Context Protocol client can drive the platform — ChatGPT (Connectors / deep research), Claude.ai, Claude Desktop, Claude Code, Cursor, Codex and others. It reads the live analysis catalogue, so new procedures appear automatically as tools; the feature tools below are updated with every release. ### 32.1 Connect 1. Ask your administrator for the MCP URL (`https:///mcp`) and the shared secret. 2. **ChatGPT / Claude.ai** — add a custom connector with that URL; authentication *Bearer* with the secret (or append `?key=…`). 3. **Claude Desktop / Claude Code / Cursor** — add to the client's MCP config: `{"mcpServers": {"instabizintel": {"type": "http", "url": "https:///mcp", "headers": {"Authorization": "Bearer "}}}}` 4. Check `https:///mcp/health` and the agent card at `https:///mcp/.well-known/agent.json`. Each user can also mint a personal API token in-app (`POST /api/auth/api-token`) for scripts and the CLI (`python mcp/ibi_cli.py`). ### 32.2 Tools (35) | Group | Tools | |---|---| | Orientation | `whoami`, `list_datasets`, `describe_dataset`, `list_analyses`, `describe_analysis`, `search`, `fetch` | | Run & trust | `run_analysis`, `ask` (InstaGenie end-to-end), `list_runs`, `verify_run`, `bundle_link`, `robustness`, `apa_writeup`, `method_advisor` | | Forecast & briefs | `forecast`, `explain_change` | | PLS-SEM | `list_pls_models`, `run_pls_sem`, `pls_reviewer_battery` | | Causal & hypotheses | `causal_effect`, `causal_discover`, `hypothesis_tournament` | | Publishing | `reviewer_gate`, `methods_section`, `compliance_statement`, `export_docx_link`, `living_review` | | Research & teaching | `deep_research`, `research_templates`, `careless_screen`, `pls_sample_size`, `agent_memory` | | Data | `instant_baseline`, `synthesize_dataset` | **Example prompts (in ChatGPT or Claude):** *"List my datasets, then run the method advisor for comparing profit across regions and execute the top recommendation."* · *"Estimate the causal effect of discount on profit with quantity as a confounder and revenue as a mediator, and refute it."* · *"Run the reviewer gate on this paragraph with these references."* · *"Generate the NIH DMS plan for the survey dataset."* Every result returns with its run id and Trust Score; cite the run id in your paper's supplement. --- ## 33. Appearance and accessibility Use the **Light / Dark** control at the bottom of the sidebar. Both themes are now fully readable: table header bands, hovered rows, status pills (Trust, Premium, confirmatory/exploratory), notes and error boxes all use contrast-checked colours in light mode. A quick visual check of every UI idiom is available at `/theme-check` (no login required). --- ## 34. Finance Lab — investment analytics Open **Research → Finance Lab**. It brings the capabilities popularised by open-source financial LLM projects (news sentiment, news-augmented forecasting, filing question-answering) together with classical investment analytics, on live public data, inside the same audit trail as the rest of the platform. Every output carries a disclaimer: this is research tooling, not investment advice. **Common inputs (top card).** Start typing two or more letters of a company, index, ETF, currency pair or coin name in the **Tickers** box and pick from the live suggestions (symbol, name, exchange); each pick becomes a chip and you can keep adding, remove with ×, or paste a comma-separated list. Tickers use Yahoo Finance symbols (`RELIANCE.NS`, `TCS.NS`, `AAPL`, `BTC-USD`, `EURUSD=X`). Choose the **benchmark** from the grouped list (Nifty 50, Sensex, Bank Nifty, S&P 500, NASDAQ, FTSE 100, DAX, Nikkei, Hang Seng, Bitcoin, gold, US 10-year and more) or search for any other index or ETF. Then set the date range and the risk-free rate. Prices come from Yahoo Finance with a stooq fallback. **Save prices as dataset** stores the panel (date, symbol, close, return) as a workspace dataset so Statistics, Forecasting Lab, Charts and InstaGenie can use it. **On every result: interpret, download, add to report.** Under each tab's results an action bar offers **✨ Interpret with AI** (a 250–400-word analyst reading of the actual numbers: key takeaways, what the numbers say, watch-outs, next steps), **Download** as **PDF**, **Word**, **Excel** (one sheet per table plus a summary sheet), **CSV (zip)** or **Charts PNG** (exports include the charts, tables and the interpretation), and **➕ Add to report**. The **Research note** tab's **Report builder** collects everything you added, downloads the complete report as PDF / Word / Excel, or opens it in the **Reports module** (§13) as an editable, brandable document with live charts. Filing Q&A answers can be added to the report and downloaded the same way. ### 34.1 Performance & risk Total return, CAGR, annualised volatility, Sharpe, Sortino, maximum drawdown, Calmar, CAPM beta and alpha, correlation with the benchmark, historical VaR and CVaR (95%), skewness, kurtosis, best/worst day, tracking error and information ratio, plus a correlation matrix. Charts: growth of 1 unit, drawdown, rolling 63-day volatility, correlation heatmap. ### 34.2 Portfolio Long-only mean-variance construction on daily returns: **maximum Sharpe**, **minimum variance**, **risk parity**, **equal weight** or **maximum return at a target volatility**, with a per-asset weight cap. Outputs the weights with expected return, volatility and risk contribution, the efficient frontier, and an in-sample back-test with monthly, quarterly or annual rebalancing against equal weight and the benchmark. ### 34.3 Technicals SMA 20/50/200, RSI(14), MACD(12, 26, 9), Bollinger %B, distance from the 52-week high and low, ATR and a trend classification per ticker, with price/moving-average and MACD/RSI charts for the first ticker. ### 34.4 News & sentiment Type a company or topic. Headlines are pulled from Google News RSS (with GDELT as fallback), de-duplicated and scored with the **finance lexicon** (VADER extended with market vocabulary such as *upgrade, beat, plunge, buyback, probe, impairment*), or with the **AI analyst**, which adds a one-line rationale per headline (FinBERT is offered automatically if the server has the model). You get the scored headline table, a daily sentiment index with a 7-day average, headline volume and the positive/negative mix. ### 34.5 AI brief (FinGPT-Forecaster pattern) For one ticker the platform assembles 60 days of prices, the technical board and the last three weeks of headlines, and asks the configured AI provider for a horizon view that must be grounded in that evidence: direction, confidence, expected move range, thesis, positive and negative factors, catalysts, risks, what to watch and a stance. A damped-ETS statistical baseline with an 80% band is drawn next to it so the two views can be compared. ### 34.6 Fundamentals & DCF For US-listed tickers, annual XBRL facts are read from SEC EDGAR (revenue, operating income, net income, operating cash flow, capex, assets, liabilities, equity, cash, long-term debt, EPS) and turned into growth, margin, ROE, ROA, leverage, free-cash-flow and net-debt series with a revenue/net-income chart. The **DCF** valuer takes base FCF, a fading growth path, terminal growth, WACC, net debt and share count (pre-filled from EDGAR where available) and returns the projection, enterprise and equity value, fair value per share and a WACC × terminal-growth sensitivity heatmap. Non-US companies can be valued by typing the inputs from the annual report. ### 34.7 Filings Q&A Retrieval-augmented question answering over the latest 10-K, 10-Q, 20-F or 8-K: the filing is fetched from EDGAR, split into passages, the most relevant passages are retrieved by TF-IDF and the AI answers only from them, citing `[passage n]`. Presets cover risk factors, MD&A, business strategy and debt/liquidity; or ask your own question. The cited passages are shown under the answer. ### 34.8 Event study Market-model event study for the first ticker against the benchmark: the model is estimated on 120 trading days ending 10 days before each event, abnormal returns and CAR over [-5, +5] are reported per event with a t-test on the estimation-window residual SD, and the average CAR path is charted. Use earnings dates, policy announcements, index inclusions or M&A dates. ### 34.9 Credit risk A credit-portfolio simulator for lenders, treasury and credit-research use. Paste obligors as `name, PD %, exposure (EAD), LGD %` or take them from a dataset. Each obligor defaults as a **Bernoulli trial** with its own probability; a one-factor Gaussian copula (the Vasicek / Basel IRB model) makes defaults move together through the asset correlation ρ. Over the chosen number of scenarios the tab reports total exposure, expected loss, unexpected loss, credit VaR and expected shortfall at 95 / 99 / 99.9%, economic capital (VaR 99.9% − EL), the analytic Vasicek loss at 99.9% for comparison, loss-exceedance probabilities, the loss distribution histogram and each obligor's contribution to the 99% tail. ### 34.10 Research note Assembles everything run in the other tabs into an equity research note (executive summary, developments, price and risk, technical picture, fundamentals and valuation, portfolio context, outlook, methodology) written by the AI from the tables only, with a Word (.docx) download that appends the underlying tables. ### 34.11 From InstaGenie and the AI Analyst The agent has a `finance_lab` tool, so requests such as *"compare RELIANCE.NS, TCS.NS and INFY.NS against ^NSEI over two years and build a maximum-Sharpe portfolio"* or *"give me a five-day analyst brief on HDFCBANK.NS"* run the same engines and render the charts inline.