Theme A — Insights & analysis quality

Status: done for wave (deterministic + presentation closeout + deferred charts/B18 MVP); deepen-in-place continues under ROADMAP §A
Last updated: 2026-10-19

Roadmap home: docs/ROADMAP.md §A
Product constraint: Prefer deepen-in-place; label Deterministic vs Local AI honestly; do not keep weak deterministic fallbacks merely to claim non-AI coverage (PRODUCT.md).

Goal

Stronger deterministic / hybrid insights: clearer, more useful non-LLM outputs with less noise; trustworthy empty/partial states; hybrid LLM paths inherit a cleaner deterministic base.

Locked decisions

  1. Shared signal first — raise quality in phrase_quality + insight_eligibility before inventing new insight detectors.

  2. Insights themes use theme_label_policy — same bar as summary theme labels (stricter than content_phrase_policy).

  3. Abstain over pad — if fewer than min_themes_for_signal themes pass the floor, emit empty themes + status: insufficient_signal rather than weak fillers.

  4. Schema v3 — each theme/idea row may carry evidence_quote_ids, confidence, preference_tier, and score breakdown; bump schema_version to 3.

  5. Topic soft boost only — topic_modeling may boost an existing theme when labels overlap; it never invents themes. Dependency remains soft (missing topics = no boost).

  6. Overview focus honesty — deterministic overview omits a focus clause when no high-tier theme labels survive; no fallback to emblematic filler phrases for focus text.

  7. Hybrid inherits base — narrative_summary continues to rewrite only the deterministic summary findings; improving summary improves hybrid without prompt inventiveness.

  8. B18 LLM insight narratives — optional insights_narrative finalize module (gated by analysis.insights.narrative_enabled); Local AI only, never replacing deterministic theme lists; uses P2-lite evidence_quote_ids + highlight quote text.

Signal rules

Layer

Rule

Phrase hard rejects

Empty, all-stopwords, discourse formulas, light-verb constructions, tic/discourse mask, pronoun shards, no content token, short shards

Eligibility floor

Default min_score=0.18; single-token candidates require spread > 0 or recurrence > 0

Insights compose

Diversity via select_diverse_themes; caps from analysis.insights config

Confidence bands

high if total≥0.55 and (spread≥0.15 or recurrence≥0.2); medium if total≥0.35; else low (low rows may be dropped by floor)

Schema v3 sketch

{
  "schema_version": 3,
  "status": "ok | insufficient_signal",
  "status_reason": null,
  "key_themes": [
    {
      "phrase": "budget risk",
      "score": {"total": 0.72, "spread": 0.3, "recurrence": 0.4},
      "confidence": "high",
      "preference_tier": 0,
      "evidence_quote_ids": ["q:..."],
      "topic_corroborated": false
    }
  ],
  "recurring_ideas": [],
  "style_markers": {},
  "notable_moments": [],
  "phrase_quality_version": 3
}

Empty-state matrix (UI)

Condition

User-facing copy

Insights module not run / failed

Existing quiet unavailable / module-required hint

status == insufficient_signal (content focus)

“Not enough clear content themes in this transcript.” (info only — no sparse theme list chrome)

Eligibility / highlights missing for content block

“Content themes need Insight eligibility and Highlights in this run.”

Style markers empty

“No style indicators.” (unchanged)

Deterministic summary abstains on focus

Overview still shows speaker count / duration; no invented focus clause

Config knobs (analysis.insights)

Field

Default

Role

top_themes

8

Cap for key themes

top_recurring_ideas

8

Cap for recurring ideas

top_notable_moments

8

Cap for notable moments

min_theme_score

0.18

Compose-time floor (aligned with eligibility)

min_themes_for_signal

2

Below this → abstain

overview_theme_cap

5

Suggested UI cap on Overview

topic_boost

0.05

Soft score boost when topic label overlaps

narrative_enabled

false

Allow insights_narrative finalize module to run when selected

narrative_min_evidence_quotes

1

Minimum themes with resolved highlight quote text before B18 narrative

narrative_max_chars

1200

Cap on Local AI theme narrative length

Eligibility: analysis.insight_eligibility.min_score default 0.18; require_spread_or_recurrence_for_singletons default true.

B18 operator path (insights_narrative)

  1. Run insights (needs insight_eligibility + highlights) so schema-v3 rows include evidence_quote_ids.

  2. Set analysis.insights.narrative_enabled to true (Settings → Configuration or profile).

  3. Include module insights_narrative in the run (not in the default recommended list; finalize-phase, requires Ollama).

  4. In Insights → Themes and ideas, deterministic lists stay canonical; optional Theme narrative appears below with a Local AI badge when the artifact exists. Theme rows may show a one-line Evidence caption when quote text resolves.

Does not replace llm_summary or Overview summary hero prose.

Acceptance checklist

  • Banned fillers never appear in insights.key_themes or summary.key_themes (unit + smoke)

  • Abstention path covered when only weak phrases exist

  • Evidence quote ids present when highlights themes attach

  • Deterministic overview omits filler focus

  • Commitments require content beyond light-verb stems

  • Insights UI hides empty/abstention sections instead of sparse tables

  • PHRASE_QUALITY_VERSION bumped when hard-reject semantics change (3 — bump only when hard-reject semantics change again)

  • Corpus before/after notes for 2–3 English meetings — standing evidence in Corpus probe notes below (mini + large fixture probes, 2026-08-10); optional owner-machine meeting notes still welcome

Out of scope / follow-on

  • Multilingual phrase-resource expansion (Theme M / known limitation)

  • SQLite analytics (Theme J)

  • Reintroducing Insights → Analysis tab

  • Full shared P2 provenance platform for B10/B11 (B18 MVP uses P2-lite on insights rows only)

  • Per-chart manual Charts catalogue pass (registry matrix + monologue overview honesty landed 2026-10; see overview_presentation_0_9_9.md P2)

Corpus probe notes (deep-test 2026-08-10)

Run

Transcript

Insights status

Notable themes

mini Python

tests/fixtures/mini_transcript.json

ok

google cloud support, new york stakeholders, market street

large Theme A modules

tests/fixtures/analysis_probes/large_norm.json (1007 segs / 13 speakers)

ok after log-freq + greeting rejects (was insufficient_signal with linear freq + min_score 0.28)

kitchen fairy, cape town, south africa, social media

group

mini + large

member runs green; group finalize partial with manifest/run_results present

—

Hardening during deep-test: log1p frequency scoring; greetings/et cetera discourse rejects; eligibility/insights floor 0.18; emblematic-phrase fallback before abstention; topic_modeling optional so insights stays on quick/balanced.