Pipeline Contracts (Authoritative)

This document defines behavioral invariants for the core pipeline layering.
Guides and quickstarts are non-authoritative and should defer to these contracts.

Run Status Model

  • execution_status: terminal status from execution before persistence.

  • final_status: terminal status after persistence outcomes are evaluated.

  • status: compatibility/read convenience alias and must equal final_status.

  • This run-level execution_status is distinct from modules_skipped[].execution_status in run_results.json, where the field describes module skip classification (skipped or blocked).

  • Precedence: aborted > failed > partial > succeeded.

  • Optional persistence failures may only downgrade succeeded -> partial.

  • Required persistence failures produce final_status=failed.

  • Cancellation with required persistence failure:

    • execution_status=aborted, final_status=failed, status=failed,

    • termination_reason=cancellation.

Planner and Executor Boundaries

  • Planner API: DAGPlanner.plan(requested_modules, registry_snapshot) -> ExecutionPlan.

  • Planner consumes immutable RegistrySnapshot.

  • Planner fails closed for unresolved dependencies.

  • Executor owns run-local state and outcome reduction.

  • Executor does not persist, log, or report directly.

  • Path routing rules:

    • planner cannot consume raw filesystem paths,

    • executor does not construct/normalize/route filesystem paths.

Persistence Rules

  • Required writes: canonical run outputs (run_results.json and related artifacts), artifact manifest, and run report.

  • Conditional required writes: processing state when a matching managed state entry exists. If no processing state file or matching entry exists, processing-state persistence is an optional no-op.

  • Event emission is best-effort runtime notification, not a durable persistence write.

  • Optional writes: execution plan artifact, auxiliary artifact index, non-critical snapshots.

  • Persistence transaction strategy: fail-fast per write, no rollback of already durable writes.

  • Outcomes must be surfaced via RunResult.persistence_outcomes.

Event Contract

  • Event sequence:

    • run_started,

    • per-module terminal event (module_completed|module_skipped|module_failed),

    • single terminal run event (run_completed|run_failed) emitted at most once.

  • Setup/context failures must emit run_failed.

  • Callback failures are isolated and must not corrupt run outcome.

Cleanup Guarantees

Cleanup must execute via try/finally for:

  • transcript output-dir override lifecycle,

  • runtime config override lifecycle,

  • draft override lifecycle,

  • PipelineContext.close().

Legacy analysis modules

  • Registry entries may set legacy: bool on ModuleInfo. Legacy modules are excluded from ModuleRegistry.get_default_modules(..., include_legacy=False) (the default when analysis.include_legacy_modules is False).

  • Users may still run legacy modules by explicitly naming them in the module list, or by setting analysis.include_legacy_modules=True to pull them back into default-style plans without listing IDs.

  • Default semantic analysis uses semantic_similarity. Legacy IDs semantic_similarity / semantic_similarity_advanced remain stable for outputs and backward compatibility.

Semantic similarity v2

  • Module id: semantic_similarity. Outputs use *_semantic_similarity_*.json with top-level schema_version: semantic_similarity.1.1 (major still 1 under parse_schema_major). Motif envelope fields: motifs, motif_export_status, provenance, eligible_segment_count, comparability (TF-IDF incomparable).

  • Presets: analysis.active_semantic_similarity_profile selects analysis.semantic_similarity_profiles (fast_v2, balanced_v2, deep_v2). Runtime merge: dataclass defaults → preset dict → per-field user overrides (values that differ from defaults on analysis.semantic_similarity) → when the preset omits mode, analysis.analysis_mode (quick/full) sets mode to basic/advanced.