# Import runs

## Activation gate

Same gate as [notebook-corpus.md](notebook-corpus.md) — **satisfied**; this contract is runtime-normative. Single-file ingest via `IngestService` + `.ingest-journal.json` remains a supported import path. This contract defines the bulk orchestration layer used by bulk-import UI/CLI.

## Lifecycle

```text
scan → plan → validate → commit
```

Only **commit** mutates the corpus (notebook entities, managed bytes, corpus index, ImportRun outcomes). Scan/plan/validate are read-only w.r.t. corpus authority (they may write disposable plan drafts under the ImportRun store).

### Adapters

Folder-per-notebook, naming conventions, scanner batches, PDF trees, etc. are **import adapters**. They emit one canonical `ImportPlan`. Adapter heuristics must not leak into notebook identity (`notebook_id` is always preallocated/generated—never “folder name”).

### Notebook cover (folder-per-notebook / file-name heuristic)

When creating or filling a notebook from imported image sources:

1. If a source basename is `cover.jpg`, `cover.jpeg`, or `cover.png` (case-insensitive), and the notebook has no `cover_page_id` yet, set `cover_page_id` to that source’s page (within-source `page_index` 0).
2. Otherwise leave `cover_page_id` unset; display/Open fall back to the **first page in notebook order** (`project.pages[0]`), not earliest dated page.
3. Do not overwrite an existing user-set `cover_page_id`. Cover PDFs are out of scope for this heuristic.

### Ordering ambiguity

Natural sort may be the **proposed** order in a plan. Ambiguous ordering (duplicate numbers, mixed PDF/image without an explicit rule, missing indices, conflicting cues) is a **validate error**. Commit is refused until the plan is resolved. Silent guessing is non-conformant.

## Plan operations

An `ImportPlan` is a list of items. Each item is exactly one of:

| `op` | Meaning |
|------|---------|
| `create_notebook` | Create a new managed notebook directory + `project.json`, then register it in the corpus index, optionally with initial pages/sources |
| `import_into_notebook` | Append sources/pages into an **existing** `notebook_id` already present in the corpus index / on disk |

These must not be collapsed into one ambiguous “import” op. Recovery paths differ (see Crash boundaries).

## Stable IDs and preallocation

Before commit begins, the validated plan must contain:

| ID | Rule |
|----|------|
| `plan_id` | Stable UUID for this plan document; immutable for the plan’s life |
| `item_id` | Stable UUID per plan item; immutable |
| Resulting `notebook_id` | Preallocated for `create_notebook`; fixed target for `import_into_notebook` |
| Resulting `source_id` / `page_id` / `render_id` | Preallocated for every page/source the item would create |

**Retry/idempotency must not regenerate these IDs.** A resumed commit uses the same plan bytes / same IDs. If an item already committed those IDs, recommit is a no-op for that item.

## Plan fingerprint and policy

| Field | Rule |
|-------|------|
| `import_manifest.schema_version` | Integer; `1` for this contract generation |
| `import_policy_id` | Stable string naming duplicate/ordering policy package (e.g. policies that include `skip_existing` or `create_duplicate`) |
| `plan_fingerprint` | Lowercase hex SHA-256 of the **canonical plan body** defined below |

### Canonical plan body (fingerprint input)

Compact UTF-8 JSON with sorted keys containing exactly:

- `schema_version`
- `plan_id`
- `import_policy_id`
- `items` — array in plan order; each item includes `item_id`, `op`, target/`notebook_id`, ordered intended page specs with preallocated IDs, source fingerprints (`sha256`), media type, within-source `page_index` list, and policy-relevant flags (e.g. explicit corpus-wide dedupe request). **Exclude** mutable external paths from the fingerprint, or include them only inside a nested `provenance` object that is **omitted** from the fingerprint input.

### Idempotent equality (replaces vague “same plan + policy”)

A commit attempt is an idempotent retry of a prior attempt iff **all** hold:

1. `plan_id` equal
2. `plan_fingerprint` equal
3. `import_policy_id` equal
4. Item `item_id` set equal (same multiset of items)

Otherwise it is a **different** plan and must not reuse another run’s committed item outcomes as no-ops unless an explicit “continue with new plan” product flow creates a new `ImportRun`.

## ImportRun storage

| Field | Value |
|-------|--------|
| Format | `transcribe.import-run` |
| `schema_version` | `1` |
| Location | `{TRANSCRIBE_DATA_DIR}/corpus/import-runs/<import_run_id>.json` |
| Registry / index (optional companion) | `{TRANSCRIBE_DATA_DIR}/corpus/import-runs/index.json` listing run IDs + terminal status (derived convenience; each run file is authoritative for that run) |
| Writes | Atomic replace under corpus lock (short critical section) |

### Run document shape (normative fields)

- `import_run_id`, `format`, `schema_version`
- `plan_id`, `plan_fingerprint`, `import_policy_id`, `import_manifest.schema_version`
- Immutable history: frozen plan snapshot reference or inlined canonical plan body used for fingerprint; discovery inputs summary; timestamps `created_at`
- Mutable execution state: `status`, per-item outcomes, `updated_at`, error summaries
- Per-item outcome: `item_id`, `state` (`pending` \| `committed` \| `skipped` \| `failed` \| `cancelled_pending`), resulting IDs, skip classification, failure code/message

### Immutable history vs mutable execution

- Plan identity fields and canonical plan body are **immutable** after run creation.
- Item outcome records append/advance forward only (pending → committed/skipped/failed/cancelled_pending). Committed outcomes must not be rewritten to a different ID set.
- Final `status` is written once to a terminal value (see Cancellation).

### Retention

- ImportRun files are **retained by default** (audit/resume). Deletion is an explicit operator action, not automatic GC on success.
- Optional future compaction may archive terminal runs older than a configured age; compaction must not delete the only record of committed page provenance linkage (`import_run_id` on sources) without a documented export.

## Intra-notebook ingest constraint

Existing `.ingest-journal.json` permits **at most one active ingest transaction per notebook**. Bulk orchestration **must enforce** this: never start a second source commit against a notebook that has a live journal; never assume intra-notebook parallelism. Parallelism across notebooks is allowed only under [notebook-corpus.md](notebook-corpus.md) lock rules.

## Cancellation

- Cancellation stops **pending** items; it **never rolls back** already committed pages/sources/notebooks.
- Terminal statuses:

| Status | Meaning |
|--------|---------|
| `complete` | All items committed or intentionally skipped per policy; none failed |
| `partial` | At least one committed and at least one failed (and/or cancelled pending), not a clean full success |
| `failed` | No items committed; one or more failed (or planning/commit aborted before any commit) |
| `cancelled` | Operator cancel with **no** items committed (clean no-op cancellation) |
| `cancelled_with_commits` | Operator cancel after one or more items already committed |

UI/CLI must not collapse `cancelled` and `cancelled_with_commits`.

## Crash / recovery boundaries

At each boundary, recovery picks an **authoritative winner** and is resume-safe.

| Boundary | After success looks like | Authoritative winner on restart |
|----------|--------------------------|--------------------------------|
| 1. Corpus registration (create) | Corpus index entry present for `notebook_id` | If `project.json` exists with that `id` but index lacks entry → **register** (complete registration). If index entry exists but `project.json` missing → **error/quarantine** (do not invent notebook). |
| 2. Notebook creation | `project.json` exists with preallocated `notebook_id` | Winner is on-disk `project.json`. Incomplete create without valid manifest → roll back directory only if never registered; if registered, doctor error. |
| 3. Source promotion | Managed source file present + matches planned `sha256` | File + hash win; journal continues toward manifest commit. |
| 4. Render promotion | Render PNG present + matches planned render hash | File + hash win. |
| 5. `project.json` commit | Pages/sources/renders include preallocated IDs | Manifest wins; clear per-notebook ingest journal only after manifest matches journal intent. |
| 6. ImportRun item commit | Item outcome `committed`/`skipped` with IDs | ImportRun item record wins for idempotency; do not recreate. |
| 7. Final run-state commit | Terminal `status` set | Terminal status wins; do not reopen terminal runs except via explicit new plan. |

**Create vs register race:** If `project.json` was created with the preallocated `notebook_id` but corpus registration did not commit, recovery **must** complete registration (append index entry) rather than creating a second notebook or deleting the valid project. If registration exists without a loadable project, report integrity error—do not delete the index entry silently.

## Journals: corrupt / malformed

- Per-notebook `.ingest-journal.json` and ImportRun execution journals/scratch must not be **silently discarded** when unreadable or schema-invalid.
- Required behaviour: **report** (doctor/CLI error) and **quarantine** (e.g. rename to `.ingest-journal.corrupt.<timestamp>` / move under `corpus/quarantine/`) leaving an audit trail.
- Automatic delete of unreadable journals is **non-conformant** for bulk-import generation safety. Recovery may roll back only when the journal is well-formed and indicates a non-`manifest_pending` incomplete state per [project-on-disk.md](project-on-disk.md).

## Relation to per-notebook ingest

ImportRun orchestrates many notebook-local commits. Each page/source commit still uses the existing stage → journal → promote → atomic `project.json` replace mechanism. ImportRun does not replace that journal; it records outcomes around it.

## Non-goals

- One filesystem transaction for an entire multi-thousand-page run
- Silent path-based resume when planned IDs are absent
- Offering `replace_source` in v1 policy IDs