dot-iu-cutter v0.1 — Operational Problem Statement rev5d

Status: READY_FOR_USER_REVIEW_AFTER_THREAD_RETRIEVAL_HYGIENE Version: rev5d (thread retrieval + user interaction + hygiene) Date: 2026-05-15 Scope: User review of problem statement only. No Agent design, no implementation. Purpose: Define the complete operating problem for "Cắt luật A" before assigning any design work. Revision history: rev5 (GPT) → rev5a (Opus PG-first) → rev5b (Opus semantic threading) → rev5c (Opus + GPT legal alignment + canonicalization) → rev5d (Opus thread-first retrieval + user interaction + GPT hygiene/access-control patch)

0. Control Status

problem_statement_status: READY_FOR_USER_REVIEW_AFTER_THREAD_RETRIEVAL_HYGIENE
authoritative_revision: rev5d
previous_status_blocks: historical_only
agent_design_allowed: false_until_user_approval
implementation_allowed: false
scope: approve_problem_statement_only
assembly_axes_added: true
uosl_dieu44_added: true
tac_runtime_reality_acknowledged: true
pg_driven_assembly_required: true
dot_pair_requirement_added: true
positive_recursion_required: true
decision_backlog_registry_required: true
cross_temporal_semantic_threading_added: true
semantic_intake_flow_defined: true
industry_standards_referenced: true
user_directed_threads_required: true
dieu37_in_canonical_foundations: true
dieu44_guardrail_controlled_draft: true
dieu24_no_parallel_taxonomy: true
dieu39_universal_edges_first: true
dieu32_risk_approval_binding: true
dieu33_43_pg_location_explicit: true
birth_gate_distinguish_base_draft_runtime: true
reviewer_notes_added: true
legal_alignment_patch: completed
canonicalization_completed: true
thread_first_retrieval_added: true
business_facing_search_readiness_added: true
thread_context_pack_required: true
retrieval_metrics_required: true
search_gap_feedback_required: true
audience_scoped_search_required: true
access_control_guardrail_added: true
thread_retrieval_hygiene_completed: true

This document is the operating problem statement. It is not an implementation design and must not be treated as approval to build.

1. Core Goal

The goal is not merely to split a document into pieces.

The goal is to build an operating process for intelligent information infrastructure where data must become: correct, complete, clean, living, and increasingly intelligent over time.

Information units are a foundation for Text-as-Code, Knowledge Graph, semantic assembly, governance, review, correction, and future automation. Therefore, dot-iu-cutter must not be designed as a one-time text cutter. It must be designed as a lifecycle:

cut → use → detect mismatch → correct → learn from new capabilities → upgrade policy/tooling → cut better next time

This is the principle of positive recursion: a good system must absorb its own improvements and use them to produce a smarter next generation of itself.

2. User-Facing Problem Statement

When the user says "Cắt luật A", the system must run a closed lifecycle:

Resolve source → Check existing cut / collision / history → Mark semantic manifest under C1A → Review manifest independently → Cut deterministically from PASSed manifest → Verify round-trip / no-loss / no-overlap / invariants → Report result + rollback keys → Observe actual usage → Detect segmentation health issues → Review correction options → Apply Split / Merge / Edge / Context-Pack / NoAction → Verify again → Report again → Intake Text-as-Code / Knowledge Graph progress → Produce impact diff or no-impact record → Upgrade cutter policy when approved

The central questions: Where to cut? Why? Who decided? Who reviewed? How is correctness proven? How is mismatch detected? How are corrections handled? How does the cutter absorb TAC/KG improvements? How does it support assembly along multiple axes?

3. Canonical Foundations

3.1 C1A — Canonical Segmentation Law

Mandatory source: knowledge/dev/laws/dieu38-trien-khai/C1A-segmentation-operating-model.md (OFFICIAL — User PASS / GPT PASS).

The design must inherit, not rewrite: 3-question test (clear title? independently editable? not too hard to edit?), SR-1→SR-7, OD-PILOT-01→06, NL1→NL4, length management, CI-1→CI-12, controlled vocabulary discipline, no mechanical A/B/C splitting, one unit = one canonical parent, every new unit must pass birth gate.

3.2 Điều 38 / Text-as-Code

The cutter must dogfood Text-as-Code: manifest as code, policy as code, diffable review, versioned decisions, validation gates, rollback keys, release/readiness states, semantic lint when available.

3.3 Điều 39 / Knowledge Graph

The cutter must not be separated from KG. KG capabilities must feed back over time: edge-aware marking, semantic neighborhood retrieval, co-citation/co-edit/co-retrieval analytics, orphan/overlap detection, impact analysis before split/merge, edge reassignment suggestions, context-pack generation.

If KG is not mature enough, the cutter must still emit metadata hooks: semantic_role, source_span, parent/child, candidate_edges, edge_readiness_notes, review_flags, body_source_policy.

3.4 Điều 43 and Intelligent Infrastructure

Điều 38, 39, 43, and this cutter are parts of intelligent infrastructure, not isolated tools. The design must treat dot-iu-cutter as a governance and assembly component.

3.5 Điều 44 / UOSL — Object Schema Foundation (Compatibility Target)

Điều 44 / UOSL is the current controlled DRAFT (v0.1.2, not yet enacted) and serves as the compatibility target for object schema design.

The design must align with UOSL concepts where applicable: OQC, G1–G12 field groups, Family Registry, Relation Layer / universal_edges compatibility, maturity M0–M4, IU profile expectations.

Guardrail: UOSL is a controlled draft. Do not treat draft fields as final enacted schema. Flag unmapped fields as schema gaps, not silent assumptions.

3.6 Điều 37 / Governance Organization Law

Owner, reviewer, escalation queue, council/AI roles, governance queue, and registry responsibility for cutter and semantic threading must map to Điều 37. If no valid mapping exists, flag a governance gap. No parallel governance organization.

4. Runtime Reality to Respect

The system already has real TAC/IU implementation. The cutter design must not pretend it starts from zero.

Known constraints: existing TAC/IU PG schema (tac_logical_unit, tac_unit_version, tac_publication), proven round-trip reconstruction on prior pilots, existing fn_iu_create gateway and birth system, hierarchy/profile in JSON profile fields.

Design must be compatible with existing PG-first architecture. Schema additions require separate migration design, risk review, rollback plan, and acceptance tests.

5. Two Initial Assembly Axes

5.1 Axis 1 — Document Reconstruction

Reconstruct the original document exactly. Requires: source_span, render_order, parent_unit_id, canonical_address, source_revision, publication membership, body_source_policy.

Verification: render units by publication/render order → compare with source → 0 drift = PASS.

5.2 Axis 2 — Semantic Domain Assembly

Assemble information across document types by professional/semantic domain (law, design, requirements, code, reports, reviews, runbooks, operational records).

Requires: section_type, unit_kind, classification labels, semantic_role, candidate_edges, edge_readiness_notes, canonical_address, universal_edges compatibility, vector projection readiness.

5.3 Both Axes Are First-Class

A segmentation decision is incomplete if it supports round-trip reconstruction but destroys later semantic assembly.

6. Three Executor-Facing Flows

Flow 1 — Cut: Resolve → Mark → Review → Cut → Verify → Report

Flow 2 — Health / Correction: Observe → Detect → Review → Split/Merge/Edge/Thread/Context-Pack/NoAction → Verify → Report

Flow 3 — Upgrade: New capability → Impact diff → Review → Patch policy/tool or NoImpact → Report

If a future design cannot be reduced to these three flows, it is too complex.

7. Questions the Design Must Answer (Q1–Q56)

A. Source and User Command

Q1. How does the user command work? System must resolve source automatically. Q2. What is the source of truth? Clarify KB markdown, TAC source, publication/snapshot, authority constraints. Q3. What if the source was already cut? Collision/history check mandatory.

B. Marking / Segmentation Decision

Q4. What is MARK output? Must be a manifest with minimum fields: source_path, source_revision, source_span_start/end, title, section_type, unit_kind, parent, hierarchy_depth, body_source_policy, C1A_rule_refs, three_question_test_result, cut_reason, confidence, length_flag, edge_readiness_notes, review_flags. Q5. Which rules determine boundaries? C1A: 3-question test, SR rules, OD-PILOT, length as trigger, no mechanical splitting. Q6. How are section_type and unit_kind chosen? No invented vocabulary. Missing vocabulary flagged to governance. Q7. What is heading/container/body policy?

C. Review / Authority

Q8. Who reviews the manifest? Default: independent AI review. Human only for ambiguity, new vocabulary, suspected data loss, legal/governance change, high-risk. Q9. What does REVIEW check? no-loss, no-overlap, source coverage, C1A compliance, semantic cohesion, title clarity, section_type/unit_kind/hierarchy/body policy/length flag validity, canonical parent, birth gate readiness. Q10. When does the system ask the human? Only when policy cannot decide safely.

D. Cut / Verify / Rollback

Q11. How does CUT execute? Only from PASSed manifest. Q12. How is correctness proven? Round-trip verification mandatory. Q13. What happens if verification fails? Automatic rollback, no partial publish. Q14. How does rollback work? Rollback keys, change-set, exact-key rollback.

E. Split / Merge / Edge / Context-Pack

Q15. How does Split work? New units, old superseded, history preserved, spans/roles mapped, edges reassigned, verify again. Q16. How does Merge work? New unit, old superseded, aliases/redirects, edges reassigned, verify again. Q17. When add edge/context-pack instead of merge? When units remain independently meaningful but often travel together.

F. Operability

Q18. Can this run from one command? Yes for normal low-risk cases. Q19. Which state machine? Three flows: Cut, Health/Correction, Upgrade. Q20. Does this apply beyond laws? Yes, with different unit_kind/section_type. Q21. How are decisions/manifests/reviews/reports persisted? PG-native (rows/JSONB in TAC schema). Versioned, diffable, reviewable. Markdown is documentation mirror, not authority.

G. Post-Cut Usage Feedback

Q22. How are poor segmentation decisions detected? Track: co_citation, co_edit, co_retrieval, edge_density_overlap, context_pack_dependency, orphan_or_underused_unit, misclassification_signal. Q23. Are these signals available now? Classify each as: available_now / requires_instrumentation / future_capability. Q24. Is there a Segmentation Health Report? Yes. Triggered by N new units, Y days, M events, strong coupling, or complaint. Q25. How is split/merge after usage evidence reviewed? Detect → evidence bundle → AI proposal → independent review → apply → verify → report.

H. Text-as-Code / KG Upgrade Loop

Q26. How does cutter absorb TAC progress? Capability Intake with: capability_name, source_law, status, impact, required_change, risk, owner, decision. Q27. How does cutter absorb KG progress? KG Capability Intake. Q28. How does cutter avoid becoming stale? Cutter Self-Review triggered by milestones, release cycles, patterns, complaints.

I. Decision Backlog Registry

Q29. How avoid forgetting decisions? Decision Backlog Registry — PG table SSOT (directus DB, Lớp KHO). KB markdown = mirror only. Minimum fields: decision_id, date_discussed, summary, status, dependencies, next_review_date, owner, source_discussion, related_law_or_design, risk. Q30. When is the registry swept? Every governance review, health report, upgrade review.

J. Assembly Axes

Q31. Does each unit have enough metadata for document reconstruction? render_order, parent, canonical_address, source_span, publication membership, source_revision. Q32. Does each unit have enough metadata for semantic domain assembly? section_type, unit_kind, labels, semantic_role, candidate_edges, edge_readiness_notes, universal_edges compatibility. Q33. How are both axes verified? Axis 1: round-trip. Axis 2: semantic assembly test cases.

K. TAC Runtime Compatibility

Q34. Is the cutter compatible with current TAC PG schema and runtime? Must check before proposing changes.

L. Constitutional / Law Compatibility

Q35. Does the design comply with Assembly First and PG-first? Output must be PG-driven IU rows and relations. Q36. Does the cutter require DOT pairs? Must define or justify why not.

M. Cross-Temporal Semantic Threading

Q37. What is a Semantic Thread? Define per Đ39 + Đ44, not a parallel graph. Q38. How does the system thread new units into old threads across weeks/months? Semantic intake flow. Q39. Where are candidate edges/thread memberships stored? PG-first. Compatible with universal_edges or flag gap. Q40. What evidence supports link proposals? section_type, label, semantic_role, citation, code symbol, embedding, co-citation, user direction. Q41. How are wrong links detected? Retrieval noise, reviewer rejection, usage removal, contradiction, low confidence, health signals. Q42. How are missing links detected? High similarity, repeated co-retrieval, lifecycle expected chain missing, reports cite without edge. Q43. Where are issues routed? Decision Backlog Registry / Health Report / governance queue. No new notification system. Q44. Do threads have split/merge lifecycle? Yes. Q45. Owner/reviewer/escalation mapping? Đ37 + Đ32 for approval/risk. Q46. Does each thread have an expected lifecycle chain, and how is a missing link detected?

N. Thread Retrieval and User Interaction

Q47. How does AI/Agent use semantic threads to find information instead of raw vector search? Q48. How does the system resolve a query to the right thread? Q49. What does a Thread Context Pack contain, and in what order? Q50. When does the system use thread-first retrieval vs fallback to vector search? Q51. Does thread retrieval result include source/revision/authority/risk/health status? Q52. When a thread is missing, wrong, or noisy, what signal is created? Q53. How does business-facing search differ from agent-facing search? Q54. Where are audience/visibility/readiness/publication filters applied? Q55. What can employees/partners/customers see within a thread? Q56. How does search usage feed back into semantic health and the enrichment loop?

8. Mandatory Design Principles (P1–P15)

P1. C1A is law; cutter is operation. Do not rewrite segmentation law. P2. Decision separate from execution. MARK/REVIEW decides; CUT executes PASSed manifest only. P3. Manifest as Code. Versioned, diffable, reviewable, traceable. P4. Round-trip verification mandatory. P5. Split/Merge is lifecycle, not manual repair. Supersession, aliases, redirects, edge reassignment. P6. Usage evidence is governance evidence. P7. Prefer graph enrichment before structural change. Do not merge merely because units appear together. P8. Cutter must dogfood Text-as-Code. P9. Cutter must feed from Knowledge Graph. P10. Positive recursion. Every improvement feeds the next generation. P11. Anti-forgetting through registry. P12. Simple at operation layer. Three flows only. P13. Two assembly axes are mandatory. Document reconstruction + semantic domain assembly. P14. PG-driven assembly. PG is SSOT and runtime authority for assembly, manifest, registry, health. Markdown = documentation layer. P15. UOSL compatibility. Map to Đ44 or flag schema gaps.

9. Acceptance Criteria (1–50)

A future design must be rejected if it does not satisfy ALL:

Inherits C1A explicitly.
Defines three flows: Cut, Health/Correction, Upgrade.
Defines manifest as code.
Defines independent review before cut.
Requires round-trip verification.
Defines rollback.
Defines split/merge lifecycle.
Supports edge/context-pack/no-action, not auto-merge.
Includes post-cut usage review.
Includes Segmentation Health Report.
Classifies detection signals as available_now / requires_instrumentation / future_capability.
Includes Text-as-Code Capability Intake.
Includes KG Capability Intake.
Includes Cutter Self-Review.
Includes Decision Backlog Registry.
Produces impact diff / no-impact record for new capabilities.
Avoids hardcoding to current infrastructure state.
Dogfoods Text-as-Code.
Provides KG feedback hooks.
States clearly what infrastructure is missing.
Defines human escalation for authority boundaries.
Prevents loss of discussed decisions.
Supports document reconstruction axis.
Supports semantic domain assembly axis.
References Đ44 / UOSL and maps profile/edge concepts.
Acknowledges current TAC/IU PG runtime and does not redesign from zero.
Enforces PG-driven assembly, manifest persistence, and registry storage.
Addresses DOT-pair / dual-engine verification requirement.
Separates schema gaps from implementation tasks.
Produces enough metadata for future cross-document professional-domain assembly.
Supports cross-temporal semantic threading.
Defines semantic intake flow for new units/objects.
Defines candidate edge/thread membership lifecycle (candidate → accepted/rejected → stale).
Stores negative knowledge for rejected links.
Defines missing/wrong link detection signals.
Defines thread split/merge lifecycle.
Maps governance roles to Điều 37.
Uses existing governance channels, does not create parallel notification system.
Leverages industry standards (SKOS, entity linking, co-citation, W3C PROV, CDC) — does not reinvent.
Supports both system-discovered and user-directed threads as first-class inputs.
Each TAC/KG progress must feed back into semantic intake quality (positive recursion).
Defines thread-first retrieval for AI/Agent use.
Defines thread resolution from user/agent query.
Defines Thread Context Pack structure.
Defines fallback strategy from thread retrieval to vector search.
Defines retrieval quality metrics.
Defines search-gap feedback signals.
Defines audience-scoped search for employee/partner/customer/business-facing views.
Defines visibility/readiness/publication filters.
Ensures search usage feeds back into semantic health and positive recursion.

10. Expected Deliverables After User Approval

Only after user approves this problem statement:

Operational Design — dot-iu-cutter-v0.1-operational-design.md
Manifest and Operator Contract — dot-iu-cutter-v0.1-manifest-and-operator-contract.md
Segmentation Health / Usage Feedback Design — dot-iu-cutter-v0.1-segmentation-health-design.md
Capability Intake and Upgrade Loop Design — dot-iu-cutter-v0.1-capability-intake-design.md
Decision Backlog Registry Design — dot-iu-cutter-v0.1-decision-backlog-registry-design.md
Assembly Axes and Metadata Contract — dot-iu-cutter-v0.1-assembly-axes-metadata-contract.md
UOSL / Đ44 Compatibility Note — dot-iu-cutter-v0.1-uosl-compatibility-note.md
Design Report — dot-iu-cutter-v0.1-design-report.md
Cross-Temporal Semantic Threading Design — dot-iu-cutter-v0.1-cross-temporal-semantic-threading-design.md
Legal Alignment / Governance Compatibility Report — dot-iu-cutter-v0.1-legal-alignment-governance-compatibility-report.md
Thread Retrieval and User Interaction Design — dot-iu-cutter-v0.1-thread-retrieval-user-interaction-design.md

11. Forbidden Scope Before User Approval

no_agent_dispatch, no_implementation, no_code_change, no_pg_mutation, no_migration, no_vector_work, no_qdrant_work, no_event_outbox, no_ui_cutover.

12. Cross-Temporal Semantic Threading

12.1 Problem

Metadata alone is not operational capability. The system must self-propose and verify professional-domain links between units created at different times, in different documents — without depending on human memory.

12.2 Semantic Thread (Chuỗi chuyên môn)

A living professional-domain axis connecting units across documents, object types, and time. Example: birth_gate thread connects HP principles → Đ0-G → C1 lifecycle → C1A segmentation → Đ44 schema → fn_iu_create code → test report → incident report.

Properties: lives in PG/KG (not a document), compatible with universal_edges (Đ39) and Đ44/UOSL, two formation sources (system-discovered + user-directed), both converge into same governance lifecycle.

12.3 Industry Standards — Not Reinventing

Problem	Standard/Tool	Incomex Application
Concept organization	SKOS (W3C)	Thread hierarchy, relationships
Auto topic discovery	BERTopic / embedding clustering	Candidate thread discovery
Entity linking	Mention → canonical entity	Map IU content → thread membership
Citation analysis	Co-citation (bibliometrics)	Co-occurrence detection
Data provenance	W3C PROV	Evidence trail for thread membership
Change detection	CDC (PG trigger + NOTIFY/LISTEN)	Trigger semantic intake
Semantic search	Vector similarity (Qdrant)	Candidate discovery
Relation query	PG recursive CTE + universal_edges	Thread traversal, impact analysis

Principle: PG-native maximum (tsvector, pg_trgm, triggers, materialized views, NOTIFY/LISTEN). Qdrant for vectors. No new stack unless gap proven.

12.4 PG-First Objects (Problem Statement Level)

Object	Role	Notes
semantic_thread	Living domain/topic	Target DB per Đ33/Đ43 rules
semantic_thread_membership	Approved: unit X in thread Y	Use universal_edges if sufficient
semantic_thread_candidate	Proposed, not reviewed, with confidence	PG table with status lifecycle
semantic_thread_evidence	Evidence for proposals	PG JSONB or structured
semantic_thread_health_signal	Missing/wrong/stale/overbroad/narrow signals	Route to Decision Backlog

Before proposing separate tables, first evaluate whether universal_edges + edge status + evidence payload suffices.

12.5 Semantic Intake Flow

New unit/object → Extract signals (section_type, keywords via tsvector, embedding similarity, labels — available_now; referenced laws, co-citation/co-edit — requires_instrumentation; lifecycle chain — future_capability) → Compare with existing threads → Propose candidates with confidence → Store in PG → Auto-accept only if low-risk + 2+ independent signals + no negative knowledge conflict + policy allows; otherwise candidate_for_review → Notify governance queue if high-risk.

12.6 Semantic Enrichment Loop

New data → intake → candidates → evidence → AI/policy review → accepted enrich graph, rejected become negative knowledge → future matching learns from history → TAC progress improves extraction → KG progress improves discovery → positive recursion.

12.7 Thread Membership Lifecycle

candidate → [review] → accepted / rejected / needs_human. accepted → [usage] → stale / superseded. rejected → [new evidence] → re-candidate if materially different.

Each membership: status, confidence, evidence_bundle (JSONB), reviewed_by, reviewed_at.

12.8 Detecting Missing/Wrong Links (extends Flow 2)

Flow 2 expanded: Observe → Detect → Review → Split/Merge/Edge/Thread/Context-Pack/NoAction → Verify → Report

Missing: high similarity unlinked, co-retrieval without edge, lifecycle chain gap. Wrong: reviewer rejection, retrieval noise, contradiction, semantic_role mismatch. Thread anomaly: overbroad (>50 units), too narrow (<3), stale (no activity > N days).

12.9 User-Directed Threads

User can: create threads, assign memberships, reject AI suggestions, request discovery, merge/split threads. User direction = first-class input, equal to AI discovery. System must not silently override user decisions but may flag inconsistencies.

13. Reviewer Notes / Design Constraints

13.1 Design Constraints for Semantic Threading

Thread ≠ Edge. Thread is living concept/axis containing memberships, edges, evidence, signals. Edge is specific typed relation. Do not flatten threads into edge type.
SKOS conceptual only. No RDF/SPARQL/triple-store in v0.1. Map concepts to PG + universal_edges.
W3C PROV as evidence discipline. Minimum fields: evidence_source, evidence_method, generated_by, generated_at, source_object_id, target_thread_or_object, confidence, review_status, reviewer, decision_reason.
User-directed = authoritative intent, not automatic graph truth. Memberships carry provenance (accepted_by_user/policy/ai, candidate_by_ai, rejected_by_user/review). Health signals still apply. AI may flag but not silently override.
Auto-accept risk-gated. Only when: low-risk, 2+ independent signals, no negative knowledge conflict, no governance/legal/code-impact flag, policy explicitly allows.
Expected lifecycle chain required as hook. Example: law → design → code → test → report. Missing link = health signal expected_artifact_missing. v0.1 may use JSONB field on thread. Hook must exist from day one.
PG + Qdrant + universal_edges first. No Neo4j, graph DB, triple store, or ML pipeline unless gap proven.
Negative knowledge suppresses repeats. Rejected links persist. Re-proposal only with materially different evidence.
Thread health signals minimum set: missing_link, wrong_link, stale, overbroad, too_narrow, noisy_retrieval, expected_artifact_missing.
Thread split/merge preserves provenance. Old → superseded with reference. Memberships migrated with trail. Aliases maintained. Decision in Backlog Registry.

13.2 Legal Alignment / Conflict Prevention

Đ44/UOSL: Controlled draft, compatibility target, not enacted law.
Đ37: Already in Section 3.6. All governance roles must map.
Birth Gate: Distinguish base Đ0-G (enacted), L4 IU extension (draft), current runtime. Flag gaps.
Đ24 Label Law: No parallel taxonomy. SKOS = conceptual model only. Operational labels → Đ24 or flag vocabulary gap.
Đ39/universal_edges: Evaluate universal_edges first. Separate tables only if lifecycle/evidence/health cannot be represented. No parallel graph authority.
Đ33/Đ43 PG location: Final design must explicitly choose target DB and layer per object. No unresolved "tùy thiết kế."
Đ32 approval/risk: Memberships affecting legal/governance interpretation, code-impact, approval, or publication must follow Đ32. Cannot accept by confidence score or user direction alone.

13.3 PG-First Data Structure Requirements

Data Structure	SSOT	Mirror
Decision Backlog Registry	PG table (directus DB, Lớp KHO)	KB markdown (report only)
Cut Manifest	PG rows in TAC schema	KB report for human review
Capability Intake records	PG table (directus DB)	KB doc for governance
Health metrics / detection signals	PG views/queries	Dashboard / generated report

14. Thread-First Retrieval and User Interaction Scenarios

14.1 AI/Agent Thread-First Retrieval

This is the primary use case. AI/Agent should not default to raw vector search when structured thread knowledge exists.

Flow:

Query (user or agent)
→ Thread Resolution: match query to candidate threads by keyword, label, embedding
→ Candidate Thread Ranking: rank by relevance, recency, authority, health status
→ Context Pack Build: assemble canonical units from best thread
→ Authority/Recency/Source Ordering: sort by lifecycle stage, then recency, then authority
→ Missing/Suspect Signal Attach: flag known gaps, suspect links, stale units
→ Answer with citations: each claim traces to source unit + revision
→ Fallback: if no reliable thread found or thread confidence < threshold → vector search

Thread-first retrieval turns semantic threads into a retrieval primitive — not just metadata, but the primary way AI finds structured domain knowledge.

14.2 Business / Audience-Scoped Search

Different audiences see different slices of the same thread:

Audience	Sees	Does not see
AI/Agent (internal, authorized)	Material allowed by role, task scope, tool permission, and data classification	Secrets, unauthorized drafts, restricted incidents, out-of-scope data, materials blocked by approval/risk policy
Employee	Approved SOP, runbook, enacted policy	Draft designs, code internals, incident raw
Partner	Published guides, API docs, integration specs	Internal governance, incidents, drafts
Customer	FAQ, public policy, product docs	Everything internal

Access-control guardrail: Audience-scoped search is an access-control requirement, not merely a retrieval filter. wrong_audience_result must be treated as a security/governance issue, not only a search quality signal.

Flow:

Query
→ Identify audience/scope (from auth context or explicit parameter)
→ Apply visibility/readiness/publication filters
→ Resolve thread/process/document view for that audience
→ Return only approved, published, audience-appropriate units
→ Hide/redact internal-only material
→ Log search_gap if filtered result is weak or empty

v0.1 boundary: v0.1 prepares metadata, visibility, readiness, publication, and audience hooks. It does not approve building external-facing UI/search.

14.3 Thread Context Pack

When AI/Agent requests a thread, the system returns a Thread Context Pack — a pre-assembled, ordered bundle of relevant units.

Minimum fields:

thread_id
thread_title
purpose
canonical_units (ordered list of core units)
supporting_units (related but secondary)
latest_reports (most recent reports/reviews in this domain)
related_design_code_links (if available)
known_gaps (expected artifacts missing)
suspect_links (low-confidence or flagged memberships)
last_health_status (latest health check result)
authority_or_risk_level
recommended_reading_order
source_revisions (revision of each unit included)

Agent should NOT read the entire thread raw. Agent receives a structured context pack, already ordered and annotated.

14.4 Retrieval Quality Metrics

To know if thread-first retrieval actually works, measure:

thread_hit_rate — % of queries resolved by thread (vs fallback)
fallback_vector_rate — % of queries requiring vector search fallback
context_sufficiency_rate — % of queries where context pack was sufficient (no extra search needed)
wrong_thread_rate — % of queries matched to wrong thread
missing_thread_rate — % of queries where no thread existed but should have
noisy_context_rate — % of context packs containing irrelevant units
agent_extra_search_rate — % of times agent searched outside thread after receiving pack
user_reject_rate — % of thread suggestions rejected by user

Initial mature target: >80% thread-first resolution. Strategic ambition: >90% for recurring AI/Agent domain queries after enough threads and instrumentation exist.

These metrics require instrumentation (requires_instrumentation status). v0.1 design must define the metrics and leave hooks; implementation follows when retrieval layer is built.

14.5 Search Gap Feedback

When search fails or returns weak results, the system must create feedback signals:

search_gap — query had no good result from any source
missing_thread — query implies a domain that has no thread yet
weak_thread — thread exists but too few units or low confidence
noisy_thread — thread matched but units were irrelevant
wrong_audience_result — internal content leaked to wrong audience scope
expected_artifact_missing — thread lifecycle chain has gaps

These signals route to: Semantic Health Report, Decision Backlog Registry, governance queue. No new notification system.

Search gap feedback completes the positive recursion loop: usage → gap detection → thread enrichment → better retrieval next time.

15. Final One-Line Summary

dot-iu-cutter v0.1 is not a text splitter. It is the first operating mechanism for living information units that can be reconstructed as original documents, assembled across professional domains, corrected from usage evidence, and improved recursively as Text-as-Code and Knowledge Graph capabilities mature.