AgentData MCP Large-File Latency — Defer Ruling

Date: 2026-05-18 Authority: User final decision + GPT synthesis

Ruling

large_file_chunk_cursor_phase_2: DEFERRED
current_policy: accept_slow_large_file_reads_temporarily
focus: continue_information_unit_and_constitution_path

The user made the final decision: temporarily accept slower reads for large KB files. Do not divert the main workstream into MCP chunk/cursor implementation now.

Reasoning

The large-file read/chunking issue is recognized as part of the fundamental Information Unit problem:

same_core_problem:
  - large content must be split into smaller units
  - units/chunks must be ordered
  - completeness must be verifiable
  - reconstruction must be possible
  - provenance/version/checksum must be preserved

However, the immediate priority is to finish the Information Unit / Constitution path first. Once the IU system is further completed, the same unit-integrity design can be reused to improve MCP/KBgateway chunking.

Operational decision

near_term:
  - continue using existing KB/MCP tools
  - accept occasional slowness on long files
  - use narrow reads, batch_read snippets, exact doc reads, and retries when needed
  - avoid making chunk/cursor API a blocker for Constitution path

later:
  - revisit chunk/cursor after IU integrity/manifest model matures
  - align MCP chunking with Unit Integrity Contract rather than treating it as a one-off technical patch

Current infrastructure state

Phase 1 route observability/timeout patch remains valid:

phase_1_status:
  GPT_route_timeout_aligned: true
  sanitized_access_log_enabled: true
  nginx_reload_success: true
  probes_fast: true

But phase 2 chunk/cursor is deferred.

Final status

status: LARGE_FILE_LATENCY_ACCEPTED_TEMPORARILY__PHASE_2_DEFERRED
next_focus: dot_iu_cutter_constitution_source_document_seed_command_review