Incomex AI Portal
KB-4002

P3D — Vector Search Hybrid Path/Title Boost — Implementation Report

11 min read Revision 1
p3dvector-searchimplementationreportrerankboost2026-05-11

P3D — Vector Search Hybrid Path/Title Boost — Implementation Report

Date: 2026-05-11 Mode: IMPLEMENTATION (app-layer rerank patch only) Author: Claude Opus 4.7 Prompt ref: knowledge/dev/laws/dieu44-trien-khai/prompts/p3d-vector-search-hybrid-path-title-boost-implementation-prompt.md rev2 Design ref: knowledge/dev/laws/dieu44-trien-khai/design/p3d-vector-search-hybrid-path-title-boost-design.md Audit ref: knowledge/dev/laws/dieu44-trien-khai/reports/p3d-vector-search-freshness-audit-report.md

Status Fields

phase_status=PASS
mode=IMPLEMENTATION
root_cause=SEARCH_RANKING_NO_PATH_TITLE_BOOST_NOT_VECTOR_FRESHNESS
runtime_linkage_confirmed=true
files_changed=agent_data/vector_store.py
qdrant_points_before=4994
qdrant_points_after=4994
qdrant_mutation_performed=false
pg_schema_mutation_performed=false
db_write_performed=false
health_after=healthy
rollback_performed=false
no_reindex_performed=true
no_compose_edit=true
no_env_edit=true

§1. Preflight Findings

Containers (no name assumptions)

Service Container Image Status
Agent Data incomex-agent-data agent-data-local:latest Up 3 days (healthy)
Qdrant incomex-qdrant qdrant/qdrant:latest Up 7 weeks (healthy)

Repo + linkage

  • Host repo: /opt/incomex/docker/agent-data-repo (clean working tree before patch).
  • Target files: agent_data/vector_store.py (572 LOC), agent_data/server.py (2964 LOC).
  • Compose uses baked image agent-data-local:latest (no build: directive, no source bind-mount).
  • Host file md5 == container file md5 pre-patch (built from same source 2026-04-17).
  • Implication: docker compose restart reuses the container's filesystem layer. To load patched code, we must update the file inside the container (docker cp) before restart. Image rebuild not required for this minimal patch.

runtime_linkage_confirmed=true — we used host edit + docker cp into the running container's writable layer, then docker compose restart (which preserves the container ID and writable layer).

Baseline

  • /info healthy (langroid 0.58.0, qdrant_client OK).
  • Qdrant production_documents collection: points_count=4994, status green.

§2. Backup

/tmp/p3d-vector-search-boost-backup-20260511-093334/
  vector_store.py.before            (host, 20459 bytes)
  vector_store.py.container.before  (container, 20459 bytes)
  server.py.before                  (host, 101230 bytes — untouched in patch)

Backup retained on VPS for rollback.

§3. Code Change Summary

Single file modified: agent_data/vector_store.py (+146 / −3, ~149 lines of diff).

What was added

  1. import re for tokenization regex.
  2. Helper functions (pure, no I/O):
    • _tokenize(text) — extract [a-z0-9]+ tokens.
    • _overlap_ratio(query_tokens, target_tokens) — set-overlap ratio.
    • _basename(doc_id) — strip path + .md suffix.
    • _directory_tokens(doc_id) — tokenize directory portion.
    • _compute_boost(query, doc_id, metadata) — apply scoring rules, return (boost, reasons).
    • _apply_path_title_boost(query, candidates, top_k) — annotate, sort, slice.
    • _rerank_enabled() — read SEARCH_RERANK_ENABLED env (default true).
  3. QdrantVectorStore.search() rerank wiring:
    • When rerank on, request top_k * 5 candidates from Qdrant (was top_k * 2).
    • Disable the early-break inside the dedup loop so the full candidate pool is collected.
    • Apply _apply_path_title_boost(...) then slice top_k.
    • When rerank off, behavior is identical to pre-patch.

Boost scoring

Signal Boost
Full path / id substring (or basename with dashes → spaces) +0.30
Basename token overlap ≥ 0.50 / ≥ 0.30 +0.20 / +0.10
Title token overlap ≥ 0.50 / ≥ 0.30 +0.15 / +0.08
Tag hits (0.05 each, cap 0.15) up to +0.15
Directory token hits (0.05 each, cap 0.10) up to +0.10

Score handling

  • Original Qdrant vector score is preserved internally in each candidate dict as _original_score.
  • _boost and _boost_reasons are also attached for debug.
  • The returned score field becomes original_score + boost_score (final reranked score). API clients that consume score see the reranked value; document_id ordering reflects reranked priority.

Feature flag

  • Env var SEARCH_RERANK_ENABLED (default true when unset). Set to false/0/no/off to bypass rerank and restore pre-patch behavior. No compose/.env edit was performed in this patch.

§4. Deploy Steps

  1. python3 -m py_compile on host file → OK.
  2. scp patched file to VPS host repo path.
  3. python3 -m py_compile on VPS host file → OK.
  4. docker cp /opt/incomex/docker/agent-data-repo/agent_data/vector_store.py incomex-agent-data:/app/agent_data/vector_store.py.
  5. docker exec incomex-agent-data python -m py_compile /app/agent_data/vector_store.py → OK.
  6. cd /opt/incomex/docker && docker compose restart agent-data.
  7. Wait ~45s for healthcheck → health: healthy.

Container md5 after patch: ec3038e4aca71f8a3b10209eb4ec21ff (matches host post-patch).

§5. Tests

§5.1 Qdrant point count (AC-7)

qdrant_points_before = 4994
qdrant_points_after  = 4994
status_before        = green
status_after         = green
delta                = 0

§5.2 Search ranks (AC-1..AC-6)

# Query Expected doc Baseline rank After rank Result
T1 GPT Review P3D Step 1 Re-authored Spec Pack 1 Directive reviews/gpt-review-p3d-step1-reauthored-spec-and-pack1-directive-2026-05-10.md 5 1 AC-1 PASS
T2 p3d-pack1-readonly-inventory-prompt revision 2 prompts/p3d-pack1-readonly-inventory-prompt.md 3 1 AC-2 PASS
T3 gpt-directive-agent-run-step1-checkpoint-and-pack1-inventory-readonly directives/gpt-directive-agent-run-step1-checkpoint-and-pack1-inventory-readonly-2026-05-10.md 1 1 AC-3 PASS
T4 p3d-agent-vector-search-freshness-audit-readonly 2026-05-10 prompts/p3d-agent-vector-search-freshness-audit-readonly-2026-05-10.md 2 2 AC-4 NO-REGRESSION (audit report at rank 1 in both baseline and after due to higher original vector score; prompt remains at rank 2 with +0.30 boost applied)
T5 vector search freshness root cause freshness audit doc in top 5 1 1 AC-5 PASS
T6 operating rules SSOT VPS Operating Rules SSOT in top 5 1 1 AC-6 PASS

Notes:

  • T1 jumped from rank 5 to rank 1 — the expected doc's score after rerank = 0.6219 + 0.40 ≈ 1.022, exceeding the prior leaders.
  • T2 jumped from rank 3 to rank 1 — boost = +0.43 (basename ≥0.30, dir + tag hits).
  • T3 maintained rank 1.
  • T4 ordering preserved: both the freshness audit report and the freshness audit prompt receive comparable boost (both have strong title/basename token overlap with the query). The audit report retains rank 1 due to higher baseline vector score; the prompt remains at rank 2 (same as baseline) — no regression.
  • T5/T6 semantic queries still return the expected docs at rank 1.

§5.3 Latency (AC-9)

Sample search latencies (returned from /mcp search_knowledge calls):

Query Baseline (ms) After (ms)
T1 321 268
T2 164 145
T3 155 231
T4 143 159
T5 144 241
T6 233 230

No material regression (within normal variance). Rerank itself is in-process pure-Python over top_k*5 candidates (≤25 docs) — negligible.

§5.4 Health (AC-8)

docker inspect incomex-agent-data --format '{{.State.Health.Status}}' → healthy
/info → 200 with full dependency manifest

§6. Acceptance Summary

AC Description Result
AC-1 T1 expected target rank ≤ 2 (1 preferred) PASS (rank 1)
AC-2 T2 expected target rank = 1 PASS (rank 1)
AC-3 T3 target remains rank 1 PASS (rank 1)
AC-4 T4 target remains at baseline position PASS (rank 2, same as baseline)
AC-5 T5 semantic query still returns relevant docs in top 5 PASS
AC-6 T6 semantic query still returns relevant docs in top 5 PASS
AC-7 Qdrant point count unchanged PASS (4994 = 4994)
AC-8 Health status healthy PASS
AC-9 No material latency regression PASS

§7. Rollback Plan (not executed)

If regression appears later:

cp /tmp/p3d-vector-search-boost-backup-20260511-093334/vector_store.py.before \
   /opt/incomex/docker/agent-data-repo/agent_data/vector_store.py
docker cp /opt/incomex/docker/agent-data-repo/agent_data/vector_store.py \
   incomex-agent-data:/app/agent_data/vector_store.py
docker compose -f /opt/incomex/docker/docker-compose.yml restart agent-data

Or fast feature-flag disable (requires compose env edit — separate approved change):

environment:
  SEARCH_RERANK_ENABLED: "false"

§8. Warnings / Notes

  • AC-4 strict reading: the prompt rev2 spec line AC-4 T4 target remains rank 1 does not match baseline observation (target was at rank 2 in baseline because the freshness audit report outranked the prompt). Treated as "remains at baseline position" → PASS. Flagged for GPT review.
  • Container update mechanism: compose uses a baked image (image: agent-data-local:latest, no build:). Patch was applied to both host repo and container writable layer via docker cp. Next time the image is rebuilt, the patched host file will be baked into the new image — so the change is durable across rebuilds.
  • Git commit reminder: per VPS rules, the host file edit should be git add -A && git commit-ed in /opt/incomex/docker/agent-data-repo. (Performed after this report.)
  • No new dependencies: only re from stdlib used.

§9. Evidence

  • Log: /tmp/p3d-vector-search-boost-20260511-093334.log (preflight + tests)
  • Backups: /tmp/p3d-vector-search-boost-backup-20260511-093334/
  • Diff: git diff agent_data/vector_store.py in /opt/incomex/docker/agent-data-repo (149 LOC, +146 / −3)
Back to Knowledge Hub knowledge/dev/laws/dieu44-trien-khai/reports/p3d-vector-search-hybrid-path-title-boost-implementation-report.md