Phase 2F-D2a — MCP JSON/SSE Content Negotiation — Implementation Report

Date: 2026-05-14 Status: PASS — deployed to VPS, committed. Commit: 34ca724 on main of /opt/incomex/docker/agent-data-repo. Design reference: knowledge/current-state/reports/phase-2f-c-rev2-mcp-production-patch-design-2026-05-14.md S3 Prior phase: D1 19315ce (list_documents SQL pushdown) — regression intact.

---

1. Code changes summary

Single-file change: agent_data/server.py (+207 / -28 lines).

New helpers

Function	Purpose
_parse_accept(header)	Parses an Accept header into (wants_json, wants_sse, q_json, q_sse, any_valid_media). Tolerant of */*, application/*, text/*, bare *, missing q-values, malformed input. Never raises.
_wants_sse(request)	Returns True iff the client explicitly prefers SSE over JSON for this response. Tie-breaks to JSON when q-values are equal.
_accept_is_unsupported(request)	Returns True iff the Accept advertises one or more real media types yet none are application/json or text/event-stream. Malformed Accept that yields no valid media at all is NOT treated as unsupported — it falls back to JSON.
_mcp_respond(request, payload, status_code=200)	Serialises a JSON-RPC envelope as either JSONResponse (default) or a single-event SSE Response with Content-Type: text/event-stream; charset=utf-8, Cache-Control: no-cache, no-transform, X-Accel-Buffering: no.

Endpoints touched

mcp_jsonrpc (/mcp) and _mcp_filtered_handler (used by /mcp-readonly, /mcp-gpt, /mcp-gpt-full): every return site that delivers a JSON-RPC envelope now routes through _mcp_respond(request, payload). The two exceptions kept JSON:

• API-key 401 — kept as plain JSONResponse to avoid leaking SSE framing to misconfigured clients before authentication succeeds.
• notifications/* — kept as Response(status_code=204) regardless of Accept (no body to deliver under either media type, per spec).

GET /mcp (info endpoint) — added an Accept guard: if the client sends Accept: text/event-stream and does NOT also accept JSON, return 405 Method Not Allowed with Allow: POST and a JSON-RPC error envelope. Otherwise unchanged.

Unsupported-media 406

Each POST handler additionally returns 406 Not Acceptable if Accept advertises any real media type yet none are JSON or SSE (e.g. application/xml). Malformed Accept like ;;;malformed falls back to JSON, not 406.

2. Exact Accept negotiation rule

parse Accept
  if absent / empty                 -> wants_json=true (default)
  if no valid media token at all    -> wants_json=true (default; treat as malformed)
  for each "type/subtype" token with q > 0:
    application/json | */* | application/*  -> wants_json, q_json
    text/event-stream | text/*              -> wants_sse,  q_sse
decision
  if neither wanted, but any valid media token seen -> HTTP 406
  if not wants_sse                              -> JSON
  if wants_sse and not wants_json               -> SSE
  if both, q_sse > q_json                       -> SSE
  if both, q_json >= q_sse                      -> JSON  (tie-break)

This rule keeps every existing client byte-identical:

• curl -H 'Accept: application/json' -> JSON OK
• FastMCP / Claude.ai sending Accept: application/json, text/event-stream -> JSON (tie-break) OK
• A client that strictly wants SSE (Accept: text/event-stream) -> SSE OK
• A client weighting SSE higher (Accept: application/json;q=0.5, text/event-stream;q=1) -> SSE OK

3. SSE response format

A request that resolves to SSE receives exactly one event, then the connection closes:

HTTP/1.1 200 OK
Content-Type: text/event-stream; charset=utf-8
Cache-Control: no-cache, no-transform
X-Accel-Buffering: no

event: message
data: <single-line JSON-RPC envelope>
\n

No keep-alive pings, no session-id (Mcp-Session-Id), no Last-Event-ID. Those are D2b scope. For request/response operations, one event is sufficient.

4. Internal Accept matrix — test results

24 / 24 PASS, run from inside incomex-agent-data against http://127.0.0.1:8000.

#	Request	Expected	Result
U1	Accept: application/json POST /mcp-gpt tools/list	200 JSON	PASS
U2	Accept: application/json, text/event-stream	200 JSON (tie-break)	PASS
U3	Accept: text/event-stream	200 SSE w/ event: message frame	PASS
U4	Accept: application/json;q=0.5, text/event-stream;q=1	200 SSE	PASS
U5	Accept: application/json;q=1, text/event-stream;q=0.5	200 JSON	PASS
U6	no Accept	200 JSON	PASS
U7	Accept: ;;;malformed	200 JSON fallback	PASS
U8	Accept: */*	200 JSON	PASS
U9	Accept: application/xml	406	PASS
U10a-d	tools/list count: /mcp=11, /mcp-readonly=5, /mcp-gpt=8, /mcp-gpt-full=11	exact counts	PASS x4
U11	notifications/initialized w/ JSON Accept	204, empty body	PASS
U12	notifications/initialized w/ SSE Accept	204, empty body	PASS
U13	malformed JSON body w/ JSON Accept	JSON-RPC -32700 parse error	PASS
U14	malformed JSON body w/ SSE Accept	SSE-framed parse error	PASS
U15	tools/call list_documents over SSE	valid frame, items returned	PASS
U16	GET /mcp w/ SSE Accept	405 + Allow: POST	PASS
U17	GET /mcp w/ JSON Accept	200 JSON info	PASS
U18	GET /mcp no Accept	200 JSON info	PASS
U19	/mcp-gpt-full SSE Accept	200 SSE	PASS
U20	/mcp internal SSE Accept	200 SSE	PASS
U21	SSE response headers	Cache-Control: no-cache, no-transform, X-Accel-Buffering: no	PASS

5. Public route sanity (HTTPS via nginx)

Tested through the existing nginx /gpt-mcp/{token}/mcp location (which proxies to /mcp-gpt). The location already had proxy_buffering off; chunked_transfer_encoding on; proxy_cache off; — no nginx change required.

Case	Result
HTTPS POST Accept: application/json	200 application/json; charset=utf-8, 8 tools
HTTPS POST Accept: text/event-stream	200 text/event-stream; charset=utf-8, valid event: message + data: frame, 8 tools inside
HTTPS POST Accept: application/json, text/event-stream	200 application/json; charset=utf-8 (tie-break)

Secret token never printed.

6. FastMCP client — before / after

FastMCP 2.10.5 was already present in the container. The client uses Streamable HTTP transport and sends Accept: application/json, text/event-stream (tie-break case -> JSON in our rule).

	Result
initialize	OK
list_tools	OK — 8 tools returned
call_tool list_documents	OK — items returned
call_tool get_document	OK
"Unexpected content type" warnings	One warning remains, but only on the 204 No Content reply to notifications/initialized. The empty Content-Type for a 204 is per HTTP spec; the FastMCP/mcp-python client logs an ERROR-level message for it but continues normally — the subsequent JSON-RPC calls succeed.

Before D2a, the same FastMCP version produced "Unexpected content type" for every JSON-RPC response when the client expected SSE in Streamable HTTP mode (per Rev2 S3.1 framing). After D2a, the warning is reduced to the single 204-without-body case, which is benign and outside our control. Functional impact: zero — full conversation works. Any future fix to that quirk lives in the mcp-python library, not in our server.

7. Regression — D1 still passes

After the D2a deploy, re-ran the D1 functional + perf harness against the new binary:

• 16 / 16 D1 functional checks PASS (tools/list counts, prefix variants, escape, pagination, caps, synonym, soft-delete exclusion, get/batch/search regression).
• list_documents path=knowledge/ limit=50 c=5 n=50: p50 = 40.2 ms, p95 = 102.6 ms, max = 114.8 ms — within the 250 ms D1 target.

No tool latency degraded.

8. Commit

• Hash: 34ca724
• Branch: main
• Message: phase2f-d2a: add MCP JSON/SSE content negotiation
• Co-author trailer: Claude Opus 4.7 (1M context)
• Repo: /opt/incomex/docker/agent-data-repo (still 5 ahead, 112 behind origin/main per project convention).

9. Logs / resource snapshot

• No ERROR / Exception / Traceback in container logs since restart.
• CPU 1.0 %, memory 1.4 GiB / 2.5 GiB on incomex-agent-data.
• /health reports all three dependencies (qdrant, postgres, openai) OK.
• No nginx reload performed.

10. Rollback

ssh root@38.242.240.89
cd /opt/incomex/docker/agent-data-repo
git revert 34ca724
cd /opt/incomex/docker
docker compose build agent-data && docker compose up -d agent-data

No DB rollback. No nginx rollback.

11. Risks / notes

• R1 (acknowledged). D2a is production-compliance, NOT proven to fix ChatGPT MCP App hang. Per Rev2 S3.1 we ship it for spec correctness and SSE-strict clients; we explicitly do not advertise it as the ChatGPT unblocker. Whether ChatGPT now works is a separate test to be run only after a deliberate decision (see S12).
• R2 (benign). FastMCP/mcp-python streamable_http client still logs ONE "Unexpected content type" error per session on the 204 notifications/initialized reply. Non-fatal; the conversation completes normally. Library quirk, not a server bug. Documented in S6.
• R3 (auth always JSON). API-key 401 stays plain JSON; no SSE for unauthenticated requests. Intentional.
• R4 (q parsing). Quality factors outside [0, 1] are clamped to 1.0 if parse fails, but float values outside that range are accepted as-is. Edge case for adversarial clients; not a security concern.
• R5 (no streaming Tools/call yet). Long-running tools still produce a single SSE frame at the end. Progress events / server-pushed notifications are D2b.

12. Recommendation — D3 first or ChatGPT retry?

Run D3 first, then retry ChatGPT.

D3 (remove move_document from MCP_TOOLS / MCP_GPT_FULL_TOOLS / dispatcher) is a 4-line patch with zero functional risk. Doing it before the ChatGPT retest gives the LLM a clean tools list and prevents Tool not allowed audit noise during the test.

Concrete sequence:

1. D3 — schema cleanup, ~5 minute change + 6 tests, separate commit.
2. ChatGPT MCP App retest — fresh conversation, hit /gpt-mcp/{token}/mcp, attempt search + list + patch on knowledge/test/.... Observe whether the patch UI hang persists.
3. If hang persists -> file a separate diagnosis ticket; D2a was production-compliance, not the silver bullet.
4. If hang resolves -> ship D4 canary (still gated separately per Rev2 S5).

D4 canary remains parked until C1..C7 evidence is collected.

---

Appendix — Diff stat

agent_data/server.py | 235 +++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 207 insertions(+), 28 deletions(-)

End of report. No nginx change. No DB change. No restart of postgres/qdrant. No secret rotated.