PluginProbe ʕ •ᴥ•ʔ
AI Engine – The Chatbot, AI Framework & MCP for WordPress / 3.7.5
AI Engine – The Chatbot, AI Framework & MCP for WordPress v3.7.5
3.7.5 3.7.4 3.7.3 3.7.2 3.7.1 3.7.0 3.6.9 3.6.8 3.6.7 3.6.6 3.6.4 3.6.5 3.6.3 3.6.2 3.6.1 3.6.0 3.5.9 3.5.8 3.5.7 3.5.6 3.5.5 3.5.4 3.5.3 3.5.2 3.5.1 3.5.0 3.4.9 3.4.8 3.4.7 0.2.1 1.6.91 0.2.2 1.6.92 0.2.3 1.6.93 0.2.4 1.6.94 0.2.5 1.6.95 0.2.6 1.6.96 0.2.7 1.6.97 0.2.8 1.6.98 0.2.9 1.6.99 0.3.0 1.7.0 0.3.1 1.7.1 0.3.2 1.7.2 0.3.3 1.7.3 0.3.4 1.7.4 0.3.5 1.7.5 0.3.6 1.7.6 0.4.0 1.7.7 0.4.1 1.7.8 0.4.2 1.7.9 0.4.3 1.8.0 0.4.4 1.8.1 0.4.5 1.8.2 0.4.6 1.8.3 0.4.7 1.8.4 0.4.8 1.8.5 0.4.9 1.8.6 0.5.0 1.8.7 0.5.1 1.8.8 0.5.2 1.8.9 0.5.3 1.9.0 0.5.4 1.9.1 0.5.5 1.9.2 0.5.6 1.9.3 0.5.7 1.9.4 0.5.8 1.9.5 0.5.9 1.9.6 0.6.0 1.9.7 0.6.1 1.9.8 0.6.2 1.9.81 0.6.3 1.9.82 0.6.4 1.9.83 0.6.5 1.9.84 0.6.6 1.9.85 0.6.7 1.9.86 0.6.8 1.9.87 0.6.9 1.9.88 0.7.0 1.9.89 0.7.1 1.9.90 0.7.2 1.9.91 0.7.3 1.9.92 0.7.4 1.9.93 0.7.5 1.9.94 0.7.6 1.9.95 0.7.7 1.9.96 0.7.8 1.9.97 0.7.9 1.9.98 0.8.0 1.9.99 0.8.1 2.0.0 0.8.2 2.0.1 0.8.3 2.0.2 0.8.4 2.0.3 0.8.5 2.0.4 0.8.6 2.0.5 0.8.7 2.0.6 0.8.8 2.0.7 0.8.9 2.0.8 0.9.0 2.0.9 0.9.2 2.1.0 0.9.3 2.1.1 0.9.4 2.1.2 0.9.5 2.1.3 0.9.6 2.1.4 0.9.7 2.1.5 0.9.8 2.1.6 0.9.81 2.1.7 0.9.82 2.1.8 0.9.83 2.1.9 0.9.84 2.2.0 0.9.85 2.2.1 0.9.86 2.2.2 0.9.87 2.2.3 0.9.88 2.2.4 0.9.89 2.2.5 0.9.9 2.2.51 0.9.91 2.2.52 0.9.92 2.2.53 0.9.93 2.2.54 0.9.94 2.2.56 0.9.95 2.2.57 0.9.96 2.2.6 0.9.97 2.2.60 0.9.98 2.2.61 0.9.99 2.2.62 1.0.0 2.2.63 1.0.01 2.2.70 1.0.1 2.2.80 1.0.2 2.2.81 1.0.3 2.2.90 1.0.4 2.2.91 1.0.5 2.2.92 1.0.6 2.2.93 1.0.7 2.2.94 1.0.8 2.2.95 1.0.9 2.3.0 1.1.0 2.3.1 1.1.1 2.3.2 1.1.2 2.3.3 1.1.3 2.3.4 1.1.4 2.3.5 1.1.5 2.3.6 1.1.6 2.3.7 1.1.7 2.3.8 1.1.8 2.3.9 1.1.9 2.4.0 1.2.0 2.4.1 1.2.1 2.4.2 1.2.2 2.4.3 1.2.21 2.4.4 1.2.3 2.4.5 1.2.30 2.4.6 1.3.0 2.4.7 1.3.1 2.4.8 1.3.2 2.4.9 1.3.3 2.5.0 1.3.31 2.5.1 1.3.32 2.5.2 1.3.33 2.5.3 1.3.34 2.5.4 1.3.35 2.5.5 1.3.36 2.5.6 1.3.37 2.5.7 1.3.38 2.5.8 1.3.39 2.5.9 1.3.40 2.6.0 1.3.41 2.6.1 1.3.42 2.6.2 1.3.43 2.6.3 1.3.44 2.6.5 1.3.45 2.6.6 1.3.46 2.6.7 1.3.47 2.6.8 1.3.48 2.6.9 1.3.49 2.7.0 1.3.50 2.7.1 1.3.51 2.7.2 1.3.52 2.7.3 1.3.53 2.7.4 1.3.54 2.7.5 1.3.56 2.7.6 1.3.57 2.7.7 1.3.58 2.7.8 1.3.59 2.7.9 1.3.60 2.8.0 1.3.61 2.8.1 1.3.62 2.8.2 1.3.63 2.8.3 1.3.64 2.8.4 1.3.65 2.8.5 1.3.66 2.8.6 1.3.67 2.8.7 1.3.68 2.8.8 1.3.69 2.8.9 1.3.70 2.9.0 1.3.71 2.9.1 1.3.72 2.9.2 1.3.73 2.9.3 1.3.74 2.9.4 1.3.75 2.9.5 1.3.76 2.9.6 1.3.77 2.9.7 1.3.78 2.9.8 1.3.79 2.9.9 1.3.80 3.0.0 1.3.81 3.0.1 1.3.82 3.0.2 1.3.83 3.0.3 1.3.84 3.0.4 1.3.85 3.0.5 1.3.86 3.0.6 1.3.87 3.0.7 1.3.88 3.0.8 1.3.89 3.0.9 1.3.90 3.1.0 1.3.91 3.1.1 1.3.92 3.1.2 1.3.93 3.1.3 1.3.94 3.1.4 1.3.95 3.1.5 1.3.96 3.1.6 1.3.97 3.1.7 1.3.98 3.1.8 1.3.99 3.1.9 1.4.0 3.2.0 1.4.1 3.2.1 1.4.2 3.2.2 1.4.3 3.2.3 1.4.4 3.2.4 1.4.5 3.2.5 1.4.6 3.2.6 1.4.7 3.2.7 1.4.8 3.2.8 1.4.9 3.2.9 1.5.0 3.3.0 1.5.1 3.3.1 1.5.2 3.3.2 1.5.3 3.3.3 1.5.4 3.3.4 1.5.5 3.3.5 1.5.6 3.3.6 1.5.7 3.3.7 1.5.8 3.3.8 1.5.9 3.3.9 1.6.0 3.4.0 1.6.1 3.4.1 1.6.2 3.4.2 1.6.3 3.4.3 1.6.5 3.4.4 1.6.51 3.4.5 1.6.52 3.4.6 1.6.53 1.6.54 1.6.55 1.6.56 1.6.57 1.6.58 1.6.59 1.6.60 1.6.61 1.6.62 1.6.63 1.6.64 1.6.65 1.6.66 1.6.67 1.6.68 trunk 1.6.69 0.0.1 1.6.70 0.0.2 1.6.71 0.0.3 1.6.72 0.0.4 1.6.73 0.0.5 1.6.74 0.0.6 1.6.75 0.0.7 1.6.76 0.0.8 1.6.77 0.0.9 1.6.78 0.1.0 1.6.79 0.1.1 1.6.81 0.1.2 1.6.82 0.1.3 1.6.83 0.1.4 1.6.84 0.1.5 1.6.85 0.1.6 1.6.86 0.1.7 1.6.87 0.1.8 1.6.88 0.1.9 1.6.89 0.2.0 1.6.90
ai-engine / notes / STUDY-CONSOLIDATION.md
ai-engine / notes Last commit date
CHANGELOG-PROPOSAL.txt 1 month ago QA-LOOP.md 1 month ago RELEASE-QA-NIGHT.md 1 month ago STUDY-CONSOLIDATION.md 1 day ago STUDY-MCP.md 3 weeks ago STUDY-SETTINGS-STORAGE.md 3 weeks ago STUDY-WORKSPACE.md 1 month ago STUDY-WPAI-LEARNINGS.md 1 month ago
STUDY-CONSOLIDATION.md
119 lines
1 # STUDY-CONSOLIDATION.md — AI Engine consolidation roadmap
2
3 > **Living planning doc.** Born 2026-06-29 from a strategic review (chatbot + MCP + WP7) after the
4 > Gemini Interactions engine shipped in 3.5.6 with four silent regressions nobody caught until we
5 > drove the live API by hand. The lesson: **product breadth is outrunning verification, docs, and UX.**
6 > This phase is **consolidation, not another feature wave** — done little by little, one item per
7 > few pulses, surfaced via `/pulse`. Read `STRATEGY.md` and `STUDY-MCP.md` first for posture.
8
9 ## Core principle
10
11 Do **not** freeze chatbot features (unrealistic for a solo continuous shipper). Instead put a
12 **verification gate** and a **param schema** in front of new work, so features stop adding unverified
13 surface. Consolidate the main paths incrementally behind that gate. The threat is not competitors;
14 it is complexity outrunning verification.
15
16 ## The evidence this is needed
17
18 3.5.6 shipped the Gemini Interactions engine as the **default** for all Google users with: dropped RAG/
19 content-aware context, dropped chat history (restored discussions), broken PDF-in-chat, and a broken
20 connection test. All four were invisible to code review and the test suite; only live API driving caught
21 them. Fixed in commit `f93d978e`. This is the proof, not a hypothetical.
22
23 ## Backlog (ranked — pull the highest-value item that fits the time)
24
25 - [x] **1. Pre-release smoke gate for the chatbot/engine path.** Highest leverage. The `labs/tests`
26 harness we used would have caught every 3.5.6 regression. Make a runnable battery (text, history,
27 context/RAG, image, function-call non-stream + stream, file/PDF) across each provider env, run before
28 any release. This is the structural fix for the failure mode above.
29 **Slice one DONE 2026-07-02**: `node labs/tests/test-smoke.js` drives every configured AI env through
30 CONNECT (/ai/test_connection), TEXT, HISTORY (messages array), and STREAM (SSE live + end events),
31 which covers the 3.5.6 connection-test and dropped-history classes directly. Models are picked
32 dynamically from settings (overridable via `smoke.models` in query-api.local.conf). First live run:
33 6 envs (OpenAI, Anthropic, Google, OpenRouter, OVH, Mistral), all green, exit code wired for CI-style
34 use. Run before any `pnpm zip`.
35 **Slice two DONE 2026-07-03**: FUNC (non-stream via /simpleChatbotQuery) + FUNCSTRM (streamed via
36 mwai-ui/v1/chats/submit with a guest nonce from /start_session) run function calling through the real
37 chatbot pipeline against fixture bots (botId starting with "smoke", mapped per env; smoke-openai +
38 smoke-anthropic created on ai.nekod.net, wired to the Code Engine "[Test] Get Magic Word" snippet).
39 Validated against the real regression: with the 84ecb22b Sonnet 5 fix reverted, FUNCSTRM fails with
40 the exact user-reported "each thinking block must contain thinking" error; with the fix, all green.
41 Checks retry once so transient provider slowness (OVH) doesn't fail the gate.
42 **Slice three, MCP round-trip DONE 2026-07-23**: `node labs/tests/test-smoke.js mcp` (also runs on
43 full runs) drives the real MCP Streamable HTTP transport (bearer self-discovered from settings):
44 MCPINIT handshake, then wp_create_post/wp_update_post_meta with nested-array + unicode meta read
45 back per-key and deep-compared (a double-serialized value is called out explicitly), block-JSON
46 `\u` escapes + shortcode + unicode content through wp_get_post, fixture post force-deleted after.
47 Covers the corruption classes Dave Hilditch's PSA documented (f3a2f4fd/fcda9415, cb806da4,
48 600d86eb; window was 2.8.5 → 3.5.9). Validated against the real bug: with `maybe_serialize()`
49 reintroduced in wp_update_post_meta, METAUPD fails with "DOUBLE-SERIALIZED"; with the fix, all green.
50 **Slice three, RAG-through-chat DONE 2026-07-24**: RAGCHAT drives /simpleChatbotQuery on the
51 smoke-rag fixture bot (wired to the new Internal (WordPress DB) vector env, intern01 on
52 ai.nekod.net) and expects a fact that only exists in the knowledge base. Validated against a
53 simulated dropped-context regression.
54 **Slice three, files-in-chat DONE 2026-09-03**: IMAGE (test-data/gotokuji.jpg, reply must say
55 "cat") and PDF (test-data/invoice-slices.pdf, reply must contain the invoice number) run on the
56 same smoke-* fixture bot as FUNC, through /simpleFileUpload + /simpleChatbotQuery with fileIds.
57 SKIP when the settings model list says the bot's model lacks the vision/files tag. Item closed:
58 every regression class from 3.5.6 now has a check. `node labs/tests/test-smoke.js` before any
59 `pnpm zip`. Fixture bots live on ai.nekod.net and can vanish with a settings reset; recreate
60 smoke-openai / smoke-anthropic (function 17, maxTokens 4096) and smoke-rag (intern01) if the
61 gate prints SKIP for FUNC/RAGCHAT.
62 - [ ] **2. Gemini external MCP consumption.** OpenAI (`openai.php:~403`) and Anthropic (`anthropic.php`)
63 consume `$query->mcpServers`; the new default Google engine (`google-interactions.php`) does NOT — it
64 only wires Google's built-in tools. MCP is the core moat, so the default Google engine being unable to
65 use it is a real gap. Best shape: a provider-neutral MCP-to-function bridge. See
66 `project_gemini_no_external_mcp` memory.
67 - [ ] **3. Schema-driven chatbot params.** Replace the manual `MWAI_CHATBOT_FRONT_PARAMS` /
68 `MWAI_CHATBOT_SERVER_PARAMS` + 4-location sync (documented as a footgun in CLAUDE.md) with one
69 schema-driven model. Shrinks the regression surface; more ROI than a full `chat_submit()` rewrite.
70 - [ ] **4. MCP capability governance + onboarding.** MCP = remote admin automation, not "just a chatbot
71 feature." Role presets, mutating-tool audit log default-on (easy off), per-client/tool allowlists, and
72 onboarding that makes the admin power level unmistakable. `require_approval => never` (openai.php:440,
73 realtime.php:387) makes server-side allowlists + warnings more important.
74 - [ ] **5. Chatbot hygiene pass.** Remove/guard the `[ERROR]` test path (`ChatbotContext.js:985`); narrow
75 the `eval()` function-action surface (`ChatbotContext.js:589`, `:840`); audit the error-HTML
76 `dangerouslySetInnerHTML` rendering with a tight sanitize pass.
77 - [ ] **6. WP7 directory reconciliation.** `STRATEGY.md` says WP7 code lives in `/labs/wp7-integration/`,
78 but the actual files are flat `labs/wpai-*.php`. Reconcile the doc or move the files before this area
79 grows. Cheap; do it before the WP7 surface expands.
80 - [ ] **7. (Longer-term) Incremental `chat_submit()` / `ChatbotContext` decomposition.** Split the
81 monster submission pipeline behind the smoke gate — incrementally, NOT a big-bang rewrite (the rewrite
82 is itself a regression risk). Only after items 1 and 3 land.
83
84 ## Already-decided (the review re-derived these — confirmations, not new work)
85
86 - Deprioritize Realtime + Assistants unless users are loudly asking (`feedback_deprioritized_bugs`).
87 - MCP URL-token fallback is temporary; re-eval after 2026-12-27 (`project_mcp_url_token_revisit`).
88 - WP7 = stable connector layer for a moving framework; Abilities bridge planned (`project_wp7_mcp_eta`).
89 - Keep **image-only** behavior for image-gen Gemini models this release (no inferring mixed text+image
90 from chat text — it would risk the verified image fix).
91
92 ## Progress Log
93
94 - 2026-06-29: Roadmap created from the chatbot/MCP/WP7 strategic review. Baseline established; nothing
95 ticked yet. Context: Gemini Interactions hardening (commit f93d978e) just closed the regressions that
96 motivated this.
97 - 2026-07-02: Item 1 slice one shipped. `labs/tests/test-smoke.js` gates CONNECT/TEXT/HISTORY/STREAM
98 across all 6 AI envs in one command (~2 min, all green on first full run). Slice two (RAG, function
99 calling, PDF) needs site fixtures and is listed as SKIP in the gate output.
100 - 2026-07-03: Item 1 slice two shipped. FUNC + FUNCSTRM cover function calling (non-stream + streamed)
101 through the real chatbot pipeline via smoke-* fixture bots. Proven against the same-day Sonnet 5
102 streaming bug (84ecb22b): gate fails with the fix reverted, passes with it. Full run: 27 passed,
103 0 failed, 9 skipped (envs without fixture bots). Retry-once added for flaky providers.
104 - 2026-07-23: Item 1 slice three (MCP round-trip) shipped. `node labs/tests/test-smoke.js mcp` drives
105 the real Streamable HTTP transport: MCPINIT, METAREAD, METAUPD, CONTENT, MCPCLEAN. Validated against
106 the real bug (maybe_serialize reintroduced fails METAUPD with DOUBLE-SERIALIZED). Covers the
107 2.8.5 to 3.5.9 meta/content corruption classes from Dave Hilditch's PSA.
108 - 2026-07-24: Item 1 slice three (RAG through chat) shipped. RAGCHAT on the smoke-rag fixture bot,
109 wired to the Internal (WordPress DB) env intern01. Validated against a simulated dropped-context
110 regression.
111 - 2026-09-03: Item 1 CLOSED. IMAGE + PDF checks shipped (the last two SKIP lines are gone). The
112 fixture bots had been wiped from ai.nekod.net since July (every FUNC/RAGCHAT was silently SKIP);
113 recreated, and the note above says how to do it again. Full run: OpenAI 8/8, Anthropic 8/8,
114 RAG 1/1, MCP 5/5. First catch of the new slice: a fixture bot with maxTokens 128000 on Haiku
115 4.5 400s at Anthropic ("max_tokens: 128000 > 64000"), the same thing a user hits when switching
116 a chatbot to a smaller Claude model. Fixed by capping max_tokens to the model's
117 maxCompletionTokens in `anthropic.php` (max_tokens_for). Next: item 3 (schema-driven chatbot
118 params) or item 2 (Gemini external MCP), Jordy's pick.
119