404-solution / includes / diagnostics / ResponseBodyDeliveryEvidence.php

ResponseBodyDeliveryEvidence.php in 404 Solution trunk, at includes/diagnostics/ResponseBodyDeliveryEvidence.php

358 lines 17.9 KB

Raw Download Zip

1	<?php
2
3	if (!defined('ABSPATH')) {
4	exit;
5	}
6
7	/**
8	* What the server WROTE against what the browser RECEIVED, joined per browser
9	* session.
10	*
11	* The one comparison that names a body rewritten in transit, and the one thing
12	* neither side can say alone. The server knows exactly how many bytes it
13	* encoded and echoed; only the browser can report how many bytes actually
14	* arrived, and it can report that even when the payload will not parse,
15	* because Resource Timing's `decodedBodySize` is measured on the wire and not
16	* by the JSON parser.
17	*
18	* Support report 2026-08-27 (plugin 4.3.4, Azure App Service for Linux, nginx
19	* + brotli, `wp-content-copy-protector` and `gtranslate` both active) is the
20	* case this exists for. The ladder's `stream` step came back `parsererror`
21	* with `truncated_on_arrival = false`: the server emitted 3072 bytes, the
22	* browser's Resource Timing reported 6089, and the report was read as a
23	* mid-response-flush failure for a week. A body that arrives COMPLETE,
24	* UNPARSEABLE, and at roughly twice the size the server wrote has been
25	* rewritten by an intermediary, and both halves of that comparison were
26	* already in the journal. Nothing joined them, so the ladder could only ever
27	* say "the step failed".
28	*
29	* This class owns the join and nothing else: which records belong to one
30	* browser session, which emitted count pairs with which delivered count, and
31	* the accounting around that. It owns no decision rule -- the tolerance and
32	* the verdict live in ABJ_404_Solution_ResponseBodyRewriteVerdict, pure and
33	* directly tested -- no transport, and no formatting. That is the same split
34	* ABJ_404_Solution_DetachAbEvidence already uses against
35	* ABJ_404_Solution_DetachAbVerdict.
36	*
37	* It does not own what a measured byte count IS either. Whether a reported
38	* value is a real measurement or a refusal to answer, and the vocabulary
39	* naming which source produced it, are shared with the delivered half in
40	* ABJ_404_Solution_MeasuredBodyBytes: both halves have to answer that question
41	* identically or their difference means nothing, so it is one definition
42	* rather than a copy on each side. Nor does it own READING the journal:
43	* turning JSONL text into the three typed maps joined below, and deciding
44	* which fields of a half-written or older-version record can be read at all,
45	* is ABJ_404_Solution_BodyDeliveryObservations.
46	*
47	* Three properties are load-bearing rather than defensive:
48	*
49	* 1. Unknown is never a match. An absent `decodedBodySize` is a finding
50	* about the BROWSER (no Resource Timing entry, an opaque response, a
51	* cleared buffer), not evidence that the body arrived intact. A
52	* comparison with either half missing is reported as such and reaches
53	* no verdict.
54	* 2. Emitted means every body byte the server wrote, not just the JSON.
55	* The `stream` step echoes a leading whitespace block OUTSIDE
56	* json_encode(), so on a host where that block is actually emitted the
57	* encoded size understates the body by exactly
58	* AjaxCanaryStreamFlush::WHITESPACE_BYTES. That is exact accounting
59	* the journal already carries, not slack to be absorbed by a tolerance.
60	* 3. Session scoping. The checkpoint journal is site-wide and two admin
61	* tabs write into one file, so a comparison is only ever built from
62	* records this session's own browser produced.
63	*/
64	final class ABJ_404_Solution_ResponseBodyDeliveryEvidence {
65
66	private const MAX_REPORTED_STEP_CHARS = 32;
67
68	/** The journal event this class writes its joined evidence under. */
69	const EVIDENCE_EVENT = 'body_delivery_evidence';
70
71	/** The join was computed from this session's own records. */
72	const STATUS_COMPUTED = 'computed';
73
74	/** The client sent no session id, so nothing can be scoped to one browser. */
75	const STATUS_NO_SESSION = 'no_session';
76
77	/** The journal could not be read or joined; the reason travels with the record. */
78	const STATUS_ERROR = 'error';
79
80	/** Both halves are known, so the pure rule may compare them. */
81	const STATE_COMPARABLE = 'comparable';
82
83	/** The server never journaled a size for this response. */
84	const STATE_EMITTED_UNKNOWN = 'emitted_unknown';
85
86	/** The browser reported no usable `decodedBodySize` for this response. */
87	const STATE_DELIVERED_UNKNOWN = 'delivered_unknown';
88
89	/** Neither half is known. */
90	const STATE_BOTH_UNKNOWN = 'both_unknown';
91
92	/**
93	* The pseudo-step naming the real admin-table request the ladder run
94	* exists to explain. It is not a ladder step and never will be, but it is
95	* the response the whole comparison is ultimately about, so it is carried
96	* in the same list rather than as a second shape a reader has to join by
97	* hand.
98	*/
99	const REAL_REQUEST_STEP = 'real_request';
100
101	// The byte-provenance vocabulary (which source produced a count, and
102	// whether a reported value is a measurement at all) is shared with the
103	// delivered half of the comparison and lives in
104	// ABJ_404_Solution_MeasuredBodyBytes. Both halves must answer "is this a
105	// real measurement" identically or the comparison is meaningless, which is
106	// why it is one definition rather than a copy on each side.
107
108	// No cap on how many comparisons ride the record, deliberately: the count
109	// is already bounded by the fixed CanaryLadderStep::DISPATCHED list plus one
110	// real-request row, so a cap could only ever start silently dropping rows
111	// off a diagnostic record if the ladder grew. There is no separate total
112	// field either -- with nothing truncated, a stored count could only ever
113	// agree with `comparisons` or be wrong about it, and a reader trusting the
114	// wrong one of two facts is worse than counting the rows.
115
116	/**
117	* The full emitted-against-delivered join for one browser session, ready
118	* to journal and ready to hand to the pure interpretation rule.
119	*
120	* Never throws and never returns a partial shape: every field is present
121	* on every path, and `status` says why the record is what it is.
122	*
123	* @return array<string, mixed>
124	*/
125	public static function forSession(string $sessionId): array {
126	$sessionKey = ABJ_404_Solution_DetachAbExperiment::sessionKey($sessionId);
127	try {
128	if ($sessionId === '') {
129	$record = self::emptyRecord($sessionKey);
130	$record['status'] = self::STATUS_NO_SESSION;
131	return $record;
132	}
133	// ONE read of the checkpoint journal, shared by both halves of the
134	// comparison. Reading it twice let a record appended in between
135	// pair an emitted size from one snapshot with a delivery record
136	// from another -- a comparison neither read observed, reported with
137	// the confidence of a real one.
138	$source = ABJ_404_Solution_CheckpointJournalReader::supportCollectionSource();
139	$checkpointLines = ABJ_404_Solution_DiagnosticJournalExcerpt::readAllLines($source['paths']);
140	$traceSource = ABJ_404_Solution_AjaxTraceJournal::supportCollectionSource();
141	$traceLines = ABJ_404_Solution_DiagnosticJournalExcerpt::readAllLines($traceSource['paths']);
142	return self::fromLines(
143	$checkpointLines,
144	$sessionKey,
145	ABJ_404_Solution_EncodedTableResponseSize::fromLines(
146	$traceLines, $checkpointLines, $sessionId)
147	);
148	} catch (Throwable $e) {
149	$record = self::emptyRecord($sessionKey);
150	$record['status'] = self::STATUS_ERROR;
151	$record['error'] = substr($e->getMessage(), 0, 200);
152	return $record;
153	}
154	}
155
156	/**
157	* The same record built from journal lines a caller already holds.
158	*
159	* ABJ_404_Solution_CanaryReceiptEvidence reconstructs the ladder matrix
160	* offline, from exactly these receipts, when the live `interpret`
161	* response never reached the browser -- which is the case this whole
162	* diagnostic exists for. Recomputing the matrix there WITHOUT this join
163	* would silently drop the one verdict that names the incident from the
164	* support payload, so both paths go through one implementation rather
165	* than through two that can disagree.
166	*
167	* @param array<int, string> $lines Checkpoint JSONL lines, oldest first.
168	* @param string $sessionKey ABJ_404_Solution_DetachAbExperiment::sessionKey().
169	* @param array{bytes: int\|null, source: string, request_id: string} $realRequest
170	* @return array<string, mixed>
171	*/
172	public static function fromLines(array $lines, string $sessionKey, array $realRequest): array {
173	$record = self::emptyRecord($sessionKey);
174	$joined = self::comparisonsIn($lines, $sessionKey, $realRequest);
175	$record['comparisons'] = $joined['comparisons'];
176	$record['stream_flush_reached_sapi'] = $joined['stream_flush_reached_sapi'];
177	$record['journal_lines_scanned'] = count($lines);
178	$record['verdict'] = ABJ_404_Solution_ResponseBodyRewriteVerdict::fromComparisons(
179	$joined['comparisons'], self::MAX_REPORTED_STEP_CHARS);
180	return $record;
181	}
182
183	/**
184	* The record every path starts from: a complete shape with no evidence in
185	* it yet, so "nothing to join" can never drift out of step with the shape
186	* a joined one has.
187	*
188	* @return array<string, mixed>
189	*/
190	private static function emptyRecord(string $sessionKey): array {
191	return array(
192	'status' => self::STATUS_COMPUTED,
193	'session_key' => $sessionKey,
194	'comparisons' => array(),
195	// null, never false: "this session journaled no stream step" and
196	// "the stream step ran and reached nobody" are opposite findings.
197	'stream_flush_reached_sapi' => null,
198	'journal_lines_scanned' => 0,
199	// Produced by the real rule rather than hand-written as a
200	// literal, so a "nothing to join" record can never drift out of
201	// step with the shape a decided one has.
202	'verdict' => ABJ_404_Solution_ResponseBodyRewriteVerdict::fromComparisons(array()),
203	);
204	}
205
206	/**
207	* One session's emitted-against-delivered comparisons, in ladder order.
208	*
209	* Pure and side-effect free: it takes journal lines rather than reading
210	* them, so every join case (a foreign session, a step with no receipt, a
211	* receipt with no Resource Timing, a response the server never sized) is
212	* directly assertable without a journal on disk that happens to contain
213	* it.
214	*
215	* The static-asset probe is deliberately absent from the result. It never
216	* reaches PHP, so there is no encoded response to compare against and its
217	* emitted half is not merely unknown but nonexistent; listing it would put
218	* a permanent `emitted_unknown` row in every record.
219	*
220	* @param array<int, string> $lines JSONL lines, oldest first.
221	* @param string $sessionKey ABJ_404_Solution_DetachAbExperiment::sessionKey().
222	* @param array{bytes: int\|null, source: string, request_id: string} $realRequest
223	* ABJ_404_Solution_EncodedTableResponseSize::forSession(). Unknown fields
224	* are tolerated; only `bytes` and `request_id` are read.
225	* @return array{comparisons: array<int, array<string, mixed>>,
226	* stream_flush_reached_sapi: bool\|null}
227	*/
228	public static function comparisonsIn(array $lines, string $sessionKey, array $realRequest): array {
229	$receipts = ABJ_404_Solution_BodyDeliveryObservations::receiptsInSession($lines, $sessionKey);
230	$encoded = ABJ_404_Solution_BodyDeliveryObservations::encodedSizesIn($lines);
231	$streamFlush = ABJ_404_Solution_BodyDeliveryObservations::streamFlushOutcomesIn($lines);
232
233	// Scoped by the record's OWN session first, so "did this session's
234	// stream step reach the wire" survives a lost browser receipt: that
235	// receipt travels on the delivery channel under diagnosis, and making
236	// the answer depend on it is the cross-channel join that already cost
237	// this investigation a week. The receipt-keyed lookup below is the
238	// fallback for records written before the stamp existed.
239	$streamReachedSapi = null;
240	foreach ($streamFlush as $outcome) {
241	if ($sessionKey !== '' && $outcome['session_key'] === $sessionKey) {
242	$streamReachedSapi = $outcome['reached_sapi'];
243	}
244	}
245
246	$comparisons = array();
247	foreach (ABJ_404_Solution_CanaryLadderStep::DISPATCHED as $step) {
248	if (!isset($receipts[$step])) {
249	continue;
250	}
251	$requestId = $receipts[$step]['request_id'];
252	$flush = $streamFlush[$requestId] ?? null;
253	if ($streamReachedSapi === null
254	&& $step === ABJ_404_Solution_CanaryLadderStep::STREAM && $flush !== null) {
255	$streamReachedSapi = $flush['reached_sapi'];
256	}
257	$comparisons[] = self::comparison(array(
258	'step' => $step,
259	'request_id' => $requestId,
260	'emitted' => self::emittedBytes($encoded[$requestId] ?? null, $flush),
261	'delivered' => $receipts[$step]['delivered_bytes'],
262	'timing_state' => $receipts[$step]['resource_timing_state'],
263	));
264	}
265
266	$realId = ABJ_404_Solution_AjaxRequestLedger::normalizeId($realRequest['request_id'] ?? null, '');
267	// Through the same rule the receipt sizes use: two copies would
268	// eventually disagree about zero and the -1 sentinel, and both halves
269	// of a comparison must treat "unknown" identically.
270	$realEmitted = ABJ_404_Solution_MeasuredBodyBytes::disclosed($realRequest['bytes'] ?? null);
271	if ($realId !== '' \|\| $realEmitted !== null) {
272	$reported = ABJ_404_Solution_DeliveredTableResponseSize::forRequest($lines, $realId);
273	$comparisons[] = self::comparison(array(
274	'step' => self::REAL_REQUEST_STEP,
275	'request_id' => $realId,
276	'emitted' => $realEmitted === null
277	? array('bytes' => null, 'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_UNAVAILABLE)
278	: array('bytes' => $realEmitted, 'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_ENCODE),
279	'delivered' => $reported['bytes'],
280	'timing_state' => $reported['resource_timing_state'],
281	));
282	}
283	return array('comparisons' => $comparisons, 'stream_flush_reached_sapi' => $streamReachedSapi);
284	}
285
286	/**
287	* One comparison row. The known-ness of each half is NAMED here; whether a
288	* known pair actually differs is a decision and stays in
289	* ABJ_404_Solution_CanaryLadderInterpretation::interpret().
290	*
291	* Takes one keyed row rather than positional arguments. `step`,
292	* `request_id` and `timing_state` are all strings, so in positional form a
293	* caller could transpose any two of them and produce a perfectly typed row
294	* that attributes one step's delivery evidence to another. Nothing
295	* downstream could detect it: every field would still be a plausible
296	* string. Keys make the transposition unwriteable, and PHPStan checks the
297	* shape at each call site. (PHP 7.4 is the floor here, so named arguments
298	* are not available and a positional value-object constructor would move
299	* the same hazard rather than remove it.)
300	*
301	* @param array{step: string, request_id: string, emitted: array{bytes: int\|null, source: string}, delivered: int\|null, timing_state: string} $row
302	* @return array<string, mixed>
303	*/
304	private static function comparison(array $row): array {
305	$emitted = $row['emitted'];
306	$delivered = $row['delivered'];
307	if ($emitted['bytes'] === null && $delivered === null) {
308	$state = self::STATE_BOTH_UNKNOWN;
309	} else if ($emitted['bytes'] === null) {
310	$state = self::STATE_EMITTED_UNKNOWN;
311	} else if ($delivered === null) {
312	$state = self::STATE_DELIVERED_UNKNOWN;
313	} else {
314	$state = self::STATE_COMPARABLE;
315	}
316	return array(
317	'step' => $row['step'],
318	'request_id' => $row['request_id'],
319	'emitted_bytes' => $emitted['bytes'],
320	'emitted_source' => $emitted['source'],
321	'delivered_bytes' => $delivered,
322	'delivered_source' => $delivered === null
323	? ABJ_404_Solution_MeasuredBodyBytes::SOURCE_UNAVAILABLE : ABJ_404_Solution_MeasuredBodyBytes::SOURCE_RESOURCE_TIMING,
324	'resource_timing_state' => $row['timing_state'],
325	'state' => $state,
326	);
327	}
328
329	/**
330	* Every body byte the server wrote for one response, or a named absence.
331	*
332	* A stream step whose whitespace count is UNKNOWN yields an unknown total,
333	* not the encode count alone: the step put a prefix on the wire, and
334	* understating that is indistinguishable from a body rewritten in transit.
335	*
336	* @param int\|null $encodedBytes The `json_encode` record's own count.
337	* @param array{reached_sapi: bool\|null, whitespace_bytes: int\|null}\|null $flush
338	* @return array{bytes: int\|null, source: string}
339	*/
340	private static function emittedBytes(?int $encodedBytes, ?array $flush): array {
341	if ($encodedBytes === null) {
342	return array('bytes' => null, 'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_UNAVAILABLE);
343	}
344	if ($flush !== null && $flush['whitespace_bytes'] === null) {
345	return array('bytes' => null, 'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_UNAVAILABLE);
346	}
347	$whitespace = $flush === null ? 0 : max(0, $flush['whitespace_bytes']);
348	if ($whitespace === 0) {
349	return array('bytes' => $encodedBytes, 'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_ENCODE);
350	}
351	return array(
352	'bytes' => $encodedBytes + $whitespace,
353	'source' => ABJ_404_Solution_MeasuredBodyBytes::SOURCE_ENCODE_PLUS_STREAM,
354	);
355	}
356
357	}
358