ByteTransformer
1 week ago
ReadStream
1 week ago
WriteStream
1 week ago
LICENSE.md
1 week ago
README.md
1 week ago
class-bytepipe.php
1 week ago
class-bytestreamexception.php
1 week ago
class-filereadwritestream.php
1 week ago
class-memorypipe.php
1 week ago
class-notenoughdataexception.php
1 week ago
composer.json
1 week ago
README.md
205 lines
| 1 | --- |
| 2 | slug: bytestream |
| 3 | title: ByteStream |
| 4 | install: wp-php-toolkit/bytestream |
| 5 | |
| 6 | see_also: |
| 7 | - filesystem | Filesystem | Back file reads and writes with the same stream primitives. |
| 8 | - zip | Zip | Read and write archive entries one stream at a time. |
| 9 | - httpclient | HttpClient | Process request and response bodies incrementally. |
| 10 | --- |
| 11 | |
| 12 | Composable streaming primitives for reading, writing, transforming, hashing, and compressing byte data. Pull/peek/consume semantics let parsers look ahead with bounded buffering, and deflate, inflate, and checksum filters snap together like Lego. |
| 13 | |
| 14 | ## Why this exists |
| 15 | |
| 16 | <p>PHP's native streams are powerful but inconsistent. <code>fread</code> on a socket may return short reads with no warning; <code>stream_filter_append</code> is awkward to compose; gzip helpers and file handles expose different APIs. The ByteStream component normalizes these behind one small interface — <code>pull / peek / consume</code> — so a parser, a hash function, and a deflate filter all see the same shape.</p> |
| 17 | |
| 18 | <p>The split between <em>pull</em> (buffer up to N bytes) and <em>consume</em> (advance past N bytes) is the secret. Parsers can <code>peek</code> ahead to detect a record boundary and decide whether to <code>consume</code>, while keeping buffer ownership inside the stream.</p> |
| 19 | |
| 20 | ## Read a file in chunks |
| 21 | |
| 22 | <p>The canonical loop. <code>pull(N)</code> reads up to <code>N</code> bytes from the underlying source into an internal buffer and returns how many ended up there; <code>consume(N)</code> reads <code>N</code> bytes from that buffer and advances past them. The buffer never grows beyond the chunk size you ask for.</p> |
| 23 | |
| 24 | <!-- snippet: |
| 25 | filename: teaser-read.php |
| 26 | runnable: true |
| 27 | --> |
| 28 | ```php |
| 29 | <?php |
| 30 | require '/php-toolkit/vendor/autoload.php'; |
| 31 | |
| 32 | use WordPress\ByteStream\ReadStream\FileReadStream; |
| 33 | |
| 34 | $path = tempnam( sys_get_temp_dir(), 'demo' ); |
| 35 | file_put_contents( $path, str_repeat( "log line\n", 200 ) ); |
| 36 | |
| 37 | $reader = FileReadStream::from_path( $path ); |
| 38 | $total = 0; |
| 39 | while ( ! $reader->reached_end_of_data() ) { |
| 40 | $n = $reader->pull( 256 ); |
| 41 | if ( 0 === $n ) break; |
| 42 | $total += strlen( $reader->consume( $n ) ); |
| 43 | } |
| 44 | $reader->close_reading(); |
| 45 | echo "Read {$total} bytes in 256-byte chunks.\n"; |
| 46 | ``` |
| 47 | |
| 48 | <!-- expected-output --> |
| 49 | ``` |
| 50 | Read 1800 bytes in 256-byte chunks. |
| 51 | ``` |
| 52 | |
| 53 | ## MemoryPipe as write-then-read buffer |
| 54 | |
| 55 | <p><code>MemoryPipe</code> is bidirectional: you <code>append_bytes()</code> as a writer and <code>pull/consume</code> as a reader. Easiest way to wire one component's output into another's input.</p> |
| 56 | |
| 57 | <p>Gotcha: <strong>A producer must call <code>close_writing()</code> when done — otherwise the consumer eventually throws <code>NotEnoughDataException</code> instead of seeing EOF.</strong> </p> |
| 58 | |
| 59 | <!-- snippet: |
| 60 | filename: memory-pipe.php |
| 61 | runnable: true |
| 62 | --> |
| 63 | ```php |
| 64 | <?php |
| 65 | require '/php-toolkit/vendor/autoload.php'; |
| 66 | |
| 67 | use WordPress\ByteStream\MemoryPipe; |
| 68 | |
| 69 | $pipe = new MemoryPipe(); |
| 70 | $pipe->append_bytes( "first chunk\n" ); |
| 71 | $pipe->append_bytes( "second chunk\n" ); |
| 72 | $pipe->append_bytes( "third chunk\n" ); |
| 73 | $pipe->close_writing(); |
| 74 | |
| 75 | while ( ! $pipe->reached_end_of_data() ) { |
| 76 | $n = $pipe->pull( 1024 ); |
| 77 | if ( 0 === $n ) break; |
| 78 | echo "got: " . $pipe->consume( $n ); |
| 79 | } |
| 80 | ``` |
| 81 | |
| 82 | <!-- expected-output --> |
| 83 | ``` |
| 84 | got: first chunk |
| 85 | second chunk |
| 86 | third chunk |
| 87 | ``` |
| 88 | |
| 89 | ## Compress on the way in, decompress on the way out |
| 90 | |
| 91 | <p>Wrap a stream in <code>DeflateReadStream</code> to get compressed bytes out; wrap it in <code>InflateReadStream</code> to get decompressed bytes out. Both are full <code>ByteReadStream</code> implementations, so they nest into anything else that takes a stream. These filters use PHP's <code>zlib</code> functions.</p> |
| 92 | |
| 93 | <!-- snippet: |
| 94 | filename: deflate-roundtrip.php |
| 95 | runnable: true |
| 96 | --> |
| 97 | ```php |
| 98 | <?php |
| 99 | require '/php-toolkit/vendor/autoload.php'; |
| 100 | |
| 101 | use WordPress\ByteStream\MemoryPipe; |
| 102 | use WordPress\ByteStream\ReadStream\DeflateReadStream; |
| 103 | use WordPress\ByteStream\ReadStream\InflateReadStream; |
| 104 | |
| 105 | $original = str_repeat( "the quick brown fox. ", 50 ); |
| 106 | |
| 107 | $src = new MemoryPipe( $original ); |
| 108 | $src->close_writing(); |
| 109 | $deflated = new DeflateReadStream( $src, ZLIB_ENCODING_DEFLATE ); |
| 110 | $compressed = $deflated->consume_all(); |
| 111 | |
| 112 | $src2 = new MemoryPipe( $compressed ); |
| 113 | $src2->close_writing(); |
| 114 | $inflated = new InflateReadStream( $src2, ZLIB_ENCODING_DEFLATE ); |
| 115 | $round = $inflated->consume_all(); |
| 116 | |
| 117 | printf( "original : %d bytes\n", strlen( $original ) ); |
| 118 | printf( "deflated : %d bytes (%.1f%%)\n", strlen( $compressed ), 100 * strlen( $compressed ) / strlen( $original ) ); |
| 119 | printf( "round-trip: %s\n", $round === $original ? 'OK' : 'BROKEN' ); |
| 120 | ``` |
| 121 | |
| 122 | <!-- expected-output --> |
| 123 | ``` |
| 124 | original : 1050 bytes |
| 125 | deflated : 45 bytes (4.3%) |
| 126 | round-trip: OK |
| 127 | ``` |
| 128 | |
| 129 | ## Line-by-line reads from a chunked source |
| 130 | |
| 131 | <p>Reading text by line means handling chunk boundaries that fall mid-line. Keep the trailing partial line and prepend it to the next pull. The rest of the loop pretends the data was always whole.</p> |
| 132 | |
| 133 | <!-- snippet: |
| 134 | filename: lines.php |
| 135 | runnable: true |
| 136 | --> |
| 137 | ```php |
| 138 | <?php |
| 139 | require '/php-toolkit/vendor/autoload.php'; |
| 140 | |
| 141 | use WordPress\ByteStream\MemoryPipe; |
| 142 | |
| 143 | $pipe = new MemoryPipe(); |
| 144 | $pipe->append_bytes( "alpha\nbravo\ncharl" ); |
| 145 | $pipe->append_bytes( "ie\ndelta\necho\n" ); |
| 146 | $pipe->close_writing(); |
| 147 | |
| 148 | $tail = ''; |
| 149 | $count = 0; |
| 150 | while ( ! $pipe->reached_end_of_data() ) { |
| 151 | $n = $pipe->pull( 8 ); |
| 152 | if ( 0 === $n ) break; |
| 153 | $buf = $tail . $pipe->consume( $n ); |
| 154 | $lines = explode( "\n", $buf ); |
| 155 | $tail = array_pop( $lines ); |
| 156 | foreach ( $lines as $line ) { |
| 157 | printf( "[%d] %s\n", ++$count, $line ); |
| 158 | } |
| 159 | } |
| 160 | if ( '' !== $tail ) { |
| 161 | printf( "[%d] %s\n", ++$count, $tail ); |
| 162 | } |
| 163 | ``` |
| 164 | |
| 165 | <!-- expected-output --> |
| 166 | ``` |
| 167 | [1] alpha |
| 168 | [2] bravo |
| 169 | [3] charlie |
| 170 | [4] delta |
| 171 | [5] echo |
| 172 | ``` |
| 173 | |
| 174 | ## Limit a stream to a fixed window |
| 175 | |
| 176 | <p><code>LimitedByteReadStream</code> exposes only the next N bytes of an underlying stream as if those were the entire stream. This is how the ZIP decoder hands you the body of one entry without letting you read into the next.</p> |
| 177 | |
| 178 | <!-- snippet: |
| 179 | filename: limited.php |
| 180 | runnable: true |
| 181 | --> |
| 182 | ```php |
| 183 | <?php |
| 184 | require '/php-toolkit/vendor/autoload.php'; |
| 185 | |
| 186 | use WordPress\ByteStream\MemoryPipe; |
| 187 | use WordPress\ByteStream\ReadStream\LimitedByteReadStream; |
| 188 | |
| 189 | $source = new MemoryPipe( "HEADER:42|BODY:hello there|FOOTER:done" ); |
| 190 | $source->close_writing(); |
| 191 | |
| 192 | $source->pull( 10 ); |
| 193 | $source->consume( 10 ); |
| 194 | |
| 195 | $body = new LimitedByteReadStream( $source, 16 ); |
| 196 | echo "body sees: " . $body->consume_all() . "\n"; |
| 197 | echo "remaining in source: " . $source->consume_all() . "\n"; |
| 198 | ``` |
| 199 | |
| 200 | <!-- expected-output --> |
| 201 | ``` |
| 202 | body sees: BODY:hello there |
| 203 | remaining in source: |FOOTER:done |
| 204 | ``` |
| 205 |