1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
|
---
title: 使用可读文件流
slug: Web/API/Streams_API/使用可读文件流
translation_of: Web/API/Streams_API/Using_readable_streams
---
<div>{{apiref("Streams")}}</div>
<p class="summary">作为一个js开发者,一块一块的读取和操作一个从网上获取的数据流是非常实用的功能!但是如何使用Streams API操作数据流呢? 可以在这里看到基本的介绍.</p>
<div class="note">
<p><strong>提示</strong>: 本文要求您已了解流文件相关知识,如果还不了解,建议您先查看 <a href="/en-US/docs/Web/API/Streams_API#Concepts_and_usage">文件流概念及简介</a> 以及 dedicated <a href="/en-US/docs/Web/API/Streams_API/Concepts">文件流API</a> 然后再阅读此文.</p>
</div>
<div class="note">
<p><strong>提示</strong>: 如果你正在查询关于可读写的文件流, 请查看<a href="/en-US/docs/Web/API/Streams_API/Using_writable_streams">使用可写文件流</a> .</p>
</div>
<h2 id="Browser_support">Browser support</h2>
<p>You can consume Fetch {{domxref("Body")}} objects as streams and create your own custom readable streams in Firefox 65+ and Chrome 42+ (and equivalent Chromium-based browsers). <a href="/en-US/docs/Web/API/Streams_API/Concepts#Pipe_chains">Pipe chains</a> are only supported in Chrome at the moment, and that functionality is subject to change.</p>
<h2 id="Finding_some_examples">Finding some examples</h2>
<p>We will look at various examples in this article, taken from our <a href="https://github.com/mdn/dom-examples/tree/master/streams">dom-examples/streams</a> repo. You can find the full source code there, as well as links to the examples.</p>
<h2 id="Consuming_a_fetch_as_a_stream">Consuming a fetch as a stream</h2>
<p>The <a href="/en-US/docs/Web/API/Fetch_API">Fetch API</a> allows you to fetch resources across the network, providing a modern alternative to <a href="/en-US/docs/User%3Amaybe/webidl_mdn/XMLHttpRequest">XHR</a>. It has a number of advantages, and what is really nice about it is that browsers have recently added the ability to consume a fetch response as a readable stream.</p>
<p>The {{domxref("Body")}} mixin now includes the {{domxref("Body.body","body")}} property, which is a simple getter exposing the body contents as a readable stream. This mixin is implemented by both the {{domxref("Request")}} and {{domxref("Response")}} interfaces, so it is available on both, although consuming the stream of a response body is perhaps a bit more obvious.</p>
<p>As our <a href="https://github.com/mdn/dom-examples/tree/master/streams/simple-pump">Simple stream pump</a> example shows (<a href="https://mdn.github.io/dom-examples/streams/simple-pump/">see it live also</a>), exposing it is a matter of just accessing the <code>body</code> property of the response:</p>
<pre class="brush: js">// Fetch the original image
fetch('./tortoise.png')
// Retrieve its body as ReadableStream
.then(response => response.body)</pre>
<p>This provides us with a {{domxref("ReadableStream")}} object.</p>
<h3 id="Attaching_a_reader">Attaching a reader</h3>
<p>Now we’ve got our streaming body, reading the stream requires attaching a reader to it. This is done using the {{domxref("ReadableStream.getReader()")}} method:</p>
<pre class="brush: js">// Fetch the original image
fetch('./tortoise.png')
// Retrieve its body as ReadableStream
.then(response => response.body)
.then(body => {
const reader = body.getReader();</pre>
<p>Invoking this method creates a reader and locks it to the stream — no other reader may read this stream until this reader is released, e.g. by invoking {{domxref("ReadableStreamDefaultReader.releaseLock()")}}.</p>
<p>Also note that the previous example can be reduced by one step, as <code>response.body</code> is synchronous and so doesn't need the promise:</p>
<pre class="brush: js">// Fetch the original image
fetch('./tortoise.png')
// Retrieve its body as ReadableStream
.then(response => {
const reader = response.body.getReader();</pre>
<h3 id="Reading_the_stream">Reading the stream</h3>
<p>Now you’ve got your reader attached, you can read data chunks out of the stream using the {{domxref("ReadableStreamDefaultReader.read()")}} method. This reads one chunk out of the stream, which you can then do anything you like with. For example, our Simple stream pump example goes on to enqueue each chunk in a new, custom <code>ReadableStream</code> (we will find more about this in the next section), then create a new {{domxref("Response")}} out of it, consume it as a {{domxref("Blob")}}, create an object URL out of that blob using {{domxref("URL.createObjectURL()")}}, and then display it on screen in an {{htmlelement("img")}} element, effectively creating a copy of the image we originally fetched.</p>
<pre class="brush: js"> return new ReadableStream({
start(controller) {
return pump();
function pump() {
return reader.read().then(({ done, value }) => {
// When no more data needs to be consumed, close the stream
if (done) {
controller.close();
return;
}
// Enqueue the next data chunk into our target stream
controller.enqueue(value);
return pump();
});
}
}
})
})
.then(stream => new Response(stream))
.then(response => response.blob())
.then(blob => URL.createObjectURL(blob))
.then(url => console.log(image.src = url))
.catch(err => console.error(err));</pre>
<p>Let’s look in detail at how <code>read()</code> is used. In the <code>pump()</code> function seen above we first invoke <code>read()</code>, which returns a promise containing a results object — this has the results of our read in it, in the form <code>{ done, value }</code>:</p>
<pre class="brush: js">return reader.read().then(({ done, value }) => {</pre>
<p>The results can be one of three different types:</p>
<ul>
<li>If a chunk is available to read, the promise will be fulfilled with an object of the form { value: theChunk, done: false }.</li>
<li>If the stream becomes closed, the promise will be fulfilled with an object of the form { value: undefined, done: true }.</li>
<li>If the stream becomes errored, the promise will be rejected with the relevant error.</li>
</ul>
<p>Next, we check whether done is <code>true</code>. If so, there are no more chunks to read (the value is <code>undefined</code>) so we return out of the function and close the custom stream with {{domxref("ReadableStreamDefaultController.close()")}}:</p>
<pre class="brush: js">if (done) {
controller.close();
return;
}</pre>
<div class="note">
<p><strong>Note</strong>: <code>close()</code> is part of the new custom stream, not the original stream we are discussing here. We’ll explain more about the custom stream in the next section.</p>
</div>
<p>If <code>done</code> is not true, we process the new chunk we’ve read (contained in the <code>value</code> property of the results object) and then call the <code>pump()</code> function again to read the next chunk.</p>
<pre class="brush: js">// Enqueue the next data chunk into our target stream
controller.enqueue(value);
return pump();</pre>
<p>This is the standard pattern you’ll see when using stream readers:</p>
<ol>
<li>You write a function that starts off by reading the stream.</li>
<li>If there is no more stream to read, you return out of the function.</li>
<li>If there is more stream to read, you process the current chunk then run the function again.</li>
<li>You keep running the function recursively until there is no more stream to read, in which case step 2 is followed.</li>
</ol>
<h2 id="Creating_your_own_custom_readable_stream">Creating your own custom readable stream</h2>
<p>The Simple stream pump example we’ve been studying throughout this article includes a second part — once we’ve read the image from the fetch body in chunks, we then enqueue them into another, custom stream of our own creation. How do we create this? The <code>ReadableStream</code> constructor.</p>
<h3 id="The_ReadableStream_constructor">The ReadableStream constructor</h3>
<p>It is easy to read from a stream when the browser provides it for you as in the case of Fetch, but sometimes you need to create a custom stream and populate it with your own chunks. The {{domxref("ReadableStream.ReadableStream()")}} constructor allows you to do this via a syntax that looks complex at first, but actually isn’t too bad.</p>
<p>The generic syntax skeleton looks like this:</p>
<pre class="brush: js">const stream = new ReadableStream({
start(controller) {
},
pull(controller) {
},
cancel() {
},
type,
autoAllocateChunkSize
}, {
highWaterMark,
size()
});</pre>
<p>The constructor takes two objects as parameters. The first object is required, and creates a model in JavaScript of the underlying source the data is being read from. The second object is optional, and allows you to specify a <a href="/en-US/docs/Web/API/Streams_API/Concepts#Internal_queues_and_queuing_strategies">custom queueing strategy</a> to use for your stream. You’ll rarely have to do this, so we’ll just concentrate on the first one for now.</p>
<p>The first object can contain up to five members, only the first of which is required:</p>
<ol>
<li><code>start(controller)</code> — A method that is called once, immediately after the <code>ReadableStream</code> is constructed. Inside this method, you should include code that sets up the stream functionality, e.g. beginning generation of data or otherwise getting access to the source.</li>
<li><code>pull(controller)</code> — A method that, when included, is called repeatedly until the stream’s internal queue is full. This can be used to control the stream as more chunks are enqueued.</li>
<li><code>cancel()</code> — A method that, when included, will be called if the app signals that the stream is to be cancelled (e.g. if {{domxref("ReadableStream.cancel()")}} is called). The contents should do whatever is necessary to release access to the stream source.</li>
<li><code>type</code> and <code>autoAllocateChunkSize</code> — These are used — when included — to signify that the stream is to be a bytestream. Bytestreams will be covered separately in a future tutorial, as they are somewhat different in purpose and use case to regular (default) streams. They are also not implemented anywhere as yet.</li>
</ol>
<p>Looking at our simple example code again, you can see that our <code>ReadableStream</code> constructor only includes a single method — <code>start()</code>, which serves to read all the data out of our fetch stream.</p>
<pre class="brush: js"> return new ReadableStream({
start(controller) {
return pump();
function pump() {
return reader.read().then(({ done, value }) => {
// When no more data needs to be consumed, close the stream
if (done) {
controller.close();
return;
}
// Enqueue the next data chunk into our target stream
controller.enqueue(value);
return pump();
});
}
}
})
})
</pre>
<h3 id="ReadableStream_controllers">ReadableStream controllers</h3>
<p>You’ll notice that the <code>start()</code> and <code>pull()</code> methods passed into the <code>ReadableStream</code> constructor are given <code>controller</code> parameters — these are instances of the {{domxref("ReadableStreamDefaultController")}} class, which can be used to control your stream.</p>
<p>In our example we are using the controller’s {{domxref("ReadableStreamDefaultController.enqueue","enqueue()")}} method to enqueue a value into the custom stream after it is read from the fetch body.</p>
<p>In addition, when we are done reading the fetch body we use the controller’s {{domxref("ReadableStreamDefaultController.close","close()")}} method to close the custom stream — any previously-enqueued chunks can still be read from it, but no more can be enqueued, and the stream is closed when reading has finished.</p>
<p>Reading from custom streams</p>
<p>In our Simple stream pump example, we consume the custom readable stream by passing it into a {{domxref("Response.Response", "Response")}} constructor call, after which we consume it as a blob().</p>
<pre class="brush: js">.then(stream => new Response(stream))
.then(response => response.blob())
.then(blob => URL.createObjectURL(blob))
.then(url => console.log(image.src = url))
.catch(err => console.error(err));</pre>
<p>But a custom stream is still a <code>ReadableStream</code> instance, meaning you can attach a reader to it. As an example, have a look at our <a href="https://github.com/mdn/dom-examples/blob/master/streams/simple-random-stream/index.html">Simple random stream demo</a> (<a href="https://mdn.github.io/dom-examples/streams/simple-random-stream/">see it live also</a>), which creates a custom stream, enqueues some random strings into it, and then reads the data out of the stream again once the <em>Stop string generation</em> button is pressed.</p>
<p>The custom stream constructor has a <code>start()</code> method that uses a {{domxref("WindowTimers.setInterval()")}} call to generate a random string every second. {{domxref("ReadableStreamDefaultController.enqueue()")}} is then used to enqueue it into the stream. When the button is pressed, the interval is cancelled, and a function called <code>readStream()</code> is invoked to read the data back out of the stream again. We also close the stream, as we’ve stopped enqueueing chunks to it.</p>
<pre class="brush: js">const stream = new ReadableStream({
start(controller) {
interval = setInterval(() => {
let string = randomChars();
// Add the string to the stream
controller.enqueue(string);
// show it on the screen
let listItem = document.createElement('li');
listItem.textContent = string;
list1.appendChild(listItem);
}, 1000);
button.addEventListener('click', function() {
clearInterval(interval);
readStream();
controller.close();
})
},
pull(controller) {
// We don't really need a pull in this example
},
cancel() {
// This is called if the reader cancels,
// so we should stop generating strings
clearInterval(interval);
}
});</pre>
<p>In the <code>readStream()</code> function itself, we lock a reader to the stream using {{domxref("ReadableStream.getReader()")}}, then follow the same kind of pattern we saw earlier — reading each chunk with <code>read()</code>, checking whether <code>done</code> is <code>true</code> and then ending the process if so, and reading the next chunk and processing it if not, before running the <code>read()</code> function again.</p>
<pre class="brush: js">function readStream() {
const reader = stream.getReader();
let charsReceived = 0;
// read() returns a promise that resolves
// when a value has been received
reader.read().then(function processText({ done, value }) {
// Result objects contain two properties:
// done - true if the stream has already given you all its data.
// value - some data. Always undefined when done is true.
if (done) {
console.log("Stream complete");
para.textContent = result;
return;
}
charsReceived += value.length;
const chunk = value;
let listItem = document.createElement('li');
listItem.textContent = 'Read ' + charsReceived + ' characters so far. Current chunk = ' + chunk;
list2.appendChild(listItem);
result += chunk;
// Read some more, and call this function again
return reader.read().then(processText);
});
}</pre>
<h3 id="Closing_and_cancelling_streams">Closing and cancelling streams</h3>
<p>We’ve already shown examples of using {{domxref("ReadableStreamDefaultController.close()")}} to close a reader. As we said before, any previously enqueued chunks will still be read, but no more can be enqueued because it is closed.</p>
<p>If you wanted to completely get rid of the stream and discard any enqueued chunks, you'd use {{domxref("ReadableStream.cancel()")}} or {{domxref("ReadableStreamDefaultReader.cancel()")}}.</p>
<h2 id="Teeing_a_stream">Teeing a stream</h2>
<p>Sometimes you might want to read a stream twice, simultaneously. This is achieved via the {{domxref("ReadableStream.tee()")}} method — it outputs an array containing two identical copies of the original readable stream, which can then be read independently by two separate readers.</p>
<p>You might do this for example in a <a href="/en-US/docs/Web/API/Service_Worker_API">ServiceWorker</a> if you want to fetch a response from the server and stream it to the browser, but also stream it to the Service Worker cache. Since a response body cannot be consumed more than once, and a stream can't be read by more than one reader at once, you’d need two copies to do this.</p>
<p>We provide an example of this in our <a href="https://github.com/mdn/dom-examples/blob/master/streams/simple-tee-example/index.html">Simple tee example</a> (<a href="https://mdn.github.io/dom-examples/streams/simple-tee-example/">see it live also</a>). This example works much the same way as our Simple random stream, except that when the button is pressed to stop generating random strings, the custom stream is taken and teed, and both resulting streams are then read:</p>
<pre class="brush: js">function teeStream() {
const teedOff = stream.tee();
readStream(teedOff[0], list2);
readStream(teedOff[1], list3);
}</pre>
<h2 id="Pipe_chains">Pipe chains</h2>
<p>One very experimental feature of streams is the ability to pipe streams into one another (called a <a href="/en-US/docs/Web/API/Streams_API/Concepts#Pipe_chains">pipe chain</a>). This involves two methods — {{domxref("ReadableStream.pipeThrough()")}}, which pipes a readable stream through a writer/reader pair to transform one data format into another, and {{domxref("ReadableStream.pipeTo()")}}, which pipes a readable stream to a writer acting as an end point for the pipe chain.</p>
<p>This functionality is at a very experimental stage and is subject to change, so we have no explored it too deeply as of yet.</p>
<p>We have created an example called <a href="https://github.com/mdn/dom-examples/tree/master/streams/png-transform-stream">Unpack Chunks of a PNG</a> (<a href="https://mdn.github.io/dom-examples/streams/png-transform-stream/">see it live also</a>) that fetches an image as a stream, then pipes it through to a custom PNG transform stream <span class="pl-c">that retrieves PNG chunks out of a binary data stream.</span></p>
<pre class="brush: js">// Fetch the original image
fetch('png-logo.png')
// Retrieve its body as ReadableStream
.then(response => response.body)
// Create a gray-scaled PNG stream out of the original
.then(rs => logReadableStream('Fetch Response Stream', rs))
.then(body => body.pipeThrough(new PNGTransformStream()))
.then(rs => logReadableStream('PNG Chunk Stream', rs))</pre>
<h2 id="Summary">Summary</h2>
<p>That explains the basics of “default” readable streams. We’ll explain bytestreams in a separate future article, once they are available in browsers.</p>
|