1 |
3a515b92
|
cagy
|
# parallel-transform
|
2 |
|
|
|
3 |
|
|
[Transform stream](http://nodejs.org/api/stream.html#stream_class_stream_transform_1) for Node.js that allows you to run your transforms
|
4 |
|
|
in parallel without changing the order of the output.
|
5 |
|
|
|
6 |
|
|
npm install parallel-transform
|
7 |
|
|
|
8 |
|
|
It is easy to use
|
9 |
|
|
|
10 |
|
|
``` js
|
11 |
|
|
var transform = require('parallel-transform');
|
12 |
|
|
|
13 |
|
|
var stream = transform(10, function(data, callback) { // 10 is the parallism level
|
14 |
|
|
setTimeout(function() {
|
15 |
|
|
callback(null, data);
|
16 |
|
|
}, Math.random() * 1000);
|
17 |
|
|
});
|
18 |
|
|
|
19 |
|
|
for (var i = 0; i < 10; i++) {
|
20 |
|
|
stream.write(''+i);
|
21 |
|
|
}
|
22 |
|
|
stream.end();
|
23 |
|
|
|
24 |
|
|
stream.on('data', function(data) {
|
25 |
|
|
console.log(data); // prints 0,1,2,...
|
26 |
|
|
});
|
27 |
|
|
stream.on('end', function() {
|
28 |
|
|
console.log('stream has ended');
|
29 |
|
|
});
|
30 |
|
|
```
|
31 |
|
|
|
32 |
|
|
If you run the above example you'll notice that it runs in parallel
|
33 |
|
|
(does not take ~1 second between each print) and that the order is preserved
|
34 |
|
|
|
35 |
|
|
## Stream options
|
36 |
|
|
|
37 |
|
|
All transforms are Node 0.10 streams. Per default they are created with the options `{objectMode:true}`.
|
38 |
|
|
If you want to use your own stream options pass them as the second parameter
|
39 |
|
|
|
40 |
|
|
``` js
|
41 |
|
|
var stream = transform(10, {objectMode:false}, function(data, callback) {
|
42 |
|
|
// data is now a buffer
|
43 |
|
|
callback(null, data);
|
44 |
|
|
});
|
45 |
|
|
|
46 |
|
|
fs.createReadStream('filename').pipe(stream).pipe(process.stdout);
|
47 |
|
|
```
|
48 |
|
|
|
49 |
|
|
### Unordered
|
50 |
|
|
Passing the option `{ordered:false}` will output the data as soon as it's processed by a transform, without waiting to respect the order.
|
51 |
|
|
|
52 |
|
|
## License
|
53 |
|
|
|
54 |
|
|
MIT
|