See Scramjet.
scramjetorg / framework-js Goto Github PK
View Code? Open in Web Editor NEWSimple yet powerful live data computation framework.
Home Page: https://www.scramjet.org
License: MIT License
Simple yet powerful live data computation framework.
Home Page: https://www.scramjet.org
License: MIT License
See Scramjet.
Introduce FileStream
similar to FSStream
proposal from v4.
FileStream items would point to filesystem items like files, directories, etc and should provides methods like stat, contents.
The concept is similar to Vinyl objects so the stream should allow filesystem items manipulations (similar to how it is done in Gulp, Grunt, etc).
Manipulating files in batches, listing filesystem, files pipelines.
If you are interested in this issue or this stream type fits your use case, please upvote with ๐.
Introduce DataStream.empty()
method similar to v4:
Called only before the stream ends without passing any items.
If you are interested in this issue or this type of transforms fits your use case, please upvote with ๐.
Introduce DataStream.flatten()
method similar to v4:
A shorthand for streams of arrays or iterables to flatten them.
More efficient equivalent of: .flatmap(i => i); Works on streams of async iterables too.
If you are interested in this issue or this type of transforms fits your use case, please upvote with ๐.
Introduce BinaryStream
similar to BufferStream
from v4.
By default, each chunk flowing through the stream would be a single byte. The stream should allow manipulating binary data (grouping, splitting, parsing, shifting, etc). Also it should be possible to convert such stream into other types (most notably StringStream
and DataStream
).
Various file types transformations, parsing binary formats, binary data manipulation (images, videos, etc).
If you are interested in this issue or this stream type fits your use case, please upvote with ๐.
Introduce DataStream.all() method
similar to v4:
Processes a number of functions in parallel, returns a stream of arrays of results.
This method is to allow running multiple asynchronous operations and receive all the results at one, just like Promise.all behaves.
If you are interested in this issue or this type of transforms fits your use case, please upvote with ๐.
Introduce DataStream.assign()
method similar to v4:
Transforms stream objects by assigning the properties from the returned data along with data from original ones.
The original objects are unaltered.
If you are interested in this issue or this type of transforms fits your use case, please upvote with ๐.
I have noticed recently that CI is being unstable and some tests are failing from time to time. It seems to be the case of pipe tests mostly:
streams โบ data โบ native-interface โบ Piped DataStream can be unpiped via '.unpipe(instance)' #2
build/test/unit/streams/data/native-interface.spec.js:109
108: await stream.end();
109: t.deepEqual(await stream.toArray(), ["fo", "o\n"]);
110: resolve();
Difference:
[
'fo',
+ `oโ
+ `,
]
โบ ReadStream.<anonymous> (build/test/unit/streams/data/native-interface.spec.js:109:15)
streams โบ data โบ native-interface โบ Piped DataStream can be unpiped via '.unpipe()' #2
build/test/unit/streams/data/native-interface.spec.js:145
144: await stream.end();
145: t.deepEqual(await stream.toArray(), ["fo", "o\n"]);
146: resolve();
Difference:
[
'fo',
+ `oโ
+ `,
]
โบ ReadStream.<anonymous> (build/test/unit/streams/data/native-interface.spec.js:145:15)
โ
2 tests failed
5 tests skipped
This may mean there is some issue with how it works internally.
Introduce WindowStream
similar to WindowStream
from v4:
In essence it's a stream of Array's containing a list of items - a window.
It would be a stream for moving window calculation with some simple methods.
Any data calculations which requires moving window (e.g. calculating moving averages).
If you are interested in this issue or this stream type fits your use case, please upvote with ๐.
Introduce MultiStream
/MuxStream
similar to MultiStrean
from v4.
The idea behind a MultiStream is being able to mux and demux streams when needed. MultiStream is an object consisting of multiple streams than can be refined or muxed.
Merging streams from various sources so all data can be processed by the same pipeline. Distributing data into multiple streams.
If you are interested in this issue or this stream type fits your use case, please upvote with ๐.
In v4
.batch()
is a simple transform which groups chunks by a given number:
// input: [1,2,3,4,5,6,7,8,9,10]
// batch(amount)
.batch(2) -> [1,2], [3,4], [5,6], ...
and in pre-v5
it accepts a callback but we plan to introduce batching by amount the same as v4
too.
Now, it could be extended by adding optional step
param which will allow to change how chunks are grouped:
// input: [1,2,3,4,5,6,7,8,9,10]
// batch(amount, step = amount)
.batch(2) -> [1,2], [3,4], [5,6], ...
.batch(2, 2) -> [1,2], [3,4], [5,6], ...
.batch(2, 1) -> [1,2], [2,3], [3,4], ...
.batch(3, 2) -> [1, 2, 3], [3, 4, 5], [5, 6, 7], ...
The use case is - creating a WindowStream
out of any other stream with ability to decide how each window frame content is grouped. And in the context of the above proposal it could be:
anyStream.batch(3, 2).as(WindowStream)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.