solzimer / fileq Goto Github PK
View Code? Open in Web Editor NEWFile based FIFO queue. High-performance queue that stores JSON objects in a file-based FIFO
License: MIT License
File based FIFO queue. High-performance queue that stores JSON objects in a file-based FIFO
License: MIT License
Hi, I'm trying to have more that one reader on the same queue and when I start the second reader, it fail:
reader.js code:
const FileQueue = require("fileq");
var queue = FileQueue.from("queue", {"path": __dirname });
var i=0;
setInterval(()=>{
queue.peek((err,entry)=>{
console.log(entry);
});
},100);
Then try to run this code in 2 separated terminals as "node reader.js" ...
The first one run ok, but the second response with the below
Error:
/node_modules/levelup/lib/levelup.js:119
return callback(new OpenError(err))
^
Error [OpenError]: IO error: lock //queue/LOCK: Resource temporarily unavailable
at //node_modules/levelup/lib/levelup.js:119:23
at //node_modules/abstract-leveldown/abstract-leveldown.js:38:14
at //node_modules/deferred-leveldown/deferred-leveldown.js:31:21
at //node_modules/abstract-leveldown/abstract-leveldown.js:38:14
Emitted 'error' event on LevelUP instance at:
at //node_modules/levelup/lib/levelup.js:60:19
at //node_modules/levelup/lib/levelup.js:119:14
at //node_modules/abstract-leveldown/abstract-leveldown.js:38:14
at //node_modules/deferred-leveldown/deferred-leveldown.js:31:21
at //node_modules/abstract-leveldown/abstract-leveldown.js:38:14
Unless I'm missing something, the path
argument to FileQueue.from()
has to be a name, and if you want to specify the location for your queue, you need to put it in the second argument, options
.
Hi there,
I was looking around for a simple file backed FIFO message queue and found this library.
"peek" usually allows you to look at the head item on the queue without removing it.
"pop" usually is for removing an item from the queue.
I'm actually looking for the "peek" feature where I can process the head item on the queue first without removing it; and then after confirming it has successfully processed; remove it.
One interesting and seemingly useful approach I saw on 'file-queue' was a "Transactional Pop".
The way it worked is you executed "tpop" and were given a "commit" and "rollback" callback functions.
You'd process the message, then call the appropriate callback when it was sorted out what to do.
A feature that would be really nice is a queue flag to indicate that a transaction is currently in progress. I was iterating over several queues in a loop and found that I needed to introduce a queue_lock to prevent from processing multiple items from the queue simultaneously.
Hope that's useful!
Thanks for the library!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.