Comments (7)
my issue is rather a question
That's fine.
if it would be possible to cache them either via Web Storage API or Cache API persistently
Great idea!
Currently, all the cached data is stored in a sparse array here: https://github.com/phiresky/sql.js-httpvfs/blob/master/src/lazyFile.ts#L36 So to generalize that you could just replace that with a simple interface Cache {get(i: number): Uint8Array, set(i: number, chunk: Uint8Array)}
.
The main problem you'll probably encounter is that the cache has to be synchronous to work with my code, since SQLite is synchronous, so e.g. IndexedDB won't work directly.
As far as I know, the localStorage API is pretty limited, both in amount of data that can be saved as well as the structure, but it should work. The Cache API probably won't work because ServiceWorkers can't respond to synchronous XHR.
So for a simple solution, I'd try using localStorage. You can also use IndexedDB (or any async storage) if you save to the cache in the background and load everything into memory on page load.
For a better solution, you probably need to make the whole thing async. The easiest way here is probably to base it on wa-sqlite/IndexedDbVFS instead of my project, and reimplement my lazyFile
on top of that. The reason is that wa-sqlite provides an API for async VFSes based on emscripten asyncify.
from sql.js-httpvfs.
You can check out how https://github.com/jlongster/absurd-sql does it. Ideally these projects would be merged!
from sql.js-httpvfs.
The main problem you'll probably encounter is that the cache has to be synchronous to work with my code, since SQLite is synchronous, so e.g. IndexedDB won't work directly.
Can you elaborate? The API of sql.js-httpvfs
is asynchronous. Why does it matter that the SQLite
is synchronous?
from sql.js-httpvfs.
All the VFS calls are called from within SQLite (like callbacks). Since SQLite calls them in a synchronous manner the VFS JS functions also have to be synchronous.
from sql.js-httpvfs.
@phiresky So you mean that in addition to the patch you have already made to sql.js
to support byte-range queries, you would need to write another patch to make sql.js
asynchronous?
from sql.js-httpvfs.
If you want to use sql.js specifically, yeah. But I wouldn't base new work on sql.js in any case
from sql.js-httpvfs.
Closing because I have no plans on continuing adding new features to sql.js-httpvfs, since the base is pretty hacky and reimplementing on top of a cleaner base would be preferrable. If I would do it again I would try using wa-sqlite which is better suited for this kind of thing.
from sql.js-httpvfs.
Related Issues (20)
- Support for Sharding? HOT 4
- Out of memory errors? HOT 4
- Server crashes on starting HOT 2
- webpack cannot bundle the worker
- New NPM release and/or build instructions? HOT 2
- Possible serverChunkSize un-awareness of lazyFile's speedup system incompatible with rangemapper db chunking. HOT 3
- Does this vfs support update, delete and insert? HOT 1
- Can't create JS function with create_function()
- Advice on speeding up initial load of a table HOT 1
- [not an issue] Questions on how the plugin works and how to speed up initial worker load HOT 7
- concurrent access HOT 6
- Failing to init WASM with Node.js / Web Worker polyfill / type: `module` HOT 2
- Not compatible with Node.js 16+ HOT 1
- Document workaround for slow HEAD
- Throw an error if server ignores RANGE header
- Any way to optimize this recursive query? HOT 2
- Firefox issue with virtual tables (gitlab pages sqlite.db file hosting) HOT 2
- Fix broken build due to invalid dep graph HOT 2
- Can't import createDbWorker in Javascript ?
- [Showcase] Using it to show how Toki Pona has grown and changed as a language!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sql.js-httpvfs.