Comments (6)
from sql.js-httpvfs.
At least with the latest version of Chrome - I haven't check other browsers - it works if the Range requests are identical - which means that the doubled chunks must be doubled in the exactly same way to benefit from caching. I am using a CDN provider that sets tons of headers, I haven't investigated if it is related.
I am afraid that the emscripten multithreading will be a nightmare. I will try the official WASM version - it is very recent - it was announced 6 weeks ago and they have the backing of both the SQLite and Chromium teams - and given that I can spare lots of time - I will probably reimplement your work on top of it.
from sql.js-httpvfs.
Finally, browser caching for HTTP Range
requests seem to be a very strange beast and I still do not understand how and when it does work.
I created https://github.com/mmomtchev/sqlite-wasm-http - an alternative implementation based on the new official SQLite WASM version. It has two different HTTP backends - a sync one which is more or less identical to yours - and a new async one which shares the cache among multiple SQLite workers. The reason for having two backends is that the async backend requires COOP/COEP which are still not available everywhere (Github Pages does not have it for example).
Frankly, for having used both SQLite versions, sql.js
is still the more mature one. There are still very few users of SQLite WASM, but their staff has been very helpful and given the weight that has been put behind it (both SQLite and Google), it will probably overtake it very soon.
from sql.js-httpvfs.
In fact, it seems that the browser cache - at least in Chrome - can handle this - which means that if you create multiple workers - they will share their caches. It is not perfect, but it works.
from sql.js-httpvfs.
Thanks for the heads up, I will look into this in more detail, but my first impression is that this new API has the same problem as the current one - it requires synchronous operation. The only advantage will be to replace sql.js
with the official WASM version from the SQLite project.
from sql.js-httpvfs.
I think adding threading support is out of scope for this project because it's not trivial to implement.
To get concurrent read access you probably have two options:
- Create multiple independent SQLite instances and share the file system cache - like you already said. I'd be a bit surprised if this just worked out of the box though because from what I know browser caching usually doesn't work for Range requests.
- This would be cleaner but more difficult: Compile SQLite emscripten with threading support. Then create multiple threads on the same SQLite database. This should be possible but I'm not sure how hard it would be.
from sql.js-httpvfs.
Related Issues (20)
- Support for Sharding? HOT 4
- Out of memory errors? HOT 4
- Server crashes on starting HOT 2
- webpack cannot bundle the worker
- New NPM release and/or build instructions? HOT 2
- Possible serverChunkSize un-awareness of lazyFile's speedup system incompatible with rangemapper db chunking. HOT 3
- Does this vfs support update, delete and insert? HOT 1
- Can't create JS function with create_function()
- Advice on speeding up initial load of a table HOT 1
- [not an issue] Questions on how the plugin works and how to speed up initial worker load HOT 7
- Failing to init WASM with Node.js / Web Worker polyfill / type: `module` HOT 2
- Not compatible with Node.js 16+ HOT 1
- Document workaround for slow HEAD
- Throw an error if server ignores RANGE header
- Any way to optimize this recursive query? HOT 2
- Firefox issue with virtual tables (gitlab pages sqlite.db file hosting) HOT 2
- Fix broken build due to invalid dep graph HOT 2
- Can't import createDbWorker in Javascript ?
- [Showcase] Using it to show how Toki Pona has grown and changed as a language!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sql.js-httpvfs.