Comments (5)
Thank you for the hint! The only issue I see is concurrent requests and backups. If you use standard SQL technology, both is not an issue. How would you handle read/write requests from concurrent processes with an embedded store? Probably locking and only run a single instance? For example when the Web frontend is running and the user wants to index new photos on the command-line.
from photoprism.
Yes, depending on which key/value store, large online-backup runs could lock the table for writes - but it generally isn't a problem. Don't let the "key/value" store throw you. Boltdb / LevelDB are vary powerful and both take different approaches to table locking / concurrent use.
LevelDB is a LSM tree so better for loads >10,000 writes/sec, while Bolt uses a B+tree so is optimized for reads. Either of them could easily handle millions of images.
https://github.com/boltdb/bolt#comparison-with-other-databases
http://blevesearch.com/bleve-bench/ can index about 1MB of text/attributes a second and search about 120 queries a second (both only using a single-core)
from photoprism.
Nice, we'll give it a try :) Although I can imagine some users/admins would like to access their data in MySQL, e.g. from a CMS.
from photoprism.
We've recently switched to TiDB as our default database, see #60. Is this enough to implement our search or do we need something else like Bleve? While Bleve looks nice, it seems to be based on the unmaintained BoltDB.
- Does Bleve also properly support LevelDB?
- How to keep indices of TiDB and Bleve in sync without much effort?
- Does TiDB have more features for search than MySQL?
- What would be the specific disadvantage of just using TiDB for now?
All of this needs to be investigated. Help is most welcome! 👍
from photoprism.
Implemented full-text search with SQL only, even without support for FULLTEXT
. So this is not needed anymore.
from photoprism.
Related Issues (20)
- MariaDB: Problems with invalid Unicode characters on a custom instance running on WSL2 HOT 8
- Videos: Support for HTTP Live Streaming (HLS) with M3U8 HOT 1
- Videos: Support workers (like Peertube runners) for distributed transcoding
- Videos: Upgrade FFmpeg to v7.x for faster transcoding and extended file format support
- Only first "Person in Image" is added to People tab when import photo with exif metadata HOT 1
- Places: Include videos in the map-based place browser (the 'Places' view) HOT 2
- Indexing: (Re)index selected photos/videos HOT 1
- Correct thumbnails, after clicking on them the wrong photo is displayed (for some photos) HOT 2
- Metadata: Don't ignore stop words when generating titles from file names HOT 2
- Photoprism gives space error and refused to start
- PhotoSync App Password HOT 1
- UX: Support timeline feature HOT 2
- Image Import seems to ignore visually similar image with different SHA1sums. HOT 2
- Portainer stack setup does not work if you change PHOTOPRISM_DATABASE_PASSWORD from "insecure" to anything else. HOT 1
- Orientation operation, returned error 500 HOT 4
- Search: Enable Title-Based Sorting HOT 3
- New face can't be named – Entity not found HOT 11
- Sharing: Add names to the secret token of the link HOT 1
- Cannot create apps and services HOT 2
- MariaDB binay logs growing more than 1GB per hour after upgrading to Build 240420-ef5f14bc4 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from photoprism.