Comments (5)
@dynaxis , sounds great!
If you can explain a bit more your use-case, it will help us to understand our users more. Not urgent, when you have time.
And if you want to know more about Oak internals, let me know. Thanks!
from oak.
Hi @dynaxis ,
Sorry for late reply, I was out of office.
Thanks for being interesting in Oak, here are my answers:
Generally it is safe to remove an entry while iterating over the map. However, you will not be able to access that entry via the buffer you have, as it is deleted. If you are using OakUnscopedBuffer
to access that entry, the access that happens after deletion will cause exception (also on other threads accessing same memory). If you are using OakUnsafeDirectBuffer
, you will be able to access, but this is unsafe and may lead to errors, as memory can be reallocated for other use.
But as you say, you delete entry after finishing dealing with it, so it should be fine.
As you noticed, the API Iterator.remove()
is not yet implemented in Oak. Not that it can not be done, we just didn't get to it yet. If you would like to contribute to our open source (to add this API), we can guide you with all needed information :)
Will be happy to answer any other questions!
Thanks,
Anastasia
from oak.
@sanastas Thank you for the answer. I'm currently using Oak for one of my constracted projects, where I want to avoid pulling in something like Redis. So far, it seems doing its job great. Thank you for the great software, and as you stated, I'll try to contribute back as my understanding on Oak internals.
from oak.
I use Oak in a web log processor, which takes from and writes to Kafka topics and eventually to a database. In the course, it tracks the last visited page for each session to decorate logs w/ their previous visits and other contextual information.
Since I wanted it to cope with failures, it creates an OakMap per each Kafka topic partition. And on partition reallocations, OakMaps are created and their states are restored by replaying some of previous logs only for the reallocated partitions (I use log compaction to reduce the amount of logs to replay. Using the topic to mimic WAL with snapshots). As you see, it's single-threaded application in terms of accesses to an OakMap.
There is one more place levaraging OakMap. Old sessions should be evicted to free up memory, and I use one OakMap for each interval of an hour from the current time to, say, 24 hours ago, and organizes sessions into them according to the last log for the sessions. So every hour, I can delete sessions in the OakMap holding the sessions inactive for more than 23 hours.
I didn't want to pull in Redis for this since it is natural and considered efficient to have an in memory map in process. So I picked Oak and am happy with the decision.
from oak.
@dynaxis thank you so much for this detailed explanation!
from oak.
Related Issues (20)
- Add delete-allocate stress test
- Improve Oak test coverage
- Add more common serializers HOT 2
- Ideas for Oak new usage and improvement
- Documentation Review and Additions
- OakHash API HOT 1
- Executers instead of Threads HOT 1
- OakBuffer benchmark
- Delete-Allocate Memory Consumption Benchmark
- Bug found!
- creating git actions for maven
- Add checkstyle to the tests
- clear() throws java.lang.UnsupportedOperationException: remove HOT 1
- oak crash on macos M1 chip
- oak map replace method throw AssertionError
- off-heap memory used by oakMap can not be released HOT 8
- Can I use one OakMapBuilder to create more than one OakMap? HOT 1
- Hi, I find a strange problem about OakMap.remove() HOT 7
- infinite loop in ThreadIndexCalculator HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from oak.