Comments (7)
I think that if you memory map a file, only the parts that are accessed are effectively loaded into physical memory. So memory mapping already functions as a cache.
from tb.
https://en.wikipedia.org/wiki/Memory-mapped_file
from tb.
Also very likely if the same file is memory mapped by multiple processes, the different mappings share their physical memory.
from tb.
You are completely missing the point. Having the files mapped creates enormous problems, especially using 7 piece TBs. Arguing that mapping acts like a cache is a water is wet statement. The point is that mapping buys nothing for typical use cases, but causes extreme memory issues for zero benefit. There is zero reason to try to speed access to DTZs at the expense of RAM that could be used for more productive purposes. I'm running multiple engines using this code and it turns RAM into such a train wreck, I need constant reboots as the system (Windows) can't even recover after the engines are halted. (Converting Leela training data to Stockfish format). The same would be true running say 32 Stockfish's in a match using let say 6 piece TBs. Say I have enough RAM to memory map the WDLs but not the DTZs being probed. Say 8 Stockfish's are in a late stage game mapping DTZs to RAM when the others need to map WDL for use in search. The WDL access times will get clobbered. WDLs need speed not DTZs.
DTZs should NEVER be mapped.
from tb.
You seem to believe that if a file is memory mapped then the whole file is loaded into memory. This is simply not true. Only the pages that are accessed are loaded into (physical) memory. Moreover such pages can be ejected again if they are no longer used. Recall that DTZ files are only accessed at root. In other words very rarely.
from tb.
You seem to believe in the Easter Bunny. Where did I say the whole file is mapped please quote me. You've clearly never used this code as I described above and seem to have a lot of "theory." I have a lot of practice. I have described the problem adequately well, if you don't believe in it - having a religious debate about your "beliefs/theories" is not a good use of my time. I suppose in your beliefs, I "imagine" the problems this causes and imagine that windows will leave my used RAM stuck at nearly 100% for hours after these processes end, and I imagine the immense disk thrashing when other programs need RAM during engine use or after all the engines are closed. I must also imagine my page file being hammered like I have no RAM free just as the OS reports.
Please stop. I know I will, good bye.
from tb.
I think there is something wrong with your setup, or else you are misdiagnosing your issues (note that a modern OS will always try to use 100% of available physical memory).
An exited process will free its allocated memory, so if that doesn't happen then something is seriously wrong. Moreover memory used by a memory mapped file is by definition "file backed memory". So when reclaimed, it will not cause trashing (trashing is caused by data which is swapped out to disc when memory is tight).
from tb.
Related Issues (20)
- Required header has been removed HOT 2
- download
- 7-men generation HOT 62
- PicoChess Error 6 Man Syzygy, 32 Bit HOT 5
- Memory leak in interface/tbcore.c HOT 3
- Could not allocate sufficient memory. HOT 3
- TB probing code for 7men
- 7 men TB checksum MD5 does not match HOT 2
- Wish to re-establish email contact with RdM
- memory issues HOT 1
- Using context mixing instead of huffman encoding or LZP HOT 1
- util.c 303 HOT 1
- UBSAN overflow in the Binomial calculation HOT 1
- pawn generator rtbgenp
- Not able to compile on windows HOT 1
- Probably a probing error for the cloud interface HOT 2
- windows MSYS / WSL compilation HOT 1
- Syzygy 8 HOT 3
- Some warnings. Don't know if need to be addressed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tb.