Comments (9)
mnunberg
commented on September 23, 2024
1
I've found some leaks, but I want to wait for the command refactor to go in before fixing them
from redisai.
lantiga
commented on September 23, 2024
Hey @mnunberg, feel free to send me a preview of the leaks you found, most of them are probably fixable quickly.
from redisai.
K-Jo
commented on September 23, 2024
Any update on this?
from redisai.
filipecosta90
commented on September 23, 2024
@gkorland @K-Jo @lantiga and @hhsecond .
re-opened this issue with further updates from new tests.
from redisai.
filipecosta90
commented on September 23, 2024
@lantiga on PR #296 I've splitted the modelrun on TF into two tests since they handle different behaviours.
from redisai.
lantiga
commented on September 23, 2024
So, I have a couple of remaining leaks:
This one
==17029== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==17029== by 0x400E809: _dl_map_object_deps (dl-deps.c:438)
==17029== by 0x4014F48: dl_open_worker (dl-open.c:278)
==17029== by 0x596C2DE: _dl_catch_exception (dl-error-skeleton.c:196)
==17029== by 0x40147C9: _dl_open (dl-open.c:605)
==17029== by 0x51DAF95: dlopen_doit (dlopen.c:66)
==17029== by 0x596C2DE: _dl_catch_exception (dl-error-skeleton.c:196)
==17029== by 0x596C36E: _dl_catch_error (dl-error-skeleton.c:215)
==17029== by 0x51DB734: _dlerror_run (dlerror.c:162)
==17029== by 0x51DB050: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==17029== by 0x680FFFF: RAI_LoadBackend_TensorFlow (in /root/RedisAI/bin/linux-x64-cpu/install/redisai.so)
==17029== by 0x680D169: RedisAI_ModelSet_RedisCommand (in /root/RedisAI/bin/linux-x64-cpu/install/redisai.so)
I believe is due to the fact that when an error is generated upon setting a model, and the backend was loaded during the modelset call, the handle to the dlopen is detected as a leak.
This one I couldn't nail yet:
==17061== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==17061== by 0x9776AA3: tfliteRunModel (in /root/RedisAI/bin/linux-x64-cpu/install/backends/redisai_tflite/redisai_tflite.so)
==17061== by 0x9774A1B: RAI_ModelRunTFLite (in /root/RedisAI/bin/linux-x64-cpu/install/backends/redisai_tflite/redisai_tflite.so)
==17061== by 0x680DC59: RedisAI_RunSession (in /root/RedisAI/bin/linux-x64-cpu/install/redisai.so)
==17061== by 0x680DD01: RedisAI_Run_ThreadMain (in /root/RedisAI/bin/linux-x64-cpu/install/redisai.so)
==17061== by 0x55ED6DA: start_thread (pthread_create.c:463)
==17061== by 0x592688E: clone (clone.S:95)
from redisai.
lantiga
commented on September 23, 2024
Running with the last Valgrind configuration + Debug build by @rafie returned a richer output.
This is still WIP.
from redisai.
lantiga
commented on September 23, 2024
Ok, I found the last TFLite leak. The rest of the leak from Valgrind are false positives, or things that we don't need to fix (e.g. related to allocating dlopen handlers, queues, or thread-locals not being freed in time from glibc from within frameworks).
from redisai.
lantiga
commented on September 23, 2024
Closing for now, feel free to reopen
from redisai.
Related Issues (20)
- MTENSORGET HOT 11
- redislabs/redisai gpu docker images not working HOT 4
- Missing tests/test_data/graph.pb ? HOT 1
- dockerwrapper fixes needed to support both ai + others
- AI.DAGEXECUTE and AI.TENSORGET without key = redis crashes HOT 4
- RedisAI arm64 Dockerfile HOT 10
- torch GPU deployment get nvrtc: error HOT 7
- AI.INFO stats 0 calls and duration for `redisAI.createModelRunner`, `redisAI.createScriptRunner` HOT 5
- Shutdown when using saved dump.rdb HOT 3
- [Documentation] AI.SCRIPTSTORE example references AI.SCRIPTSET HOT 1
- Redis AI docker image redis-cli command crash exit while running info modules HOT 2
- Retrieve model chunk size HOT 1
- AI.CONFIG MODEL_CHUNK_SIZE fails on Redis clusters HOT 7
- Looking for dockerfile.jetson with RedisAI 1.2.5 and ONNX backend
- how to use RedisAI in redis cluster? HOT 1
- How to use redisai in redis cluster environment? HOT 1
- Examples on deploy multiple models in a DAG graph? HOT 1
- What is the definition of execute_command? HOT 2
- please provide a PDF of your CLA that I can route for signature
- Responsible disclosure policy HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redisai.