Comments (8)
richardjgowers
commented on August 21, 2024
What are we using pickles for?
from datreant.data.
kain88-de
commented on August 21, 2024
python data pickles. For example if we store python objects.
from datreant.data.
richardjgowers
commented on August 21, 2024
Right yeah, but what is datreant using pickle for? git grep pickle isn't showing me anything?
from datreant.data.
kain88-de
commented on August 21, 2024
datreant.data/src/datreant/data/pydata.py
Line 41 in 89f8937
Sure you are on the right repository?
from datreant.data.
richardjgowers
commented on August 21, 2024
Ah right yeah, I was looking at core
from datreant.data.
kain88-de
commented on August 21, 2024
Oh well with string handling this is even more broken -.-. We have to check reading/writing with the bytes flags rb/wb to ensure data can be read/written independent of the python version.
from datreant.data.
kain88-de
commented on August 21, 2024
This is to say. I now also encountered that a python2 dump cannot be read by python 3
from datreant.data.
kain88-de
commented on August 21, 2024
https://bugs.python.org/issue6784
So they claim this is resolved. But I honestly haven't found a good solution today to enable reading / writing of pickles between python versions that works problem free. Some simple stiff and builtin types might work but not complex objects.
from datreant.data.
Related Issues (19)
- In order for package to be importable, need to make datreant namespace package HOT 1
- Release 0.6.0 checklist
- Update docs HOT 1
- set compression for hdf5 data storage
- Allow globbing syntax for Data.remove(). HOT 1
- Add simple query for data elements in a Treant. HOT 1
- hide pytables naming NatrualNameWarnings HOT 3
- install instructions HOT 1
- Keep track of last modification time HOT 5
- Direct access to data file path HOT 3
- Storing numpy array with same key as existing pandas object puts both in same directory
- acces data member via `__getattr__`
- pydata
- pytable warns about possible performance issues with lots of columns for dataframes
- Problem when saving a dataframe that contains pyobjects HOT 1
- allow to store xarray objects HOT 2
- do a groupby/filter on data keys
- AttributeError: module 'pandas' has no attribute 'Panel4D'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datreant.data.