Problem with saving DataFrames to root files when they contain dtype object about root_pandas HOT 9 CLOSED

You need to convert the str columns to the numpy string type S.

However, there seems to be a bug in pandas which is preventing that on reassigment,
I filed this issue here: pandas-dev/pandas#15575

In the mean time, you might need to copy your data:

df2 = pd.DataFrame()
for name, column in df.items():
    if columns.dtype == object:
        df2[name] = column.astype('S')
    else:
        df2[name] = column

The S column type requires a maximum length of the string, be default it will take the longest string
in the series. If you want to give the max length yourself, you can do column.astype('S10') for a max length of 10.

from root_pandas.

They actually consider it a bug that the dtype on new assignments is not object.

from root_pandas.

Hi,

thanks for taking the time. Unfortunately your way of circumventing this issue does not work for me :(
Even if i create this new df2 and call .astype('S') or .astype('S10') it will stay of type object
I tried what you did in pandas-dev/pandas#15575 and also there my output of

import pandas as pd
df = pd.DataFrame({'a': ['Hello', 'World']})
df['a'] = df['a'].astype('S')
df['b'] = df['a'].astype('S')
print(df.dtypes)

is just simply object for df['b'] aswell.

from root_pandas.

Hi All,

has this been resolved at all? I arrived to the same issue of saving string to the root file.

UserWarning: converter for dtype('O') is not implemented (skipping) cobj = _librootnumpy.array2tree_toCObj(arr, name=name, tree=incobj)

Best,
Mat

from root_pandas.

@MatousVozak What is the contents of the column that is an object? Strings, arrays or something else? The issue is inside root_numpy but it's unlikely to be fixed unless you're willing to make a pull request as it has been effectively depreciated in favour of uproot.

That said, a better question is you need to save to a ROOT file? You might be better served using a file format natively supported by pandas like hdf5.

from root_pandas.

Hi @chrisburr, yes it is a string and I needed to save into a root file. I simply wanted to change entries of one branch which was of a type char/string. As this was a hot fix and I couldn't find a quick work around I Eventually turned into a pyroot to do the job.

Best,
Mat

from root_pandas.

Was a workaround ever found for this?

from root_pandas.

Kind ping to @chrisburr ... though you may want to look at the uproot package at this stage?

from root_pandas.

As explicitly written in the README since a while, root_pandas, and root_numpy on which it depends, has been deprecated and effectively unmaintained for quite a while. We decided to close anthing outstanding as "won't do" and archive the package at this point.

from root_pandas.