kevin-montrose / featherdotnet Goto Github PK

It seems the feather file produced by the library is not loadable in R. I can create a similar file in R and it loads ok in C#.

C# Create Feather

using (var writer = new FeatherWriter("C:\\temp\\simple.cs.feather", WriteMode.Eager))
{
  var a = new string[] { "a" };
  var b = new string[] { "b" };
  writer.AddColumn("a", a);
  writer.AddColumn("b", b);
 }

R Load Feather

t <- read_feather("C:\\temp\\simple.cs.feather")
df <- data.frame(t)

produces the following error:

Error in Table__to_dataframe(x, use_threads = option_use_threads()) : 
  embedded nul in string: '\0'

I suspect there is an alignment / padding issue

if i create the file in R:

library(arrow)
simple <- data.frame("a" = "a", "b"= "b", stringsAsFactors = FALSE)
write_feather(simple, "C:\\temp\\simple.r.feather")

I can load it with no problems in C# and I noticed the data offset for the second column is 24 in the R generated file and only 18 in C# generated file

If for example the column data in feather file is of type byte, then Column.ToArray<int32>()
(or any other type) fails to convert the data.

It appears that this method just memcopies bytes from file into target array without any type conversion.

The only situation when this works is if both (file data and requested array type) are of identical type.

Error: Could not install package 'FeatherDotNet 0.6.0'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6.1', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author

I'm seeing this problem:

Unhandled Exception: System.InvalidOperationException: 
     Encoutered PlatformNotSupportedException trying to create a memory mapped proxy for passed bytes:   
     Named maps are not supported.  
     at FeatherDotNet.FeatherReader.ReadFromBytes(Byte[] bytes, BasisType basis) in C:\Users\kmontrose\Dropbox\FeatherDotNet\FeatherDotNet\FeatherReader.cs:line 120

Apparently Named memory mapped files are a Windows-only feature. On Linux there should be a filesystem entry or something used as a handle to share memory buffers around. See https://github.com/dotnet/corefx/issues/20091

I assume this would probably be reproducible on Windows by running tests inside the WSL Linux subsystem.

Maybe we could (for start) have an option to at least just load data into memory (i.e. without buffer sharing support).