sbentley / qvd-utils Goto Github PK

This looks like a nice candidate for a qvd lib crate for crates.io.
Are you planing to publish the / a crate as a rust library?
This would literally be the first one.
Thanks

Hi there,

Thanks for this tool. It has been really helpful!

I'm just having issues regarding Chinese/Japanese/Korean characters. Whenever they are encoded, they look garbled. I'm not sure if it's a UTF-8 parsing in the Rust or Python side.

Steps to reproduce

I've attached the sample input and output for this issue. Here's the minimum reproducible example

from qvd import qvd_reader
df = qvd_reader.read("sample_data.qvd")
df.to_csv("sample_data.csv")

Input

Download link (expires after 24h):
https://wormhole.app/wjll0#vX3tLuzrucIFtO-LaWRpgg

Output
sample_data.csv

Hello!
Please compile the library for python 3.6 and glibc 2.5 (<2.17) if possible

Hi, can't install package

Python 3.11.5:

Not familiar with Rust but was hoping somebody knew how to implement something like 'usecols' in Pandas? I have some really wide tables that would be much faster to read if I could ignore all but two of those columns. Otherwise great library, super useful for Qlik integration!

Hi Dear SBentley

I'm triying to read a QVD file with 2.446.709 rows, but the system show me error message "index out of bounds: the len is 2446709 but the index is 2446709", i tried with QVDs less than 2M rows and worked well

Thanks

Most of the fields in Qlikview and ALL the fields in QlikSense are of type DUAL
Dual is like an array/struct where first part is string and second is number

For example the command Now() in qlik generates a dual where the 1st part is a string in the default format specified in the qlik script , for example, '24/02/2023 20:32:43', and the second is the corresponding number 44981,856053241

DUAL fields can be generated manualy too : for example with DUAL('whatever',23).
I did a test: I created a QVD with just a field and a row containing DUAL ... and unfortunately I got just the text part

Thanks for this amazing work! I was wondering whether a qvd_writer package was possible, based on your current work. I'm not familiar with Rust, so there are maybe caveats I don't see. If you think there are not, I might take the time to learn a bit of Rust to write a PR.

Hi,

I get this error when I try to read qvd file:

ValueError: arrays must all be same length

Thanks

Hey guys,
I was very happy to found this project. Unfortunately there is an issue on my OS X with Python 3.8.10

pip install qvd
ERROR: Could not find a version that satisfies the requirement qvd (from versions: none)
ERROR: No matching distribution found for qvd

Any ideas?

Best regards,
Axel

Looks very good Sam. To use this library will be good if you can insert a licence file so that users know how to distribute the code

Hello!
Please compile the library for python 3.11 if it's possible, thank you so much for making this tool

Hi @SBentley , would you be interested in adding a functionality to interface this repository with Dask ? We can work on this

Hi there - loving this library, so handy for reading QVD files outside of the Qlik environment.

What would be great is to transfer the field types defined in QVD to the resulting DataFrame. So that numeric columns come over as ints/floats, dates/timestamps - as dates/times, etc. Nothing fancy but a simple mapping between QVD and Pandas data types would be very helpful in working with the resulting data frames.

Is there a way to get the metadata ?
or there's the need to expose some functions to achieve it?

Would it be possible to loosen the constraint of "pandas~=1.1.4", so pip can resolve the packages even with newer pandas versions?
In my case I would require pandas 1.3.2 that comes with AWS Glue 3.0.

First thank you for this library!

Is there a chance that it can support BytesIO content?
I'm getting the error:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "*****************/env/lib/python3.9/site-packages/qvd/qvd_reader.py", line 6, in read
    data = read_qvd(file_name)
TypeError: Can't convert <_io.BytesIO object at 0x1361585e0> to PyString

This is my code:

from io import BytesIO
from qvd import qvd_reader

qvd = BytesIO()
with open('file.qvd', 'rb') as f:
   qvd.write(f.read())

qvd.seek(0)
content = BytesIO(qvd.getvalue())

df = qvd_reader.read(content)

Thanks in advanced...