It should be possible to write a test harness that makes sure every transaction used by the Persistent data types is failure-tested to make sure everything is rolled back correctly (that is, that everything that needs to be snapshotted is).

For example, DateTime objects are designed to fit in a certain number of bytes, so they could be stored directly instead of serialized to string or pickle.

Just checking if this is the latest version of pynvm? It may not be working against the latest version of pmdk (built via vcpkg on Windows 10 & VS2017)

During pip install, I need to manually rename the pmdk obj files to remove the "lib" part of the name.

Running the account demo, the created file is jumbled unicode - 祭捡潣湵獴瀮敭潭橢漀獲 whereas it should be myaccounts.pmemobj. Running a second time fails.

It may be related to the following warning during compile time...

build\temp.win-amd64-3.6\Release_pmem.c(933): warning C4133: 'return': incompatible types - from 'const wchar_t *' to 'const char *'
(there are approx 36 similar issues relating to wchar_t)

I recall getting this working on Windows a few months ago, but since then my build environment was wiped :-)

This should be guided by profiling, but pretty clearly things like the dictionary key lookup should be recoded in C (the existing CPython code will be helpful in this process, but note that it is the 3.5 code and note the 3.6 code I used as a model for the python code).

Once "enough" features are implemented (which I would say would be the default serializer/whitelist, support for PersitentObject, and the expanded docs), we should cut a new pynvm release and upload it to pypi. We will need perone's help with that, as I presume he is the current owner of the pypi project.

The current examples in https://pynvm.readthedocs.io/en/v0.3.1/getting_started.html regarding libpmem library use a call pmem.map which doesn't seem to exist.

I suppose that it should be pmem.map_file, which seems to be similar (https://pynvm.readthedocs.io/en/v0.3.1/pmem.html#nvm.pmem.map_file). However, the documentation for map_file only explains three of the four parameters, and they are all needed (that is what I see in Python 2.7 and pynvm==0.3.1, which is my current environment).

I am unable to execute the tests nor I fully understand the meaning of the mode parameter. I don't know if this is only a documentation problem or the call is expected to have optional parameters.

from nvm.pmemobj import PersistentObjectPool

data = np.load('/data/data/feature_100000_mat.npy')
with PersistentObjectPool('hello_world.pmem', flag='c') as pool:
if pool.root is None:
pool.root = data

Traceback (most recent call last):
File "", line 3, in
File "/usr/lib64/python2.7/site-packages/nvm/pmemobj/pool.py", line 795, in root
oid = self.mm.persist(value)
File "/usr/lib64/python2.7/site-packages/nvm/pmemobj/pool.py", line 491, in persist
raise TypeError("Don't know how to persist {!r}".format(cls_str))
TypeError: Don't know how to persist 'numpy:ndarray'

Like a more traditional DB, pynvm applications are probably going to want to version their persistent data "schema", and provide automatic migrations when the schema changes. pynvm.pmemobj can facilitate this at a minimum level by providing an 'application version' string stored in the root object and exposed as a property of the pool object, and a way to treat a version mismatch as an error (and a way not to, so one can do a migration).

We could also go further and provide a migrations infrastructure. If we choose to do the latter we should look at best practices in the SQL realm. This ticket is really only about the minimal version support., though.

Also find out how much of a performance boost it gives. pminvaders2 should be a good test example for this.

The preone repo has coveralls/Travis CI enabled, we should enable it for this repo as well. That's mostly a matter of turning it on, but I haven't tested the code on pypy yet and py3 should be added to the versions tested in the .travis.yml file.

Also FrozenSet, which should almost fall out of the implementation.

Currently no locking is done. The Python C code I've translated assumes it holds the global interpreter lock, which is not true for this code, so appropriate locks need to be added.

From our point of view tuples are an immutable list, so the code should be a stripped down version of the PersistentList code. This is worth adding as a Persistent class so that we aren't going through a pickle/unpickle serialization for the whole tuple.