in libgfapi docs say that fucntion glfs_set_volfile_server may by called multiple times (https://github.com/gluster/glusterfs/blob/master/api/src/glfs.h#L141-L192)

Is there a memory leak in the API? Running the script below from the python interpreter results in a very high memory usage that stays high until exiting the python interpreter.

#foo.py

from gluster.gfapi import Volume

def main():
    for _ in xrange(20):
        v = Volume("master", "glustervolume")
        v.mount()
        v.umount()

if __name__ == "__main__":
    main()

This can also be seen using the vprof memory profiler

$ vprof -c cmh -s bar.py

Or am I missing something?

hi,
when i used gfapi in docker container,it raise a os.strerror excepiton.
here is the code and error message:

volume=gfapi.Volume('192.168.6.151','aifs')
volume.mount()
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/python3.6/lib/python3.6/site-packages/gluster/gfapi/gfapi.py", line 772, in mount
(self.fs, os.strerror(err)))
gluster.gfapi.exceptions.LibgfapiException: glfs_init(31859440) failed: Invalid argument
can anyone help me?

I don't see much action since last release two years ago.

I understand that all the important development happens in the C library code, but just wanted to make sure that this binding still works with the last versions of libgfapi.

Thank you.

We've obtained very high memory usage produced by gfapi.Volume when mounted to big volume (with large bricks count). There are few experiment results, showing memory used by python process mounted to different envs:
Before mount (VSZ / RSS): 212376 / 8932
(2 nodes) 12 bricks volume : 631644 / 21440
(6 nodes) 384 bricks: 861648 / 276516
(10 nodes) 600 bricks: 987116 / 432028

Almost half GB per process just on start! And even more when actively used. As we are planning to run near 100 client nodes with 50 processes per node, amount of memory needed becomes fantastic.
Is there any reason for gfapi to use so much memory to just mount the volume?
Does that mean that server-side scaling up requires corresponding scaling up of client side?

Hello,

It seems these bindings are leaking a lot of memory. Walking through a two level tree with 87 000 files adds a never freed 60 Mbytes memory used to my application. It's getting worst then while running fopen (I'd say around 3Mbytes per 100 fopen).

Is that known ?

Regards, Adam.

Hello,

In case you want to look at a working branch in py33/34, have a look at https://github.com/NotSqrt/libgfapi-python/tree/python3-support

I assume it's not mergeable as is, you'll want to follow your workflow.

I started from the work of @asivokon

Regards

Hi,

python-glusterfs-api.spec contains a version number, but there are no versioned tags in the git repository. Could you create the tags for the versions that were released?

Thanks!

As of today, users can only do a source install. When packaged and submitted to pypi, users should be able to do a pip install from any distribution.

Error using libgfapi on arm64 machine, C gfapi is normal.
Hopefully the libgfapi can be updated synchronously.
Is this binding up to date with the current version of GlusterFS?

The following test (specifically, test_single_thread) segfaults, though, presumably, it should not:

import unittest
import threading
import os

from gluster import gfapi


class GlusterCrashesTestCase(unittest.TestCase):

    def test_normal_connection(self):
        v = gfapi.Volume('glusterserver', 'gv', log_file='/tmp/gluster.log')
        v.mount()
        assert v.mounted
        assert isinstance(v.listdir('/'), list)  # <- this and many other functions work fine

        with v.fopen('test.tmp', 'w') as f:      # <- this will segfault in threaded environment
            f.write(b'test')

        v.umount()
        assert not v.mounted

    def test_single_thread(self):
        # Causes segfault
        thread = threading.Thread(target=self.test_normal_connection)
        thread.start()
        thread.join()


if __name__ == '__main__':
    unittest.main()

Is it possible to use libgfapi-python with threads?

doing a name..decode('utf-8') in listdir fixes this issue

I was seeing a chown in libgfapi. Is there a way to do a chmod on the same ?

In [1]: from gluster import api

AttributeError Traceback (most recent call last)
in ()
----> 1 from gluster import api

/usr/lib/python2.6/site-packages/gfapi-0.0.1-py2.6.egg/gluster/api.py in ()
321 ctypes.c_int)(('glfs_lseek', client))
322
--> 323 glfs_read = ctypes.CFUNCTYPE(ctypes.c_ssize_t,
324 ctypes.c_void_p,
325 ctypes.c_void_p,

AttributeError: 'module' object has no attribute 'c_ssize_t'

OS: centos 6.5
python: 2.6.6

Implement file locking APIs having semantics similar to fcntl.flock() or fcntl.lockf() provided in fcntl module

See: https://github.com/gluster/glusterfs/blob/release-5/api/src/glfs.h#L486-L633

Which version(s) of GlusterFS should this work with this?

Trying out GlusterFS 3.5 development head on EL6 today gives this:

$ python
Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gluster import gfapi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/gfapi-0.0.1-py2.6.egg/gluster/gfapi.py", line 54, in <module>
    api.glfs_creat.restype = ctypes.c_void_p
  File "/usr/lib64/python2.6/ctypes/__init__.py", line 366, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib64/python2.6/ctypes/__init__.py", line 371, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: glfs_creat
>>> 
$ rpm -qa|grep -i gluster|sort
glusterfs-3git-1.el6.x86_64
glusterfs-cli-3git-1.el6.x86_64
glusterfs-debuginfo-3git-1.el6.x86_64
glusterfs-devel-3git-1.el6.x86_64
glusterfs-fuse-3git-1.el6.x86_64
glusterfs-geo-replication-3git-1.el6.x86_64
glusterfs-libs-3git-1.el6.x86_64
glusterfs-rdma-3git-1.el6.x86_64
glusterfs-regression-tests-3git-1.el6.x86_64
glusterfs-server-3git-1.el6.x86_64
$

What is the best way to copy files and directories between the GlusterFS and the local machine using the API?

In python 3.6.2 when writing a string to a file directly like this:
volume = gfapi.Volume('server', 'volume') volume.mount() with volume.fopen('testfile', 'w') as f: f.write("test write")

The resulting file gets weirdly cut like this:
xxd testfile 00000000: 7400 0000 6500 0000 7300 00 t...e...s..

This works:
with volume.fopen('testfile', 'w') as f: f.write(b'test write') xxd testfile 00000000: 7465 7374 2077 7269 7465 test write

Maybe the write function should throw a type error in Python 3 if the object is not a bytearray/byte?

How can I use the API without running as root?

Using the example here in a file called foo.py I get the following results

$ python foo.py
Traceback (most recent call last):
  File "foo.py", line 5, in <module>
    volume.mount()
  File "/usr/local/lib/python2.7/dist-packages/gfapi-0.0.1-py2.7.egg/gluster/gfapi.py", line 563, in mount
    (self.fs, os.strerror(err)))
gluster.exceptions.LibgfapiException: glfs_init(42493440) failed: Transport endpoint is not connected

$ sudo python foo.py
Winter is coming.

reading any multimedia file from volume only reads a part of file.
fget size give complete size of file but read only reads part of it.
tried on .png and mp3.

Either use glfs_readdirplus() or make use of d_type field returned by glfs_readdir() in all APIs such as listdir(), walk(), rmtree(), copy()

Wither approaches prevent additional stat() call required to determine if a directory entry is a file or directory.

GlusterFS sets d_type field correctly since version 3.7.12