bosswissam / pysize Goto Github PK

P. S. Would you like to have a Travis integration automating publishing to PyPI?

Adding __slots__ to subclass of namedtuple should reduce memory usage as said in https://docs.python.org/3/library/collections.html#collections.namedtuple

I tried it but found adding __slots__ showed more memory:

class PointNoSlots(namedtuple('Point', ['x', 'y'])):
    pass

class PointWithSlots(namedtuple('Point', ['x', 'y'])):
    __slots__ = ()

print(get_size(PointNoSlots(3, 4)))  # 344
print(get_size(PointWithSlots(3, 4)))  # 468

It is caused by two reasons:

• Instance of PointNoSlots contains both __dict__ and extra array, which is not considered in get_size.
• In Python 3 instance of PointWithSlots has no __dict__, but in Python 2 there is a __dict__ propery which costs no memory in instance but results a lot in get_size.

I am taking this example class from the Python Cookbook (D. Beazley & B. Jones, O'REILLY), p.249:

class Date_simple:
    def __init__(self, year, month, day):
        self.year = year
        self.month = month
        self.day = day

class Date_slots:
    __slots__ = ['year', 'month', 'day']
    def __init__(self, year, month, day):
        self.year = year
        self.month = month
        self.day = day

d_simple = = Date_simple(2014, 5, 13)
d_slots = Date_slots(2014, 5, 13)

According to the book, d_slots should have a size of 156 bytes and d_simple should be 428 bytes. I am measuring various implementations and comparing the results:

Python's sys:

>> sys.getsizeof(d_simple)
56
>> sys.getsizeof(d_slots)
64

Your implementation:

>> get_size(d_simple)
411 # seems correct
>> get_size(d_slots)
64 # probably wrong

A similar Stack Overflow implementation:

>> stackoverflow_getsize(d_simple)
411 # agrees with your implementation
>> stackoverflow_getsize(d_slots)
148 # seems correct comparing to book

And I tried adding these two lines in your function:

if hasattr(obj, '__slots__'): # can have __slots__ with __dict__
    size += sum(get_size(s, seen) for s in obj.__slots__ if hasattr(obj, s))

Modified get_size:

>> modified_get_size(d_simple)
411 # good
>> modified_get_size(d_slots)
223 # obviously I'm doing something wrong

Would you mind taking a look into this? Thanks!

Look at the example below

from pyszie import get_size

import sys

a=[1,2,3,4,5,6,7,8,9,10]

sys.getsizeof(a)
# Out[9]: 144

get_size(a)
# Out[10]: 424

Can't sys.getsizeof get the exact size for a list that doesn't contain mutable objects?
Is there a problem of recursive double counting here?

OrderedDict is a special case. It has the same behavior as dict, but being treated as a custom object in get_size

Python 2.7.13 (default, Dec 23 2016, 05:05:58) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pysize import get_size
>>> from collections import OrderedDict
>>> a = OrderedDict()
>>> for i in xrange(1000):
...     a[i] = i
... 
>>> len(a)
1000
>>> try:
...     get_size(a)
... except Exception as e:
...     print(e)
... 
maximum recursion depth exceeded while calling a Python object

Hi. The iterable check on Line 25 doesn't work when encountering types.

I believe this should be changed to

    elif isinstance(obj, collections.Iterable) and not isinstance(obj, (str, bytes, bytearray)):

...F.FF...

FAIL: test_list_of_collections (main.TestPysize)

Traceback (most recent call last):
File "D:/pysize-master/test_pysize.py", line 23, in test_list_of_collections
pysize.get_size(collection_list))
AssertionError: 260 != 248

FAIL: test_no_double_counting (main.TestPysize)

Traceback (most recent call last):
File "D:/pysize-master/test_pysize.py", line 29, in test_no_double_counting
self.assertEqual(pysize.get_size(obj), pysize.get_size(obj2) + 8)
AssertionError: 114 != 118

FAIL: test_slots (main.TestPysize)

Traceback (most recent call last):
File "D:/pysize-master/test_pysize.py", line 119, in test_slots
self.assertEqual(pysize.get_size(s2), pysize.get_size(s1) + 28 + 4 + version_addition)
AssertionError: 60 != 78

---

Ran 10 tests in 0.002s

FAILED (failures=3)