zopefoundation / extensionclass Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 10.0 422 KB

Metaclass for subclassable extension types

License: Other

Python 44.48% C 43.97% C++ 8.96% Shell 2.59%

extensionclass's People

Contributors

Stargazers

Watchers

Forkers

jean axiros william3johnson inferno-inc global-localhost arpitjain799

extensionclass's Issues

Build / publish wheels on Appveyor

Checklist from zopefoundation/zope.proxy#24:

a copy of appveyor.yml from zope.interface or similar
someone with an Appveyor account (e.g. @mgedmin ) to enable this project
encrypted PyPI upload credentials in the appveyor.yml (no change required from the copy in zope.interface if @mgedmin is the one who enables the Appveyor account)
grant zope.wheelbuilder PyPI rights
push the appveyor.yml commit to master
(optionally) a push of a temporary branch that builds and uploads ExtensionClass 4.3.0 wheels

4.2.0 fails C compilation on windows on py27

see:
http://winbot.zope.org/builders/wineggbuilder/builds/3663/steps/release%20eggs/logs/stdio

Pure-Python ExtensionClass wraps parent but C does not

Commit afb8488 made the C implementation of ExtensionClass.Base not wrap __parent__ objects, but the pure python version still does. I ran into this while working on issue zopefoundation/Acquisition#3. (I'm working around it there for now; if I get time I'll submit a pull request here).

Compare the expected behaviour:

$ python

>>> import ExtensionClass
>>> class I(ExtensionClass.Base):
...     def __of__(self,o):
...         return 'wrapped'
...
>>> x = I()
>>> x.__parent__ = I()
>>> x.__parent__
<__main__.I object at 0x10f3b4290>

With the pure python behaviour:

$ PURE_PYTHON=1 python

>>> import ExtensionClass
>>> class I(ExtensionClass.Base):
...     def __of__(self, o):
...         return 'wrapped'
...
>>> x = I()
>>> x.__parent__ = I()
>>> x.__parent__
'wrapped'

PURE_PYTHON: Name check for setting attributes is applied to all classes, not just C classes

(Tested with 4.5.1, but this goes back to the very oldest code in this repo.)

The Python version of ExtensionClass applies the check for special methods to all classes:

ExtensionClass/src/ExtensionClass/__init__.py

Lines 210 to 219 in 14c7e10

    
           def __setattr__(self, name, value): 
        
               if name not in ('__get__', '__doc__', '__of__'): 
        
                   if (name.startswith('__') and name.endswith('__') and 
        
                           name.count('_') == 4): 
        
                       raise TypeError( 
        
                           "can't set attributes of built-in/extension type '%s.%s' " 
        
                           "if the attribute name begins and ends with __ and " 
        
                           "contains only 4 _ characters" % 
        
                           (self.__module__, self.__name__)) 
        
               return type.__setattr__(self, name, value)

But the intent in the C version is to only apply the check to other built-in types:

ExtensionClass/src/ExtensionClass/_ExtensionClass.c

Lines 477 to 486 in 14c7e10

    
           if (! (type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { 
        
               PyObject* as_bytes = convert_name(name); 
        
               if (as_bytes == NULL) { 
        
                   return -1; 
        
               } 
        
               if (_is_bad_setattr_name(type, as_bytes)) { 
        
                   Py_DECREF(as_bytes); 
        
                   return -1; 
        
               }

This leads to a noticeable difference in PURE_PYTHON mode, such that code that works with the C extension fails in PURE_PYTHON. Compare the C version:

>>> os.environ.get("PURE_PYTHON")
None
>>> from ExtensionClass import Base
>>> class X(Base):
...     pass
...
>>> X.__eq__ = 'added by decorator'
>>>

to the Python version:

>>> os.environ['PURE_PYTHON'] = '1'
>>> from ExtensionClass import Base
>>> class X(Base):
...     pass
...
>>> X.__eq__ = 'added by decorator'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python3.10/site-packages/ExtensionClass/__init__.py", line 214, in __setattr__
    raise TypeError(
TypeError: can't set attributes of built-in/extension type '__main__.X' if the attribute name begins and ends with __ and contains only 4 _ characters

I'm not sure how to reliably detect built-in classes in Python, so I'm not sure how to fix this?

Py_TPFLAGS_HAVE_CLASS in ExtensionClass.h

The PyMem_DEL macro in ExtensionClass.h uses Py_TPFLAGS_HAVE_CLASS unconditionally on both Python 2 and 3, though it's always true and gone in Python 3. Our _compat.h file has a helper macro HAS_TP_DESCR_GET for the same check, which should probably be used:

#define PyMem_DEL(O)                                   \
  if ((Py_TYPE(O)->tp_flags & Py_TPFLAGS_HAVE_CLASS) \
      && (Py_TYPE(O)->tp_free != NULL))              \
    Py_TYPE(O)->tp_free((PyObject*)(O));             \
  else                                                 \
    PyObject_FREE((O));

I'm just not sure if ExtensionClass.h should include _compat.h now or if there's some problem with that. Or if maybe more of this macro could be avoided under Python 3.

Port the C extension to Python 3?

Would it make sense to port the C extension to Python 3, too?

Review Base_getattro implementation for Python 3.7 compatibility

Python 3.7 is introducing a number of new OP codes to speed up method calling. I don't understand all the details, but object.c has a new _PyObject_GetMethod method, which is doing almost the same as _PyObject_GenericGetAttrWithDict, which is the basis for our customized Base_getattro.

The implementation on the Python side is spread out over a couple of issues, the two most relevant I could find are http://bugs.python.org/issue26110 and http://bugs.python.org/issue29263, with the initial patch adding the new method in https://hg.python.org/cpython/rev/64afd5cab40a.

Before we can support Python 3.7, someone should have a detailed look at this. Hopefully if we do nothing everything continues to work. But there might be an opportunity to opt-into the new feature and benefit from the new op code optimizations for classes deriving from ExtensionClass.

warning when installing

src/ExtensionClass/_ExtensionClass.c:800:33: warning: comparison of integers of
different signs: 'Py_ssize_t' (aka 'long') and 'unsigned long'
[-Wsign-compare]
if (typ->tp_basicsize <= sizeof(_emptyobject))

originally reported in zopefoundation/BTrees#174

GHA is not allowed to upload wheels to PyPI

See

I think that the TWINE_PASSWORD has to be set for the repository as it is done for zope.interface.

@jamadden @mgedmin Who has this password? What is the current policy to distribute it to the repositories?

C/Python discrepancy: parent as a class attribute fails in Python, works in C

Given the following example, the first assertion passes in both versions, but the second one fails with the PURE_PYTHON implementation.

# foo.py
from ExtensionClass import Base

class O(Base):
    __parent__ = None

assert O.__parent__ is None
assert O().__parent__ is None

$ python /tmp/foo.py
$ PURE_PYTHON=1 python /tmp/foo.py
Traceback (most recent call last):
  File "/tmp/foo.py", line 7, in <module>
    assert O().__parent__ is None
  File "//python3.8/site-packages/ExtensionClass/__init__.py", line 255, in Base_getattro
    raise AttributeError(
AttributeError: ("'%.50s' object has not attribute '%s'", 'O', '__parent__')

This is true in both Python 2 and Python 3, and PyPy fails the same way CPython does. The logic in Base_getattro appears to be missing a step compared to the C implementation.

Context/use-case: Given class O(Acquisition.Implicit, zope.container.contained.Contained), you can't put instances of O in a container; that raises an AttributeError trying to get the old_parent.

>>> os.environ['PURE_PYTHON'] = '1'
>>> from zope.container.btree import BTreeContainer
>>> from Acquisition import Implicit
>>> from zope.container.contained import Contained
>>> class O(Implicit, Contained):
...     pass
...
>>> BTreeContainer()['key'] = O()
Traceback (most recent call last):
  File "//python3.8/site-packages/Acquisition/__init__.py", line 786, in __getattribute__
    return super(_Acquirer, self).__getattribute__(name)
  File "///site-packages/ExtensionClass/__init__.py", line 255, in Base_getattro
    raise AttributeError(
AttributeError: ("'%.50s' object has not attribute '%s'", 'O', '__parent__')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Sources/zope.container/src/zope/container/btree.py", line 84, in __setitem__
    setitem(self, self._setitemf, key, value)
  File "//zope.container/src/zope/container/contained.py", line 575, in setitem
    object, event = containedEvent(object, container, name)
  File "//zope.container/src/zope/container/contained.py", line 311, in containedEvent
    oldparent = object.__parent__
  File "//python3.8/site-packages/Acquisition/__init__.py", line 792, in __getattribute__
    _reraise(AttributeError, AttributeError(name), tb)
  File "//python3.8/site-packages/Acquisition/__init__.py", line 74, in _reraise
    raise value.with_traceback(tb)
  File "//python3.8/site-packages/Acquisition/__init__.py", line 786, in __getattribute__
    return super(_Acquirer, self).__getattribute__(name)
  File "//python3.8/site-packages/ExtensionClass/__init__.py", line 255, in Base_getattro
    raise AttributeError(
AttributeError: __parent__

Does not build on Python 3.11.0a2

What I did:

tox -epy311

What I expect to happen:

Successful build and test run.

What actually happened:

$ tox -epy311
py311 create: /.../ExtensionClass/.tox/py311
py311 develop-inst: /.../ExtensionClass
ERROR: invocation failed (exit code 1), logfile: /.../ExtensionClass/.tox/py311/log/py311-1.log
================================================================================================= log start ==================================================================================================
Looking in indexes: https://pypi.org/simple, https://mihowitz:****@development.verdi.de/devpi/verdi/prod/+simple/
Obtaining file:///.../ExtensionClass
Collecting zope.testrunner
  Using cached https://development.verdi.de/devpi/verdi/prod/%2Bf/ae7/fbeb862a36083/zope.testrunner-5.4.0-py2.py3-none-any.whl (216 kB)
Collecting zope.exceptions
  Using cached https://development.verdi.de/devpi/verdi/prod/%2Bf/bb9/8cc07e90ebe59/zope.exceptions-4.4-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: setuptools in ./.tox/py311/lib/python3.11/site-packages (from zope.testrunner->ExtensionClass==4.5.2.dev0) (58.1.0)
Collecting zope.interface
  Using cached https://development.verdi.de/devpi/root/pypi/%2Bf/5db/a5f530fec3f09/zope.interface-5.4.0.tar.gz (249 kB)
Collecting six
  Using cached https://development.verdi.de/devpi/root/pypi/%2Bf/8ab/b2f1d86890a2d/six-1.16.0-py2.py3-none-any.whl (11 kB)
Using legacy 'setup.py install' for zope.interface, since package 'wheel' is not installed.
Installing collected packages: zope.interface, zope.exceptions, six, zope.testrunner, ExtensionClass
    Running setup.py install for zope.interface: started
    Running setup.py install for zope.interface: finished with status 'done'
  Running setup.py develop for ExtensionClass
    ERROR: Command errored out with exit status 1:
     command: /.../ExtensionClass/.tox/py311/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/.../ExtensionClass/setup.py'"'"'; __file__='"'"'/.../ExtensionClass/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /.../ExtensionClass/
    Complete output (33 lines):
    running develop
    running egg_info
    writing src/ExtensionClass.egg-info/PKG-INFO
    writing dependency_links to src/ExtensionClass.egg-info/dependency_links.txt
    writing requirements to src/ExtensionClass.egg-info/requires.txt
    writing top-level names to src/ExtensionClass.egg-info/top_level.txt
    reading manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE.txt'
    writing manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
    running build_ext
    building 'ExtensionClass._ExtensionClass' extension
    creating build/temp.macosx-11.0-x86_64-3.11
    creating build/temp.macosx-11.0-x86_64-3.11/src
    creating build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass
    /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -Isrc -I/.../ExtensionClass/.tox/py311/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/ExtensionClass/_ExtensionClass.c -o build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass/_ExtensionClass.o
    src/ExtensionClass/_ExtensionClass.c:840:16: error: expression is not assignable
      Py_TYPE(typ) = ECExtensionClassType;
      ~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:800:33: warning: comparison of integers of different signs: 'Py_ssize_t' (aka 'long') and 'unsigned long' [-Wsign-compare]
              if (typ->tp_basicsize <= sizeof(_emptyobject))
                  ~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~
    src/ExtensionClass/_ExtensionClass.c:985:32: error: expression is not assignable
      Py_TYPE(&ExtensionClassType) = &PyType_Type;
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:995:22: error: expression is not assignable
      Py_TYPE(&BaseType) = &ExtensionClassType;
      ~~~~~~~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:1003:42: error: expression is not assignable
      Py_TYPE(&NoInstanceDictionaryBaseType) = &ExtensionClassType;
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
    1 warning and 4 errors generated.
    error: command '/usr/bin/clang' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /.../ExtensionClass/.tox/py311/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/.../ExtensionClass/setup.py'"'"'; __file__='"'"'/.../ExtensionClass/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

What version of Python and Zope/Addons I am using:

MacOS 11.6.1
Python 3.11.0a2

PURE_PYTHON: empty docstring not respected

>>> from ExtensionClass import Base
>>> class B(Base): ""
... 
>>> B.__doc__
'The most base type'

Release 4.1?

Hi there,

do you already have plan when you are going to do a 4.1 release? We're using 4.1a1 in our testing scenarios and quite like it.

So do you have a timeline yet?

Regards,
Martin

	def __setattr__(self, name, value):
	if name not in ('__get__', '__doc__', '__of__'):
	if (name.startswith('__') and name.endswith('__') and
	name.count('_') == 4):
	raise TypeError(
	"can't set attributes of built-in/extension type '%s.%s' "
	"if the attribute name begins and ends with __ and "
	"contains only 4 _ characters" %
	(self.__module__, self.__name__))
	return type.__setattr__(self, name, value)

	if (! (type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
	PyObject* as_bytes = convert_name(name);
	if (as_bytes == NULL) {
	return -1;
	}

	if (_is_bad_setattr_name(type, as_bytes)) {
	Py_DECREF(as_bytes);
	return -1;
	}