rafekettler / magicmethods Goto Github PK

I followed the README:

• edit magicmethods.mkd/magicmethods.tex
• run make docs

However this doesn't build the .html file.

As an aside, why not use just one input script (preferably .markdown), which can then be generated into .tex? In that case, this bug report would be irrelevant.

Examples for __setattr__ and __delattr__ directly use __dict__, while not every object has it. (Not if you’re using __slots__: http://docs.python.org/reference/datamodel.html#slots)

I think it’s better to use eg. super(…).__setattr__ as it always does the “right thing” even on a class with __slots__ or even if you’re inheriting from a class that itself overrides __setattr__.
This guide should show the way.

Here is the adapted code:

class AccessCounter:
    '''A class that contains a value and implements an access counter.
    The counter increments each time the value is changed.'''

    def __init__(self, val):
        super(AccessCounter, self).__setattr__('counter', 0)
        super(AccessCounter, self).__setattr__('value', val)

    def __setattr__(self, name, value):
        if name == 'value':
            super(AccessCounter, self).__setattr__('counter', self.counter + 1)
        # Make this unconditional.
        # If you want to prevent other attributes to be set, raise AttributeError(name)
        super(AccessCounter, self).__setattr__(name, value)

    def __delattr__(self, name):
        if name == 'value':
            super(AccessCounter, self).__setattr__('counter', self.counter + 1)
        super(AccessCounter, self).__delattr__(name)

Also, in the previous code sample:

def __setattr__(self, name, value):
    self.name = value
    # since every time an attribute is assigned, __setattr__() is called, this
    # is recursion.
    # so this really means self.__setattr__(name, value). Since the method
    # keeps calling itself, the recursion goes on forever causing a crash

self.name = value is the same as self.__setattr('name', value), not self.__setattr__(name, value) (string 'name' vs. variable name.)

First off, great article!

In your Building Descriptor Objects example, there is a bug with conversions. See comments:

class Foot(object):
    '''Descriptor for a foot.'''

    def __get__(self, instance, owner):
        return instance.meter / .3048 # same as * 3.2808
    def __set__(self, instance, value):
        instance.meter = float(value) * 3.2808 # needs to be / 3.2808

http://www.reddit.com/r/Python/comments/fb4xz/guide_to_pythons_magic_methods_work_in_progress/c1enahq suggests a table for non-obvious magic method usages, e.g. __iter__ is invoked by for x in container, etc. I'd like to add it as an appendix or something similar.

__index__, __members__, __methods__ are all not documented. I don't know anything about the last two, but they should probably be in the guide.\

__getstate__ and __setstate__, which are used by pickle, are also undocumented

I'd like this to be one of the first results when you google 'magic methods guide python' or 'python magic methods'. Any SEO tips/experts?

You opened a parentheses and didn't close it at the end of the del paragraph, the one that reads "(...) (like always closing a connection when you're done with it. (...)"

Other than that, great job!

"Each of these methods does not return a value, because assignment in Python does not return any value. Instead, they just alter the state of the class."

Special methods that implement augmented assignment must return a value, because the return value is bound as the result of the assignment operation. For example:

>>> class IAdd(object):
...     def __iadd__(self, other):
...         self.foo = other
... 
>>> iadd = IAdd()
>>> iadd += 7
>>> iadd += 7
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int'

Evidence that it's not just me: it seems that Google has dropped http://www.rafekettler.com/magicmethods.html from search results.

Shouldn't be done. Subclass object instead and use composition.

The dir magic method is used by the built-in dir() function. All classes inherit this method, but it can be extended or overridden to control what is shown in a dir() listing. The main use case is for classes that add methods dynamically (ORMs and whatnot).

Also, there is a sizeof magic method called by sys.getsizeof().

And there is subclasshook magic method used to create abstract base classes.

Hello,RafeKettler,I have been done with the translation of this article , the goal of the translation's language is Chinese, since do not konw so much about the License detail and related legal , so what I could do is request your permission to allow me to post the tanslation version of on my blog website.
I's that OK ? or something I need to do with ? may be change the License of the translation version used in my blog ? ( CC BY-SA 4.0 )

Thank you very much!

I could not find an email for you. I just want to say thank you for magic methods article!

noticed in appendix.markdown, might be elsewhere too

This is a very important magic method. It was used to create collections.defaultdict.

In a dict subclass, this method is called whenever a key is missing from a dictionary:

>>> class M(dict):
        def __missing__(self, key):
                print 'Did not find: %r' % key
                return len(key)


>>> m = M()
>>> m['raymond'] = 'red'
>>> m['raymond']
'red'
>>> m['roger']
Did not find: 'roger'
5

From #python:

nosklo: RafeKettler: pickle shouldn't be used for important data, it corrupts, is python only, slow, insecure, and incompatible between versions of python and of your own software

Hello Rafe,

Your doc on magic method was an eye opener for me..

my favorite topic was definitely init, iter and pickes..

I would appreciate if you put example code next to expanantion column in the appendix....

Perhaps add a section on how magic methods changed in Python 3:

• nonzero renamed to bool
• unicode is gone
• div is gone
• coerce is gone
• cmp is gone
• Added bytes

Hello, I am curious as to why __next__ was not included within the containers category or perhaps under a different category, such as, Iterators?

Be sure to mention that adding the hash method usually entails adding a corresponding eq method. The invariant is: a==b implies hash(a)==hash(b).

Right now there's a style contributed by stevelosh on reddit with a minor change by me (body width increased to 800px from 640). If anyone can come up with a better style, please do so.

In section "Copying":

http://www.rafekettler.com/magicmethods.html#copying

Says:

When you want to deep copy an individual attribute, call copy.deepcopy() on that attribute with memodict as the first argument.

Official documentation for Python 3 says:

https://docs.python.org/2/library/copy.html#copy.deepcopy

If the deepcopy() implementation needs to make a deep copy of a component, it should call the deepcopy() function with the component as first argument and the memo dictionary as second argument.

So:

1. It should be nice that the wording "first", "second", etc, match. It was a little bit confusing for me.
2. At least as the way I do understand it, memodict doesn't seem to work as documented:

>>> memodict = {}
>>> data = {'a': [1,2,3]}
>>> from copy import deepcopy
>>> deepcopy(data, memodict=memodict)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: deepcopy() got an unexpected keyword argument 'memodict'

Thanks.

The reflected arithmetic section doesn't explain why I'd want to define reflected arithmetic operators (to make things like 5 + my_object work without redefining int.add, because it's not possible) nor under what conditions they are called (the object on the left must either not define the method or return NotImplemented from the method).

I've got a bit of trouble working this into the English of the section.

Need to add round, floor, ceil, and trunc.

The are used by the math module and the built-in round() function. They are included in float objects and documented in numbers.py

In Python 3, next(obj) calls obj.__next__ (py3)
https://docs.python.org/3/library/stdtypes.html#iterator.__next__

I just corrected some typos I found, but when running the make docs command, it just clean the directory.

I'm not a make file guru, but I would expect at least some errors since I don't have the pdflatex command installed.

The headline __exit__(self, exception_type, exception_value) misses the last argument, traceback.

These are documented in the copy module.

The link in the 'about' section of this repo redirects to this website which I assume is not yours.
The article is available in the HTML format here.

Btw, great article! It helped me a lot. 😄

There needs to be a note about why it should never be used

This important method makes it possible to control/override/extend the new-style string formatting.

For example, you can create a subclass of int that prints numbers with underscores between the digits:

>>> import re
>>> class UnderInt(int):
        def __format__(self, fmtstr):
                mo = re.search(r'(\d+)d', fmtstr)
                if mo:
                        width = int(mo.group(1))
                        return '_'.join(str(int(self)))
                return int.__format__(self, fmtstr)


>>> print 'There are {0:10d} characters'.format(123456)
There are     123456 characters
>>> print 'There are {0:10d} characters'.format(UnderInt(123456))
There are 1_2_3_4_5_6 characters

You missed described next() for python 2.* and next() for python 3.*. This is important methods for iterator protocol.

Sorry to be so nitpicky, but I noticed that you use an empty array as the default parameter for the FunctionalList constructor:

class FunctionalList:
    def __init__(self, values=[]):
        self.values = values

In case people don't know, this introduces a subtle bug. If this constructor is called with the default parameter more than once, the same array is shared among all FunctionalList instances:

l = FunctionalList()
l.append("foo")
l2 = FunctionalList()
l2.append("bar")

for x in l2:
    print x,
# prints "foo bar"

This is easily fixed though. I think you can fix it this way, but I don't know how most people do it.

class FunctionalList:
    def __init__(self, values=None):
        if values is None:
            values = []
        self.values = values

When I visit http://www.rafekettler.com/magicmethods.html

New in version 3.6.

abstractmethod __fspath__()
Return the file system path representation of the object.

The method should only return a str or bytes object, with the preference being for str.
https://docs.python.org/3/library/os.html#os.PathLike.__fspath__

Need to add reduce and reduce_ex

Your articles gives an example of a ContextManager that implements closing a resource. You should probably hightlight the face that contextlib.closing essentially does this (although it doesn't have the AttributeError check, as it's designed to only be used with resources that have a close method)

The description for __call__ reads:

you define __call__ as you would any other function, taking however many functions you'd like it to.

What was meant seems to be “taking however many arguments you’d like it to.”

Summary

Domain Name regististration for rafekettler.com has elapsed.

Upfront

I would like to thank you for putting together this resource. I frequently refer back to it whenever I am stumped by a python problem. And, thanks again, for making it available on github. Had it not been here, it would have been lost to me.

(Well, it could be recorded on the WaybackMachine, but I haven't checked.)

Issue

It appears that registration for the domain name rafekettler.com has elapsed, and that DreamHost is holding it for renewal, cyber-squatting protection.

curl

$ curl --trace-ascii - http://www.rafekettler.com/magicmethods.html
== Info: getaddrinfo(3) failed for www.rafekettler.com:80
== Info: Couldn't resolve host 'www.rafekettler.com'
== Info: Closing connection 0
curl: (6) Couldn't resolve host 'www.rafekettler.com'

Whois

$ whois rafekettler.com
...
   Domain Name: RAFEKETTLER.COM
   Registrar: DREAMHOST, LLC
   Sponsoring Registrar IANA ID: 431
   Whois Server: whois.dreamhost.com
   Referral URL: http://www.DreamHost.com
   Name Server: CPANEL61.FASTDNSSERVERS.COM
   Name Server: CPANEL62.FASTDNSSERVERS.COM
   Status: clientHold https://icann.org/epp#clientHold
   Updated Date: 26-oct-2016
   Creation Date: 25-oct-2010
   Expiration Date: 25-oct-2017
...

This guide is a de-facto reference for all things that surrounded by double underscore, so it would be nice to see __path__ there as well.

Thanks you @RafeKettler for this comprehensive list of Python magic tricks :)

I noticed the __slots__ is missing, although this attribute has huge effects on the class, see https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots

There isn't a concat magic method. Instead add is used for this purpose. The operator module has a concat() function that invokes PySequence_Concat() which accesses the slot used by add .

The explanation is little confusing on this part. I don't know this feature unitll I read this first time from your blog. Since it was not clear I went to the python doc to clear my doubts, there it was explained very optimistically.

I expect you would change that part to make it less ambiguous. Unless this I was enjoyed the details and I bookmarked it for future references.

Here is the link to python doc for convenience :) -
http://docs.python.org/2/reference/datamodel.html#object.__radd__

Thanks for your efforts.

For example:

python >>> not myObject

If the user instantiates multiple Distance objects, the descriptors will interfere with each other.

There are two (and-a-half) ways to fix this:

1. Attach a weakref.WeakKeyDictionary to instances of Meter. Use instance as the key and value as the value. This has the disadvantage that Distance must be hashable and probably should not override __eq__().
2. Place the information in an attribute of the Distance objects. This has the disadvantage of requiring cooperation between the Distance class and the Meter class. This requirement can be eased with the use of a metaclass, but never truly eliminated.
3. (variation of 2) Turn Distance.meter into a @property (or raw attribute) and delete the Meter class entirely.

when I call x = SomeClass(), init is not the first thing to get called. Actually, it's a method called new, which actually creates the instance,

This wasn’t clear to me. What does “actually creates” mean? Is space being allocated within a part of memory/disk that's reserved for the Python interpreter? If so, what do methods look like?

A more concrete list would be preferable, for this reader.

Thanks for writing this!

Note difference, perhaps provide an example of how each one works (current example is more of a data descriptor).
Reference is http://docs.python.org/reference/datamodel.html#invoking-descriptors