rafekettler / magicmethods Goto Github PK
View Code? Open in Web Editor NEWGuide to Python's magic methods
Home Page: http://www.rafekettler.com/magicmethods.html
Guide to Python's magic methods
Home Page: http://www.rafekettler.com/magicmethods.html
These are documented in the copy module.
New in version 3.6.
abstractmethod
__fspath__()
Return the file system path representation of the object.The method should only return a
str
orbytes
object, with the preference being forstr
.
https://docs.python.org/3/library/os.html#os.PathLike.__fspath__
This guide is a de-facto reference for all things that surrounded by double underscore, so it would be nice to see __path__
there as well.
The description for __call__
reads:
you define
__call__
as you would any other function, taking however many functions you'd like it to.
What was meant seems to be “taking however many arguments you’d like it to.”
Examples for __setattr__
and __delattr__
directly use __dict__
, while not every object has it. (Not if you’re using __slots__
: http://docs.python.org/reference/datamodel.html#slots)
I think it’s better to use eg. super(…).__setattr__
as it always does the “right thing” even on a class with __slots__
or even if you’re inheriting from a class that itself overrides __setattr__
.
This guide should show the way.
Here is the adapted code:
class AccessCounter:
'''A class that contains a value and implements an access counter.
The counter increments each time the value is changed.'''
def __init__(self, val):
super(AccessCounter, self).__setattr__('counter', 0)
super(AccessCounter, self).__setattr__('value', val)
def __setattr__(self, name, value):
if name == 'value':
super(AccessCounter, self).__setattr__('counter', self.counter + 1)
# Make this unconditional.
# If you want to prevent other attributes to be set, raise AttributeError(name)
super(AccessCounter, self).__setattr__(name, value)
def __delattr__(self, name):
if name == 'value':
super(AccessCounter, self).__setattr__('counter', self.counter + 1)
super(AccessCounter, self).__delattr__(name)
Also, in the previous code sample:
def __setattr__(self, name, value):
self.name = value
# since every time an attribute is assigned, __setattr__() is called, this
# is recursion.
# so this really means self.__setattr__(name, value). Since the method
# keeps calling itself, the recursion goes on forever causing a crash
self.name = value
is the same as self.__setattr('name', value)
, not self.__setattr__(name, value)
(string 'name'
vs. variable name
.)
For example:
python >>> not myObject
When I visit http://www.rafekettler.com/magicmethods.html
Domain Name regististration for rafekettler.com has elapsed.
I would like to thank you for putting together this resource. I frequently refer back to it whenever I am stumped by a python problem. And, thanks again, for making it available on github. Had it not been here, it would have been lost to me.
(Well, it could be recorded on the WaybackMachine, but I haven't checked.)
It appears that registration for the domain name rafekettler.com
has elapsed, and that DreamHost is holding it for renewal, cyber-squatting protection.
$ curl --trace-ascii - http://www.rafekettler.com/magicmethods.html
== Info: getaddrinfo(3) failed for www.rafekettler.com:80
== Info: Couldn't resolve host 'www.rafekettler.com'
== Info: Closing connection 0
curl: (6) Couldn't resolve host 'www.rafekettler.com'
$ whois rafekettler.com
...
Domain Name: RAFEKETTLER.COM
Registrar: DREAMHOST, LLC
Sponsoring Registrar IANA ID: 431
Whois Server: whois.dreamhost.com
Referral URL: http://www.DreamHost.com
Name Server: CPANEL61.FASTDNSSERVERS.COM
Name Server: CPANEL62.FASTDNSSERVERS.COM
Status: clientHold https://icann.org/epp#clientHold
Updated Date: 26-oct-2016
Creation Date: 25-oct-2010
Expiration Date: 25-oct-2017
...
In Python 3, next(obj)
calls obj.__next__
(py3)
https://docs.python.org/3/library/stdtypes.html#iterator.__next__
Hello, I am curious as to why __next__
was not included within the containers
category or perhaps under a different category, such as, Iterators
?
This is a very important magic method. It was used to create collections.defaultdict.
In a dict subclass, this method is called whenever a key is missing from a dictionary:
>>> class M(dict):
def __missing__(self, key):
print 'Did not find: %r' % key
return len(key)
>>> m = M()
>>> m['raymond'] = 'red'
>>> m['raymond']
'red'
>>> m['roger']
Did not find: 'roger'
5
Sorry to be so nitpicky, but I noticed that you use an empty array as the default parameter for the FunctionalList constructor:
class FunctionalList:
def __init__(self, values=[]):
self.values = values
In case people don't know, this introduces a subtle bug. If this constructor is called with the default parameter more than once, the same array is shared among all FunctionalList instances:
l = FunctionalList()
l.append("foo")
l2 = FunctionalList()
l2.append("bar")
for x in l2:
print x,
# prints "foo bar"
This is easily fixed though. I think you can fix it this way, but I don't know how most people do it.
class FunctionalList:
def __init__(self, values=None):
if values is None:
values = []
self.values = values
First off, great article!
In your Building Descriptor Objects example, there is a bug with conversions. See comments:
class Foot(object):
'''Descriptor for a foot.'''
def __get__(self, instance, owner):
return instance.meter / .3048 # same as * 3.2808
def __set__(self, instance, value):
instance.meter = float(value) * 3.2808 # needs to be / 3.2808
Be sure to mention that adding the hash method usually entails adding a corresponding eq method. The invariant is: a==b
implies hash(a)==hash(b)
.
Hello,RafeKettler,I have been done with the translation of this article , the goal of the translation's language is Chinese, since do not konw so much about the License detail and related legal , so what I could do is request your permission to allow me to post the tanslation version of on my blog website.
I's that OK ? or something I need to do with ? may be change the License of the translation version used in my blog ? ( CC BY-SA 4.0 )
Thank you very much!
I could not find an email for you. I just want to say thank you for magic methods article!
The headline __exit__(self, exception_type, exception_value)
misses the last argument, traceback
.
The link in the 'about' section of this repo redirects to this website which I assume is not yours.
The article is available in the HTML format here.
Btw, great article! It helped me a lot. 😄
I followed the README:
make docs
However this doesn't build the .html file.
As an aside, why not use just one input script (preferably .markdown), which can then be generated into .tex? In that case, this bug report would be irrelevant.
Note difference, perhaps provide an example of how each one works (current example is more of a data descriptor).
Reference is http://docs.python.org/reference/datamodel.html#invoking-descriptors
The dir magic method is used by the built-in dir() function. All classes inherit this method, but it can be extended or overridden to control what is shown in a dir() listing. The main use case is for classes that add methods dynamically (ORMs and whatnot).
Also, there is a sizeof magic method called by sys.getsizeof().
And there is subclasshook magic method used to create abstract base classes.
Thanks you @RafeKettler for this comprehensive list of Python magic tricks :)
I noticed the __slots__
is missing, although this attribute has huge effects on the class, see https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots
In section "Copying":
http://www.rafekettler.com/magicmethods.html#copying
Says:
When you want to deep copy an individual attribute, call copy.deepcopy() on that attribute with memodict as the first argument.
Official documentation for Python 3 says:
https://docs.python.org/2/library/copy.html#copy.deepcopy
If the deepcopy() implementation needs to make a deep copy of a component, it should call the deepcopy() function with the component as first argument and the memo dictionary as second argument.
So:
>>> memodict = {}
>>> data = {'a': [1,2,3]}
>>> from copy import deepcopy
>>> deepcopy(data, memodict=memodict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: deepcopy() got an unexpected keyword argument 'memodict'
Thanks.
Evidence that it's not just me: it seems that Google has dropped http://www.rafekettler.com/magicmethods.html from search results.
Right now there's a style contributed by stevelosh on reddit with a minor change by me (body width increased to 800px from 640). If anyone can come up with a better style, please do so.
Perhaps add a section on how magic methods changed in Python 3:
The explanation is little confusing on this part. I don't know this feature unitll I read this first time from your blog. Since it was not clear I went to the python doc to clear my doubts, there it was explained very optimistically.
I expect you would change that part to make it less ambiguous. Unless this I was enjoyed the details and I bookmarked it for future references.
Here is the link to python doc for convenience :) -
http://docs.python.org/2/reference/datamodel.html#object.__radd__
Thanks for your efforts.
You missed described next() for python 2.* and next() for python 3.*. This is important methods for iterator protocol.
The reflected arithmetic section doesn't explain why I'd want to define reflected arithmetic operators (to make things like 5 + my_object work without redefining int.add, because it's not possible) nor under what conditions they are called (the object on the left must either not define the method or return NotImplemented from the method).
I've got a bit of trouble working this into the English of the section.
You opened a parentheses and didn't close it at the end of the del paragraph, the one that reads "(...) (like always closing a connection when you're done with it. (...)"
Other than that, great job!
This important method makes it possible to control/override/extend the new-style string formatting.
For example, you can create a subclass of int that prints numbers with underscores between the digits:
>>> import re
>>> class UnderInt(int):
def __format__(self, fmtstr):
mo = re.search(r'(\d+)d', fmtstr)
if mo:
width = int(mo.group(1))
return '_'.join(str(int(self)))
return int.__format__(self, fmtstr)
>>> print 'There are {0:10d} characters'.format(123456)
There are 123456 characters
>>> print 'There are {0:10d} characters'.format(UnderInt(123456))
There are 1_2_3_4_5_6 characters
"Each of these methods does not return a value, because assignment in Python does not return any value. Instead, they just alter the state of the class."
Special methods that implement augmented assignment must return a value, because the return value is bound as the result of the assignment operation. For example:
>>> class IAdd(object):
... def __iadd__(self, other):
... self.foo = other
...
>>> iadd = IAdd()
>>> iadd += 7
>>> iadd += 7
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int'
Shouldn't be done. Subclass object instead and use composition.
From #python:
nosklo: RafeKettler: pickle shouldn't be used for important data, it corrupts, is python only, slow, insecure, and incompatible between versions of python and of your own software
I'd like this to be one of the first results when you google 'magic methods guide python' or 'python magic methods'. Any SEO tips/experts?
If the user instantiates multiple Distance
objects, the descriptors will interfere with each other.
There are two (and-a-half) ways to fix this:
weakref.WeakKeyDictionary
to instances of Meter
. Use instance
as the key and value
as the value. This has the disadvantage that Distance
must be hashable and probably should not override __eq__()
.Distance
objects. This has the disadvantage of requiring cooperation between the Distance
class and the Meter
class. This requirement can be eased with the use of a metaclass, but never truly eliminated.Distance.meter
into a @property
(or raw attribute) and delete the Meter
class entirely.There isn't a concat magic method. Instead add is used for this purpose. The operator module has a concat() function that invokes PySequence_Concat() which accesses the slot used by add .
There needs to be a note about why it should never be used
Your articles gives an example of a ContextManager that implements closing a resource. You should probably hightlight the face that contextlib.closing essentially does this (although it doesn't have the AttributeError check, as it's designed to only be used with resources that have a close method)
http://www.reddit.com/r/Python/comments/fb4xz/guide_to_pythons_magic_methods_work_in_progress/c1enahq suggests a table for non-obvious magic method usages, e.g. __iter__
is invoked by for x in container
, etc. I'd like to add it as an appendix or something similar.
noticed in appendix.markdown, might be elsewhere too
Need to add reduce and reduce_ex
I just corrected some typos I found, but when running the make docs
command, it just clean
the directory.
I'm not a make file guru, but I would expect at least some errors since I don't have the pdflatex
command installed.
Need to add round, floor, ceil, and trunc.
The are used by the math module and the built-in round() function. They are included in float objects and documented in numbers.py
__index__
, __members__
, __methods__
are all not documented. I don't know anything about the last two, but they should probably be in the guide.\
__getstate__
and __setstate__
, which are used by pickle
, are also undocumented
Hello Rafe,
Your doc on magic method was an eye opener for me..
my favorite topic was definitely init, iter and pickes..
I would appreciate if you put example code next to expanantion column in the appendix....
when I call x = SomeClass(), init is not the first thing to get called. Actually, it's a method called new, which actually creates the instance,
This wasn’t clear to me. What does “actually creates” mean? Is space being allocated within a part of memory/disk that's reserved for the Python interpreter? If so, what do methods look like?
A more concrete list would be preferable, for this reader.
Thanks for writing this!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.