Understanding Decorators in Python

Author: Juanjo Conti

Juanjo is a Systems Engineer. He has been programming in Python for 5 years now and he uses the language to work, investigate and have fun.

Blog: http://juanjoconti.com.ar

Email: jjconti@gmail.com

Twitter: @jjconti

A note about what's next

Next article is not an article properly speaking, it is the relief of a talk that was presented in oral form, not from an audio medium but from my memory! I hope the reader will find it useful.

Introduction

The topics I treated in the talk are the following ones:

The beginning of everything
What is a decorator?
Decorator functions
Decorators with parameters
Decorator classes
Decorating classes

The beginning of everything

Everything in Python is an object

Identity
Type
Value

In Python everything is an object. Integers, strings, lists, tuples and other weirder things: modules are objects, the source code is an object. Everything is an object. EVERYTHING.

Each object has 3 features or attributes: identity, type and value.

Objects

Let's see some examples. Number '1' is an object. Using the built-in id we can find its identity. Its type is int and its value is obviously 1.

>>> a = 1
>>> id(a)
145217376
>>> a.__add__(2)
3

Because it's an object, we can apply to it some of its methods. __add__ is the method that is called when we use the + symbol.

Other objects:

[1, 2, 3]   # lists
5.2         # floats
"hello"     # strings

Functions

If everything is an object, so are the functions.

def greeting():
    print "hello"

We can obtain a function's id using id, access its attributes or even make another name refer to the same function object:

>>> id(greeting)
3068236156L
>>> greeting.__name__
'greeting'
>>> say_hello = greeting
>>> say_hello()
hello

Decorator (non-strict definition)

Let's take a liberty for a moment and say that a decorator is a function d that receives as parameter another function a and returns a new function r.

d: function decorator
a: function to decorate
r: function decorated

We can apply the decorator using a functional notation:

a = d(a)

Let's see how to implement a generic decorator:

Code

def d(a):
    def r(*args, **kwargs):
        # a's previous execution behavior
        a(*args, **kwargs)
        # a's after execution behavior
    return r

We define a function d, our decorator, and in its body is defined a new function r, that we are going to return. In the body of r a is going to be executed, the decorated function.

Now change the comments for code that actually does something:

Code

def d(a):
    def r(*args, **kwargs):
        print "Start of execution of", a.__name__
        a(*args, **kwargs)
        print "End of execution of", a.__name__
    return r

When executing a decorated function with the above decorator, a bit of text will be shown, then the decorated function will be executed and more text will appear as it finishes.

In sum2 we keep the decorated version of sum. Now see what happens when we execute it:

Manipulating functions

def sum(a, b):
    print a + b

>>> sum(1,2)
3
>>> sum2 = d(sum)
>>> sum2(1,2)
Start of execution of sum
3
Stop of execution of sum
>>> sum = d(sum)
>>> sum(1, 2)
Start of execution of sum
3
Stop of execution of sum

Also we can keep directly in sum the decorated version of sum and now the original version will not be longer accessible.

Previous way of applying a decorator is the functional form. We have a nicer one:

Syntactic sugar

As of Python 2.4 the @ notation was incorporated for function decorators.

def sum(a, b):
    return a + b

sum = d(sum)

@d
def sum(a, b):
    return a + b

In the above code snippet you can see two examples where we compare the different ways of applying a decorator.

Next there are some examples of real decorators.

Warning

Counter-example: the evil decorator.

def evil(f):
    return False

>>> @evil
... def something():
...     return 42
...
>>> something
False
>>> something()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'bool' object is not callable

This is a cheater decorator, because instead of returning a new function, it returns a boolean object. Obviously when we try to run it, we get an error.

Chained decorators

Their application is similar to the mathematical concept of function composition.

@register_use
@measure_execution_time
def my_function(some, arguments):
    # function's body

Is equivalent to:

def my_function(some, arguments):
    # function's body

my_function = register_use(measure_execution_time(my_function))

Decorators with parameters

They allow more flexible decorators.
Example: a decorator that forces a return type of a function.

Suppose we want a decorator that converts to string all a function's returns. You could use this way:

@to_string
def count():
    return 42

>>> count()
'42'

How would you implement that? See a first approach:

def to_string(f):
    def inner(*args, **kwargs):
        return str(f(*args, **kwargs))
    return inner

This way works, but think if we can do this in a more generic way. Below you'll find the use form of a decorator named typer:

@typer(str)
def c():
    return 42

@typer(int)
def age():
    return 25.5

>>> age()
25

In fact, typer isn't a decorator, is a decorator factory.

def typer(t):
    def _typer(f):
        def inner(*args, **kwargs):
            r = f(*args, **kwargs)
            return t(r)
        return inner
    return _typer

Note that _typer is the true decorator, the outside function gets a parameter t which is used to define the nature of the decorator to be created.

Now go further and see:

Decorator classes

Features:

Decorators with status.

Better organized code.

The first example is similar to our first decorator function:

class Decorator(object):

    def __init__(self, a):
        self.variable = None
        self.a = a

    def __call__(self, *args, **kwargs):
        # a's previous execution behavior
        self.a(*args, **kwargs)
        # a's after execution behavior

An example of how to use it would be:

@Decorator
def new_function(some, parameters):
    # function's body

Step by step operation:

A Decorator type object is being instantiated with new_function as argument.
When new_function is called the method __call__ of the instantiated object is executed.

We can also apply it using the old notation:

def new_function(some, parameter):
    # function's body
new_function = Decorator(new_function)

Because of these examples you've seen, we can make a more strict definition of decorators:

Decorator (more strict definition)

A decorator is a callable d that gets as parameter an object a and returns a new object r (in general of the same type as the original or with the same interface).

d: object of a type that defines the method __call__
a: any object
r: decorated object

a = d(a)

Decorating classes(Python >= 2.6)

As of Python 2.6, the @ notation is permitted before a class definition. This gives place to the concept of class decorators. Even though if before 2.6 you were able to decorate a class (using functional notation), recently with the introduction of this syntactic sugar the class decorators audience got bigger.

A first example:

Identity:

def identity(C):
    return C

Returns the same class we are decorating.

>>> @identity
... class A(object):
...     pass
...
>>> A()
<__main__.A object at 0xb7d0db2c>

Change a class totally:

def abuse(C):
    return "hello"

>>> @abuse
... class A(object):
...     pass
...
>>> A()
Traceback (most recent call last):
  File "", line 1, in
TypeError: 'str' object is not callable
>>> A
'hello'

Similar to one of the examples you've read at the beginning, this example shows us that a decorator's return has to have a similar interface as the object we are decorating, that way it makes more sense to change the use of the original object, for a changed one.

Replace with a new class:

def replace_with_X(C):
    class X():
        pass
    return X

>>> @replace_with_X
... class MyClass():
...     pass
...
>>> MyClass
<class __main__.X at 0xb78d7cbc>

In the previous case we see that the class was changed completely for a brand new different class.

Instance:

def instantiate(C):
    return C()

>>> @instantiate
... class MyClass():
...     pass
...
>>> MyClass
<__main__.MyClass instance at 0xb7d0db2c>

As last example of class decorators we've seen a decorator that once applied, instantiates the class and links this object to its name. This can be seen as a way to implement the Singleton design pattern, studied in programming. # wikipedia singleton quote

To finish:

Where can I find decorators?

Permissions in Django

@login_required
def my_view(request):
    ...

URL routing in Bottle

@route('/')
def index():
    return 'Hello World!'

Standard library

classmethod, staticmethod, property

Thank you very much!

The talk finished thanking the public for their attention. I take this opportunity to thank César Portela and Juan BC for reading this relief draft.

Data and contact

PyConAr 2010 - Córdoba - 15/10/2010

Comments, doubts, suggestions: jjconti@gmail.com
Blog: http://www.juanjoconti.com.ar
Twitter: @jjconti
http://www.juanjoconti.com.ar/categoria/aprendiendo-python/
http://www.juanjoconti.com.ar/2008/07/11/decoradores-en-python-i/
http://www.juanjoconti.com.ar/2009/07/16/decoradores-en-python-ii/
http://www.juanjoconti.com.ar/2009/12/30/decoradores-en-python-iii/
http://www.juanjoconti.com.ar/2010/08/07/functools-update_wrapper/
Original slides: http://www.juanjoconti.com.ar/files/charlas/DecoradoresPyConAr2010.pdf

Help PET: Donate

blog comments powered by Disqus

Last Change: Sat Jul 9 15:00:35 2011. - This magazine is under a Creative Commons license