A python decorator is a function (or class) that returns another function usually after applying some transformation to it. Common examples of decorators are classmethod() and staticmethod().

Decorators have a barrage of uses ranging from memoization, profiling, access control to function timeouts. There is a collection of these and other decorator code pieces here.

I have found myself mostly using decorators for retry and occasionally timeouts for network centric operations. Normally I would have a single decorator function that deals with particular exception and does several retries before re-raising the exception.

Here is an example.

import time
from functools import wraps
from http.client import BadStatusLine

def if_http_errors_retry(func):
    """Decorator: retry calling function func in case of http.client errors.

    Decorator will try to call, the function three times, with a ten seconds
    delay between them. If the retries get maxed out, the decorator will raise
    the http error.

    args:
        func (function): function to be decorated.

    returns:
        func (function): decorated function

    """
    @wraps(func)
    def wrapper(*args, **kwargs):
        """func wrapper"""
        error = Exception
        for _ in range(3):
            try:
                return func(*args, **kwargs)
            except BadStatusLine as err:
                error = err
                time.sleep(10)
                continue
        raise error

    return wrapper

Looking keenly at the above decorator, you will realise that it bears some design flaws:

You can't specify the amount of time delay: it's hard coded

You can't specifiy the number of retries: it's hard coded

It's hard to generalize the decorator for another exception without entirely duplicating it

After noting how much my code stinks I decided to refactor it and deal with the three issues above. The motivation for this mainly came when I saw this retry template.

def auto_retry(n=3, exc=Exception):
    for i in range(n):
        try:
            yield None
            return
            except exc, err:
                # perhaps log exception here
                continue
    raise # re-raise the exception we caught earlier

It's not a decorator at all but it had all the three qualities that my decorator lacked and it gave me a basis for me to start exploring.

It turns out refactoring the decorator was pretty easy.

import time
from functools import wraps
from http.client import BadStatusLine

def auto_retry(tries=3, exc=Exception, delay=5):
    def deco(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for _ in range(tries):
                try:
                    return func(*args, **kwargs)
                except exc:
                    time.sleep(delay)
                    continue
            raise exc
        return wrapper
    return deco

# decorating

@auto_retry(tries=3, exc=BadStatusLine, delay=5)
def network_call():
    # some code using http.client

Voila, now I have a very flexible retry decorator that can be applied on any type of exception, time delay and number of retries.

The key to achieving this is having a function (auto_retry) that returns a decorator function (deco) which will in turn decorates a function (func). Thanks to the power of [closures] the parameters passed to the high order function (auto_retry) are also available to the nested functions and are used for the control flow within them.

[closures]: http://en.wikipedia.org/wiki/Closure_(computer_programming