More functional style python: decorators

Although I'm no longer the Test Scripting Lead at my job, I still get a lot of questions about python, especially since many of the engineers at my job are new with python.  I thought it would be a good idea to put down a lot of this code so that there's a permanent repository of it.

I also decided that I wanted to get better at functional style programming, so I'm still learning (and teaching to others) a more functional approach to programming.  I still have a long way to go, but hopefully this will help others who are also trying to learn a functional style from a multi-paradigm language like python.

Decorators:
Decorators are a nifty concept in python which in a nutshell are functions which take a function as a parameter, and return a modified version of the function.  It's kind of a fancy wrapper with syntactic sugar thrown on top.  I'm only going to show function decorators, though class decorators are possible too.

271 def assertKwds( key_reqs ):
272     '''
273     Tests that the passed in keyword args match what is required
274     
275     For example, if the function definition is (age, employed=False)
276         then key_reqs would be { "employed" : type(bool) }
277     
278     *args*
279         key_reqs(dict)- is a dictionary of keyword to type.  
280         
281     *usage*::
282         
283         @assert_kw( { "employed" : type(bool) }
284         def somefunc(name, employed=False):
285             if employed:
286                 print "{0} is employed".format(name)
287                 
288     '''
289     def wrap( fn ):
290         def wrapper(*args, **kwds):
291             ## check keyword args.  Also check that we didn't make a 
292             ## faulty assertion error with a bad key_req
293             failed = False
294            
295             try:        
296                 for k,v in key_reqs.items():
297                     if type(kwds[k]) != v:
298                         msg = "Invalid type {0} for keyword {1}. Should be {2}"
299                         print msg.format(type(kwds[k]), k, str(v))
300                         failed = True
301                 if failed:
302                     return None          
303             except KeyError as ke:
304                 msg = "Faulty assertion. keyword arg {0} does not exist"
305                 print msg.format(ke.args[0])
306                 return None
307             
308             return fn(*args, **kwds)
309         return wrapper
310     return wrap

So this is a decorator that can be used to check keyword args.  Let's see how you would use this function

624     @assertKwds( { "name" : str, "company" : str, "years" : int } )
625     def showWorkInfo( self, name="Sean", company="Wonderland", years=0):
626         msg = "{0} has worked at {1} for {2} years"
627         self.logger.info(msg.format(name, company, years))
628         return 1

Here we have defined a function showWorkInfo, but what's that funny @assertKwds on top of it?  That's the special decorator syntax.   Lets look at that example above.

On line 624, the showWorkInfo function and its arguments is passed to assertKwds.
On line 290, the arguments passed to showWorkInfo are examined in assertKwds
On line 296, the arguments passed to assertKwds are used to compare against the args from showWorkInfo.

If any of the arguments don't match the type requirements, then we don't even call the function and return None.  Otherwise call and return showWorkInfo (line 308).


People tend to think of passing in functions to functions as something you do for a callback.  But you can do other things than callbacks.  In functional programming, it is not uncommon for a function to modify a function.  For example, partial applications can be done so that if you have a function that takes 3 arguments, but you only have two arguments ready, you can return a function that only requires that one other argument with the two others fixed.  You can also perform currying, where an argument that takes several arguments can be morphed into a chain of functions each with only one argument.

Decorators are kind of a poor-man's macro.  They allow you to inspect arguments, modify arguments, modify (or even create) new functions, modify return values, or handle returns.  This example showed how to check the arguments and then call the function.  But you could (just a small list):

1. Check args, then conditionally call function
2. Check args, and conditionally modify args, then call function
3. Check args, conditionally modify args or modify the passed in function itself
4. Generate a new function dynamically based on args
5. Call function, and depending on return value, modify the return value
6. Call function and trap exceptions

Perhaps this last one caught your attention?

252 def genericExceptCatch( extype, handler=None ):
253     '''
254     This is a very handy function that will wrap the exception handling 
255     here instead of the function itself.  This makes for much cleaner
256     code, and the decorator makes it obvious what kind of exception
257     might get thrown
258     '''
259     def wrap( fn ):
260         def wrapper(*args, **kwds):
261             try:
262                 return fn(*args, **kwds)
263             except extype as ex:
264                 declogger.info("Error: {0}".format(str(ex)))
265                 if handler:
266                     return handler(*args, **kwds)
267                 else: return None
268         return wrapper
269     return wrap

And here is an example of how to use it.

656     @genericExceptCatch( KeyError )
657     @genericExceptCatch( AttributeError )
658     def twoExceptions(self, mydict, myobj ):
659         print mydict["somekey"]
660         myobj.nofunc()

Can you see what this is doing?  If you get an exception, then it will call a handler that takes the same arguments as the called function.  This allows you to move exception handling outside of the function that can throw the exception.

When I first started writing decorators, I worried about methods in a class versus regular methods.  But I realized that the code above will work with either.  The only trick is that for member functions, you may sometimes want to look at args[0].  Remember, self is the first thing passed to a member function, so you may need to look at the value of args[0] from *args.