Today’s blog post is going to contain fairly advanced Python hackery. We’ll
take two functions — one is a wrapper for the other, but also adds some
positional arguments. And we’ll change the signature displayed everywhere from
f(new_arg, *args, **kwargs) to something more
This blog post was inspired by F4D3C0D3 on #python (freenode IRC). I also took some inspiration from Gynvael Coldwind’s classic Python 101 (April Fools) video. (Audio and some comments are in Polish, but even if you don’t speak the language, it’s still worth it to click through the time bar and see some (fairly unusual) magic happen.)
Let’s test it.
The last line is not exactly informative — it doesn’t tell us that we need to
bar as an argument. Sure, you could define
new as just
bar) — but that means every change to
old requires editing
well. So, not ideal. Let’s try to fix this.
The existing infrastructure: functools.wraps
First, let’s start with the basic facility Python already has. The standard
library already comes with
If you’ve never heard of those two functions, here’s a crash course:
If we try to inspect the
square function, we’ll see the original name, arguments,
annotations, and the docstring. If we ran this code again, but with the
@functools.wraps(f) line commented out, we would only see
This approach gives us a hint of what we need to do. However, if we apply
update_wrapper, which is what
wraps ends up calling)
to our function, it will only have
bar as arguments, and its
name will be displayed as
So, let’s take a look at functools.update_wrapper. What does it do? Two things:
copy some attributes from the old function to the new one (
__dict__of the new function
If we try to experiment with it — by changing the list of things to copy, for
example — we’ll find out that the annotations, the docstring, and the displayed name come from
the copied attributes, but the signature itself is apparently taken from
Further investigation reveals this fact about
inspect.signature(callable, *, follow_wrapped=True)
New in version 3.5:
Falseto get a signature of callable specifically (
callable.__wrapped__will not be used to unwrap decorated callables.)
And so, this is our end goal:
Craft a function with a specific signature (that merges
new) and set it as
But first, we need to talk about parallel universes.
Or actually, code objects.
Defining a function programmatically
Let’s try an experiment.
So, there are two ways to do this. The first one would be to generate a string
with the signature and just use
eval to get a
__wrapped__ function. But
that would be cheating, and honestly, quite boring. (The inspect module could
help us with preparing the string.) The second one? Create code objects
To create a function, we’ll need the
gives us a function, but it asks us for a code object. As the docs state,
Code objects represent byte-compiled executable Python code, or bytecode.
To create one by
hand, we’ll need
types.CodeType. Well, not exactly by hand — we’ll end up doing a three-way merge between
def _blank(): pass (a function
that does nothing).
Let’s look at the docstring for
All of the arguments end up being fields of a code objects (name starts with
co_). For each
f, its code object is
f.__code__. You can find the filename in
f.__code__.co_filename, for example. The meaning of all fields can be
found in docs for the inspect module. We’ll be
interested in the following three fields:
argcount— number of arguments (not including keyword only arguments, * or ** args)
kwonlyargcount— number of keyword only arguments (not including ** arg)
varnames— tuple of names of arguments and local variables
For all the other fields, we’ll copy them from the appropriate function (one of
the three). We don’t expect anyone to call the wrapped function directly; as
inspect members don’t crash when they look into it,
Everything you need to know about function arguments
A function signature has the following syntax:
Any positional (non-optional) arguments
Variable positional arguments (
*x, name stored in
Arguments with defaults (keyword-maybe arguments); their value is stored in
Keyword-only arguments (after an asterisk); their values are stored in a dictionary. Cannot be used if
Variable keyword arguments (
**y, name stored in
We’re going to make one assumption: we aren’t going to support a
function that uses variable arguments of any kind. So, our final signature
will be composed like this:
That will be saved into
co_names. The first two arguments are counts —
the first one is
len(1+2+3+4) and the other is
len(5+6). The remaining
CodeType will be either safe minimal defaults, or things taken from
one of the three functions.
We’ll also need to do one more thing: we must ensure
__annotations__ are all in the right places.
That’s also a fairly simple thing to do (it requires more tuple/dict merging).
And with that, we’re done.
Before I show you the code, let’s test it out:
And the end result —
We did it!
PS. you might be interested in another related post of mine, in which I reverse-engineer the compilation of a function: Gynvael’s Mission 11 (en): Python bytecode reverse-engineering