I avoided decorators for so long. First, I pretended they didn’t exist. Then, I treated them like a magic spell—I’d use some of the common ones and simply copy how they’re used in the documentation. And each time I tried to learn how they work, I’d give up pretty quickly.
But eventually, I got there. And once I finally understood how decorators work, my reaction was, “Is that it?” As with many complex topics, the magic goes away once you get it, and what’s left makes perfect sense.
So, let me take you on a journey similar to the one I took to demystify decorators.
I split this decorator quest into seven parts. This article covers Parts 1 and 2. These are the most important parts to understand decorators. Take your time. I’ll publish Parts 3 to 7 in a separate article soon.
Decorators are powerful for adding reusable functionality to functions without the need to define a new function. You’ll see them often in Python, and you may even need to write your own one at some point.
Part 1 introduces one of the main characters in this story—closures. We’ll get to decorators in Part 2 later in this article.
Let’s say you want to keep track of all the arguments you pass to all the calls to print() throughout your code. And no, you don’t want to have to do this manually. You want your code to automatically keep a record of any argument you use when you call print().
Start by defining a new function to replace the built-in print(). Although you could name this new function print, I’ll add a trailing underscore in this example to keep the code focused on what really matters. First, here’s a version that won’t work as intended—the code runs without errors but doesn’t achieve the result you need:
The new print_() function calls the built-in print(), but it also appends the argument to a list named data. Look for the trailing underscore in this article whenever I write print_ or print!
However, data is defined within the function definition. Therefore, data is a local variable. It exists only when you call the function and while the program executes it. It cannot be accessed from anywhere else in the program. This variable is local to each function call.
The code creates a new list each time you call the function. That won’t work. You need a single list that contains all the arguments you pass to all the print_() function calls.
A second option is to define the list data outside the function definition:
The list data now exists in the global scope. And a function can access names defined in the global scope. Therefore, this solution works. Here’s the output from this code:
I love Python
...and also The Python Coding Stack
['I love Python', '...and also The Python Coding Stack']The list data now holds a record of all the arguments you pass to print_().
This works. However, there are some drawbacks to this solution. The list data exists in the global scope and, therefore, can be accessed from anywhere in the code. There’s the danger that another part of your code tries to access and modify this list, leading to bugs and unexpected behaviour:
The list data now includes an object that’s not been used as an argument in print_():
['I love Python', '...and also The Python Coding Stack', 'Oops, modified globally!']You want to eliminate this risk.
Also, you can only use this list with one function, the function print_(). If you want to keep track of arguments passed to other functions, you’ll need to create a separate list with a separate name for each function.
In the first (wrong) solution above, data was local to the function. In this solution, data is global. Ideally, we need something in between these two options.
Let me show the current version of the code again, but I’ll visually highlight the function’s scope:
The ideal solution for this problem—the in-between solution in which data is neither local nor global—encloses the list data along with the function’s local scope. Here’s a visual representation of what we’d like to achieve:
If we could create such a bubble or enclosure that includes the function’s local scope and data, the function would still have access to data even though it’s not a local variable. But the rest of the code won’t be able to access and modify data.
You can achieve such an enclosure by defining another function. Here’s the first step—we’ll make some additions later:
Here are the changes from the earlier code block:
-
You enclose the original function and the definition of the list
datawithin a new function calledprint_with_memory() -
You rename the function
print_asinnerto clarify that this is an inner function, nested within an outer one -
For now, you comment out the final lines since
print_(), the function with the trailing underscore, doesn’t exist (yet)
The outer function, print_with_memory(), has two local names: data and inner. However, there’s a more interesting observation we can make about this code:
The function inner also has access to the variable data.
A function defined within another function—an inner function—also has access to variables defined within the enclosing function. So, inner() also has access to names defined in print_with_memory(). This is called a closure.
Clear? Does it all make sense?
No, I didn’t think so. So let me add a bit more. When you define a standard function—one that has no nesting, like print_() earlier—the function has access to variable names defined in the global scope. A function has access to global variables, but the global scope can’t access local variables defined in functions.
A closure does the same thing but with an extra layer of nesting. The inner function, such as inner(), has access to variables defined in the scope just outside of it—the enclosing scope defined by the outer function print_with_memory(). However, the function print_with_memory() doesn’t have access to the inner function’s local variables. And the global scope doesn’t have access to any names defined either within print_with_memory() or inner().
Let’s look at all the names you created in the latest version of the code.
The outer function, print_with_memory(), has two local variables:
The name of the inner function is also a name defined within the function. Therefore, the name inner is local to the function print_with_memory(). We’ll return to this point shortly.
The inner function, inner(), has one local variable, some_obj. This local variable is the parameter you define in the function’s signature. Parameters are assigned values when you call the function, and therefore, they’re local variables.
The inner function, inner(), also has access to the outer function’s local variable, data. This is the closure concept we discussed earlier.
If you compare your current version of the code—which includes the outer function print_with_memory() and the inner function inner()—with the previous version that only had one function, print_(), you’ll note that it’s the inner function inner() that performs a similar task to the print_() function in the first version.
You need inner() to be accessible in the global scope. You can achieve this by returning inner from print_with_memory():
The outer function now returns the inner function. Note that you don’t call inner within print_with_memory(). There are no parentheses. Instead, you return the function by using just its name without parentheses.
In the global scope, you can now call print_with_memory(). This function returns another function—the inner function. You assign this inner function to the name print_ in the global scope. This is the same name you used in the first version of this code earlier.
Therefore, you can now use print_() to print objects and also keep a record of all the arguments passed to print_(). And the function print_() is the inner function inner() returned by print_with_memory(). There’s a lot of juggling of functions around in this topic!
Right, so the function print_() now has access to the list data, which is defined in print_with_memory(). Recall that this is the characteristic of a closure, which you create when you define an inner function within an outer function.
And each time you call print_(), the function will have access to the same list data. There’s only one list data even when you call print_() several times. The list data is attached to the function object print_ and isn’t created when you call print_(). It’s already there.
Since separate calls to print_() can access the same list data, closures allow calls of a function to communicate with previous and future calls through objects within the closures, such as data in this example.
So, how can you access this list if you need to see what’s inside it? You’ll deal with this properly in Part 2, but here’s a hard way of accessing data through print_. First, let’s look at the .__closure__ attribute:
Note that I’m using the standard built-in print() in the final line rather than the new print_() version.
The output shows that the function print_ has one element in its closure:
(| ,) | As mentioned earlier, the closure is attached to the function object—print_ without parentheses—and not the function call.
The output is a tuple that contains only one element—note the trailing comma, which shows that this is a tuple. You can try adding a second local variable in print_with_memory() and then referring to that second variable within inner(). You’ll see that .__closure__ will contain two elements.
In our case, there’s only one element in .__closure__ and, therefore, you can access it by indexing .__closure__ using the index 0:
This now gives the cell object rather than a tuple containing the cell object. Don’t worry about what a cell object is—it’s not relevant to our discussion:
| All you need to know is that you can show the value using .cell_contents:
This shows the list that contains all the arguments used in all the calls to print_():
['I love Python', '...and also The Python Coding Stack']It’s hard to get to this data. But that’s a good thing. You don’t want the data shared by all the calls to print_() to be easily accessible.
However, you’ll see that you don’t usually need to access this data directly using .__closure__. But more on this later in this series.
We’re at the end of Part 1 in this journey through decorators, and we haven’t even mentioned decorators. Don’t worry. We’ll get to them soon. First, let’s wrap up Part 1 with a few summary-type observations:
-
A closure allows a function to access variables that aren’t in its local scope or in the global scope. A closure has access to the enclosing scope and, therefore, to variables defined in the enclosing (outer) function when creating the closure.
-
A closure permits some data to persist when you call a function. Therefore, each call of the function can “communicate” with previous and future calls of the same function.
Great. Time to talk about decorators now. Closures have other uses in programming, but I introduced them here as they’re central to the discussion on decorators, which is what I’ll focus on!
Let’s look at the code you have so far:
The outer function print_with_memory() creates a “new version” of the built-in print() function and adds some extra functionality. It decorates the print() function—the adornment, in this case, is the ability of the main program to keep track of all the arguments passed to the new function.
This code already has many of the hallmarks of a decorator, but let’s make a few changes to make it a proper decorator. The first issue you need to address is that this function only works with the built-in print(). What if you also want to apply the same treatment to other functions? You don’t want to repeat yourself and define similar decorator-like functions for each case.
The first step is to change the name of the outer function since you want it to apply to any function, not just print():
This doesn’t change how the code works, of course. But the new name of the outer function is more generic. Great. Let’s move on. The inner function still has print() hardcoded within it. So, this code still only works for the built-in print().
Instead of hardcoding print() within the code, you can add a parameter to store_arguments() to represent any function:
There are three changes in this code compared to the previous version:
-
The outer function,
store_arguments(), now has a parameter calledfunc. You can call the parameter anything you like, butfuncis often used to show that you should pass a function to this parameter. -
The
inner()function no longer callsprint(). Instead, it now callsfunc, which is a function—the function you pass when you callstore_arguments(). -
The call to
store_arguments()at the end of the code, when you createprint_, now needs the name of a function as an argument. It’s up to you to specify which function you want to use instore_arguments()since you can now use it for any function.
In this case, you pass print to store_arguments(). Therefore, this code performs the same task as the earlier version with print_with_memory(). You can confirm that this is the case:
The two calls to print_() perform the same task as the standard print(). And the last line allows you to access the list data, which you defined when you created the closure:
I love Python
...and also The Python Coding Stack
['I love Python', '...and also The Python Coding Stack']A quick note before you move on to Step 3. You can assign the function returned by store_arguments(print) directly to the name print, the one without the trailing underscore. This overwrites the built-in print() since the name print would now refer to the inner() function returned by store_arguments(print).
You’ll see in Part 3 of this journey through decorators that reusing the original function’s name is the most common way of using decorators. However, I’ll stick with using different names by adding trailing underscores for now.
But print_() doesn’t quite work like the built-in print(). Not yet. Here’s an example:
The first call to print_() works fine, as it did earlier. However, the second and third calls don’t work. This code raises an error:
Traceback (most recent call last):
File "...", line 13, in
print_(42, 99, 256)
~~~~~~^^^^^^^^^^^^^
TypeError: store_arguments..inner() takes
1 positional argument but 3 were given The second print_() call raises this error, but the third one would also raise a similar error. The error message is also quite informative and helps you understand what’s going on. Notice how, even though the error was raised when you call print_(), the error states that store_arguments. is the problem. The name print_ refers to this function, which is the function named inner that’s a local variable in store_arguments. We’ll return to this later in this series.
Let’s get back to the failed calls. The built-in print() should work with those arguments. You can try to remove the _ to call the standard print() to confirm that all three calls would work.
The built-in print() can accept multiple arguments, and it can also accept a keyword argument using the sep keyword. In fact, it can also accept a few more keyword arguments. Here’s the signature for print():
The inner function must be able to accept these same positional and keyword arguments. You could copy the parameters in the print() signature shown above into inner(). However, you’re trying to make store_arguments() work with any function, not just print(). So, you don’t want to hardcode these specific parameters.
That’s where *args and **kwargs come in handy:
You replace the single argument some_obj with *args and **kwargs since these allow you to capture any number of positional and keyword arguments. Therefore, the inner function inner() can be used to replace any function, no matter how many arguments and what type of arguments that function takes.
You also need to use *args and **kwargs when you call func() within the inner function.
Finally, since the purpose of store_arguments()—we’ll call this a decorator soon—is to store the arguments within the list data, you also replace the argument in data.append(). I chose to use a tuple that contains args and kwargs—args is itself a tuple, and kwargs is a dictionary. I’ll be publishing a series on functions soon that will discuss *args and **kwargs in more detail.
Here’s the output from this code—I reformatted the output of the final list for clarity:
I love Python
42 99 256
10:20:30
[
(('I love Python',), {}),
((42, 99, 256), {}),
((10, 20, 30), {'sep': ':'}),
]The new print_() function now works in all situations where the standard print() would work. And the list that stores all the arguments passed to print_() now contains three elements—three tuples:
-
The first element in the list is
(('I love Python',), {}). This is a tuple that contains a tuple and a dictionary. The tuple—the first element within the outer tuple—representsargsand contains only one element, the string"I love Python". This is the only positional argument you pass toprint_()the first time you call this function. The second element in the outer tuple is an empty dictionary since there are no keyword arguments. -
The second element in the list is
((42, 99, 256), {}). This tuple contains the tuple(42, 99, 256). These values are the three positional arguments you pass toprint_()the second time you call the function. There are no keyword arguments, therefore the second element in the outer tuple is again an empty dictionary. -
The third and final element in the list is
((10, 20, 30), {'sep': ':'}). The positional arguments—the three integers10,20, and30, are included in theargstuple. Thekwargsdictionary now shows the key-value pair"sep": ":". The value associated with the key"sep"is the string containing a colon,":".
So, are you done? Does this finally work? Not quite…
Do you read articles on The Python Coding Stack often? Do you find them useful. I put a lot of time and effort into crafting each article, as you can imagine. If you’re in a position to support this publication further, you can either become a paid subscriber or make a one-off contribution.
Paid subscribers also get exclusive access to The Python Coding Place‘s members’ forum. More Python. More discussions. More fun.
And I’m planning some other events and activities for paid subscribers. More on this soon…
Your support will help me keep this content coming regularly and, importantly, will help keep it free for everyone.
Let’s try using this code with another built-in function, max():
Note how store_arguments() hasn’t changed. That’s the point of the changes you made earlier. You can use store_arguments() with any function.
However, you now pass max when you call store_arguments(), and you assign the function returned by store_arguments(max) to the name max_.
Next, you call max_() three times with different arguments. Note that you now need to use print() to show the result. A reminder that this is now the standard built-in print()!
But here’s the output from this code:
None
None
None
[
((5, 2, 5, 76, 5, 23), {}),
(('Hello', 'Goodbye', 'Au Revoir'), {}),
(('Hello', 'Goodbye', 'Au Revoir'), {'key': at 0x1008e4220>}),
] The code doesn’t raise any exceptions, and the list that’s collecting all the arguments does indeed contain all the arguments.
However, the three calls to max_() return None rather than the maximum value from the arguments you pass to max_(). Let’s see why this happens.
The new function max_() replaces the built-in max(). Recall that the function max_() is the inner function inner() returned by store_arguments().
But this inner function inner() doesn’t have a return statement. And functions that don’t have an explicit return statement return None.
The line func(*args, **kwargs) within inner() is equivalent to max(*args, **kwargs) since func is equal to max. And max() returns a value. However, inner() is discarding the value returned by max(). Let’s fix this:
Now, you assign the object returned by func() to value, and then you return value from inner(). You could merge these lines into one if you wish, but I’ll keep them separate. You’ll see why later in this journey through decorators.
Here’s the output from this code:
76
Hello
Au Revoir
[
((5, 2, 5, 76, 5, 23), {}),
(('Hello', 'Goodbye', 'Au Revoir'), {}),
(('Hello', 'Goodbye', 'Au Revoir'), {'key': at 0x100f8c220>}),
] And since inner() now returns the value returned by the function func—this is max in this case—the new function max_() now also returns the same value as max().
It works. The function max_() returns 76 in the first call, since it’s the largest number. In the second call, max_() uses lexicographic order, and therefore returns "Hello" since uppercase "H" comes after both uppercase "G" and uppercase "A". In the final call, the keyword argument key=lambda x: len(x) guides max_() to order the arguments using the length of the strings. Therefore, "Au Revoir" is the maximum value since it’s the longest of the three strings.
Congratulations! You wrote your first decorator. Even if it looks different to what you think a decorator should look like—you usually see them used with the @ notation, which we’ll discuss in Part 3—store_arguments() is a decorator. We’ll also define what a decorator is later in this decorator journey. But this example is all you need for now.
Let’s confirm that this decorator, store_arguments(), works with any function:
The decorator store_arguments() hasn’t changed. It accepts any function, and it decorates it by adding some functionality. This decorator stores all the arguments passed to all calls to the decorated function.
In this example, you decorate a user-defined function, my_special_function(). The decoration happens when you write the following line:
my_special_function = store_arguments(my_special_function)You’ll learn about a more common shortcut for this line in Part 3. But it’s important to understand this line before introducing the syntactic sugar notation.
On the right-hand side, you call store_arguments() and pass the name of your function, my_special_function.
The call to store_arguments() returns another function—the inner function within the store_arguments() function. This inner function performs the same operation as the original function but also performs additional tasks. Therefore, store_arguments() returns a function that’s similar to the original function—it returns a decorated version.
Finally, you assign this decorated function to the same name as the original function. I’m no longer using a trailing underscore to distinguish the names. In this version, the name my_special_function no longer refers to the original function. Instead, my_special_function now refers to the decorated function.
Here’s the output from this code:
JAMESJAMESJAMES
MAIVEMAIVE
[(('James', 3), {}), (('Maive', 2), {})]The first two lines are the same outputs you would get if you used the original function without decorating it. However, the last line in the output shows you that the arguments you pass to separate calls of the decorated function are stored in a common list.
Here are some final thoughts before we move on:
-
A decorator is a function that accepts another function as an argument and returns yet another function. The function it returns is a decorated version of the function you pass as an argument. (We’ll return to this definition and refine it later in this decorator journey)
-
There are three functions involved in this definition, so it can get a bit complex. The decorator, such as
store_arguments(), is a function. Its argument and its return value are also functions. These two functions perform similar tasks to each other. However, the function the decorator returns is a decorated version of the function you pass as an argument to the decorator.
Still confused? Don’t worry! It will get clearer as you make your way through the rest of this decorator journey.
One of the reasons I think decorators can be hard to understand is because they’re often introduced directly using the most common syntax for decorators. You may have seen this in the wild: it uses the @ notation. But I think this hides how decorators work.
In Parts 1 and 2—that’s this article—I chose not to use this notation at all and to build a decorator step-by-step from first principles.
In the rest of this journey, we’ll add more detail and look at more complex decorators. Here’s what to expect:
-
Part 3: Making decorators easier to use with the
@syntax (but only once you understand Parts 1 and 2 well) -
Part 4: We’ll bring everything together into a new example to consolidate everything we know so far.
-
Part 5: Extending decorators to include parameters, so that you can pass arguments directly to ‘decorators’.
-
Part 6: Using classes as decorators instead of functions
-
Part 7: Decorating classes instead of functions (no, this is not the same as Part 6!)
I’ll publish Parts 3 – 7 in a separate article soon.
Anything not clear? Just ask in the comments or on The Python Coding Place forum.
Image by Erelsa from Pixabay
Code in this article uses Python 3.13
The code images used in this article are created using Snappify. [Affiliate link]
You can also support this publication by making a one-off contribution of any amount you wish.
For more Python resources, you can also visit Real Python—you may even stumble on one of my own articles or courses there!
Also, are you interested in technical writing? You’d like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.
And you can find out more about me at stephengruppetta.com
Further reading related to this article’s topic:
Code Block #1
def print_(some_obj):
data = []
data.append(some_obj)
print(some_obj)
print_("I love Python")
print_("...and also The Python Coding Stack")
Code Block #2
data = []
def print_(some_obj):
data.append(some_obj)
print(some_obj)
print_("I love Python")
print_("...and also The Python Coding Stack")
# Using the built-in 'print' here
print(data)
Code Block #3
data = []
def print_(some_obj):
data.append(some_obj)
print(some_obj)
print_("I love Python")
print_("...and also The Python Coding Stack")
data.append("Oops, modified globally!")
# Using the built-in 'print' here
print(data)
Code Block #4
data = []
def print_(some_obj):
data.append(some_obj)
print(some_obj)
print_("I love Python")
print_("...and also The Python Coding Stack")
# Using the built-in 'print' here
print(data)
Code Block #5
data = []
def print_(some_obj):
data.append(some_obj)
print(some_obj)
print_("I love Python")
print_("...and also The Python Coding Stack")
# Using the built-in 'print' here
print(data)
Code Block #6
def print_with_memory():
data = []
def inner(some_obj):
data.append(some_obj)
print(some_obj)
# print_("I love Python")
# print_("...and also The Python Coding Stack")
#
# print(data)
Code Block #7
def print_with_memory():
data = []
def inner(some_obj):
data.append(some_obj)
print(some_obj)
return inner
print_ = print_with_memory()
print_("I love Python")
print_("...and also The Python Coding Stack")
Code Block #8
# ...
print_ = print_with_memory()
print_("I love Python")
print_("...and also The Python Coding Stack")
print(print_.__closure__)
Code Block #9
# ...
print_ = print_with_memory()
print_("I love Python")
print_("...and also The Python Coding Stack")
print(print_.__closure__[0])
Code Block #10
# ...
print_ = print_with_memory()
print_("I love Python")
print_("...and also The Python Coding Stack")
print(print_.__closure__[0].cell_contents)
Code Block #11
def print_with_memory():
data = []
def inner(some_obj):
data.append(some_obj)
print(some_obj)
return inner
print_ = print_with_memory()
Code Block #12
def store_arguments():
data = []
def inner(some_obj):
data.append(some_obj)
print(some_obj)
return inner
print_ = store_arguments()
Code Block #13
def store_arguments(func):
data = []
def inner(some_obj):
data.append(some_obj)
func(some_obj)
return inner
print_ = store_arguments(print)
Code Block #14
# ...
print_ = store_arguments(print)
print_("I love Python")
print_("...and also The Python Coding Stack")
print(print_.__closure__[0].cell_contents)
Code Block #15
def store_arguments(func):
data = []
def inner(some_obj):
data.append(some_obj)
func(some_obj)
return inner
print_ = store_arguments(print)
print_("I love Python")
print_(42, 99, 256)
print_(10, 20, 30, sep=":")
print(print_.__closure__[0].cell_contents)
Code Block #16
print(*args, sep=' ', end='\n', file=None, flush=False)
Code Block #17
def store_arguments(func):
data = []
def inner(*args, **kwargs):
data.append((args, kwargs))
func(*args, **kwargs)
return inner
print_ = store_arguments(print)
print_("I love Python")
print_(42, 99, 256)
print_(10, 20, 30, sep=":")
print(print_.__closure__[0].cell_contents)
Code Block #18
def store_arguments(func):
data = []
def inner(*args, **kwargs):
data.append((args, kwargs))
func(*args, **kwargs)
return inner
max_ = store_arguments(max)
print(max_(5, 2, 5, 76, 5, 23))
print(max_("Hello", "Goodbye", "Au Revoir"))
print(max_("Hello", "Goodbye", "Au Revoir", key=lambda x: len(x)))
print(max_.__closure__[0].cell_contents)
Code Block #19
def store_arguments(func):
data = []
def inner(*args, **kwargs):
data.append((args, kwargs))
value = func(*args, **kwargs)
return value
return inner
max_ = store_arguments(max)
print(max_(5, 2, 5, 76, 5, 23))
print(max_("Hello", "Goodbye", "Au Revoir"))
print(max_("Hello", "Goodbye", "Au Revoir", key=lambda x: len(x)))
print(max_.__closure__[0].cell_contents)
Code Block #20
def store_arguments(func):
data = []
def inner(*args, **kwargs):
data.append((args, kwargs))
value = func(*args, **kwargs)
return value
return inner
def my_special_function(name, repeat):
return name.upper() * repeat
my_special_function = store_arguments(my_special_function)
print(my_special_function("James", 3))
print(my_special_function("Maive", 2))
print(
my_special_function.__closure__[0].cell_contents
)
For more Python resources, you can also visit Real Python—you may even stumble on one of my own articles or courses there!
Also, are you interested in technical writing? You’d like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.
And you can find out more about me at stephengruppetta.com
