Avoid Mutable Default Arguments in Python: Best Practices
Written on
Chapter 1: Introduction to Mutable Default Arguments
Utilizing mutable default argument values in Python can lead to unexpected and buggy behavior. This section explores why this occurs and how to avoid it through the proper use of None and conditional statements. Grasping the fundamental concepts behind immutable default arguments is essential for creating robust and maintainable code.
Table of Contents
- The Problem: Mutable Default Arguments in Python Functions
- The Initial Misstep: Using Mutable Defaults
- Understanding the Issue
- Refactor: Employ None and Conditional Statements
- A More Complex Case: Default Dictionaries
- Refactoring with None and Conditionals
- Managing Multiple Default Arguments
- The Initial Approach with Mutable Defaults
- Advanced Insights
- Python's Handling of Default Arguments
- Leveraging Immutable Defaults
- Utilizing Tuples for Immutable Defaults
- Exceptions to the Rule?
- Conclusion
The Problem: Mutable Default Arguments in Python Functions
In Python, default argument values are evaluated once when the function is defined, not each time the function is invoked. This characteristic is a critical aspect of how Python manages function definitions. Let's explore hypothetical scenarios that often occur.
The Initial Misstep: Using Mutable Defaults
Consider a simple function that appends a string to a list. If no list is provided, we want a new list to be created by default. Initially, we set the list as a mutable default argument.
from typing import List, Optional
def add_str_item(item: str, item_list: Optional[List[str]] = None) -> List[str]:
"""
Appends a string item to a list. If no list is provided, a new list is created.
Parameters:
item (str): The string to append to the list.
item_list (Optional[List[str]]): The list to append the item to. Defaults to None.
Returns:
List[str]: The updated list with the appended item.
"""
if item_list is None:
item_list = []
item_list.append(item)
return item_list
# Usage
print(add_str_item('apple')) # Output: ['apple']
print(add_str_item('banana')) # Output: ['banana']
print(add_str_item('orange', [])) # Output: ['orange']
print(add_str_item('grape')) # Output: ['grape']
Notice the second and last print statements. We did not pass a list, so we expected a new list to be created for each call. However, values from previous calls remain, leading to unexpected outcomes. The source of this issue lies in the mutable default argument.
Understanding the Issue
In Python, default arguments can be set to any object type. It's important to differentiate between two object types:
- Mutable Objects (e.g., lists, dictionaries, sets) can change after creation.
- Immutable Objects (e.g., integers, strings, tuples) cannot be altered once created.
The key takeaway is to use immutable objects or None as default arguments. Using mutable objects can lead to unintended side effects. It's best to initialize variables within the function when a mutable type is needed.
Refactor: Employ None and Conditional Statements
Let's modify the previous function to set item_list to None as the default value and initialize the list within the function if it's None.
from typing import List, Optional
def add_str_item(item: str, item_list: Optional[List[str]] = None) -> List[str]:
"""
Appends a string item to a list. If no list is provided, a new list is created.
Parameters:
item (str): The string to append to the list.
item_list (Optional[List[str]]): The list to append the item to. Defaults to None.
Returns:
List[str]: The updated list with the appended item.
"""
if item_list is None:
item_list = []
item_list.append(item)
return item_list
# Usage remains unchanged
print(add_str_item('apple')) # Output: ['apple']
print(add_str_item('banana')) # Output: ['banana']
print(add_str_item('orange', [])) # Output: ['orange']
print(add_str_item('grape')) # Output: ['grape']
Now, every function call first checks if item_list is None; if so, a new list is created. This ensures that each call without the argument results in a new, independent list.
A More Complex Case: Default Dictionaries
Next, let's examine a function that processes data and stores results in a dictionary. Again, we want to pass an existing dictionary or create a new one if none is provided.
Initially, we use a mutable default argument for the dictionary.
from typing import Dict, List, Optional
def add_to_dict(key: str, value: str, data_dict: Optional[Dict[str, List[str]]] = None) -> Dict[str, List[str]]:
"""
Adds a value to a list associated with a key in the dictionary.
If the dictionary or the list does not exist, it is created.
Parameters:
key (str): The key to add to the dictionary.
value (str): The value to add to the dictionary.
data_dict (Optional[Dict[str, List[str]]]): The dictionary to add the key/value to. Defaults to None.
Returns:
Dict[str, List[str]]: The resulting dictionary with the new key/value pair added.
"""
if data_dict is None:
data_dict = {}
if key in data_dict:
data_dict[key].append(value)else:
data_dict[key] = [value]
return data_dict
# Usage
print(add_to_dict('fruits', 'apple')) # Output: {'fruits': ['apple']}
print(add_to_dict('fruits', 'banana')) # Output: {'fruits': ['apple', 'banana']}
print(add_to_dict('vegetables', 'carrot', {})) # Output: {'vegetables': ['carrot']}
print(add_to_dict('fruits', 'grape')) # Output: {'fruits': ['apple', 'banana', 'grape']}
As seen, the dictionary retains state between calls, resulting in unexpected behavior.
Refactoring with None and Conditional Statements
We can refactor the previous function to use None for the default dictionary.
from typing import Dict, List, Optional
def add_to_dict(key: str, value: str, data_dict: Optional[Dict[str, List[str]]] = None) -> Dict[str, List[str]]:
"""
Adds a value to a list associated with a key in the dictionary.
If the dictionary or the list does not exist, it is created.
Parameters:
key (str): The key to add to the dictionary.
value (str): The value to add to the dictionary.
data_dict (Optional[Dict[str, List[str]]]): The dictionary to add the key/value to. Defaults to None.
Returns:
Dict[str, List[str]]: The resulting dictionary with the new key/value pair added.
"""
if data_dict is None:
data_dict = {}if key in data_dict:
data_dict[key].append(value)else:
data_dict[key] = [value]
return data_dict
# Usage remains unchanged
print(add_to_dict('fruits', 'apple')) # Output: {'fruits': ['apple']}
print(add_to_dict('fruits', 'banana', {'fruits': ['apple']})) # Output: {'fruits': ['apple', 'banana']}
print(add_to_dict('vegetables', 'carrot', {})) # Output: {'vegetables': ['carrot']}
print(add_to_dict('fruits', 'grape', {'fruits': ['banana']})) # Output: {'fruits': ['banana', 'grape']}
By using None for the default argument, we create a new dictionary for each function call if no value is supplied for data_dict.
Handling Multiple Default Arguments
Consider a function that requires multiple default arguments, some of which are mutable.
Initial Implementation with Mutable Defaults
from typing import Dict, List, Optional
def process_data(data: List[int], options: Optional[Dict[str, bool]] = None, results: Optional[List[List[int]]] = None) -> List[List[int]]:
"""
Processes data based on given options and appends the result to the provided results list.
Parameters:
data (List[int]): The list of integers to process.
options (Optional[Dict[str, bool]]): A dictionary of options that modify the processing behavior. Defaults to None.
results (Optional[List[List[int]]]): A list to store the processed data. Defaults to None.
Returns:
List[List[int]]: The list of processed data, with the new result appended.
"""
if options is None:
options = {}if results is None:
results = []
if options.get('reverse'):
data = data[::-1]results.append(data)
return results
# Usage
print(process_data([1, 2, 3])) # Output: [[1, 2, 3]]
print(process_data([4, 5, 6], {'reverse': True})) # Output: [[6, 5, 4]]
print(process_data([7, 8, 9], {}, [])) # Output: [[7, 8, 9]]
print(process_data([10, 11, 12])) # Output: [[1, 2, 3], [6, 5, 4], [10, 11, 12]]
As demonstrated, the function retains state due to mutable default arguments.
Refactored Function with None and Conditionals
from typing import Dict, List, Optional
def process_data(data: List[int], options: Optional[Dict[str, bool]] = None, results: Optional[List[List[int]]] = None) -> List[List[int]]:
"""
Processes data based on given options and appends the result to the provided results list.
Parameters:
data (List[int]): The list of integers to process.
options (Optional[Dict[str, bool]]): A dictionary of options that modify the processing behavior. Defaults to an empty dictionary.
results (Optional[List[List[int]]]): A list to store the processed data. Defaults to an empty list.
Returns:
List[List[int]]: The list of processed data, with the new result appended.
"""
if options is None:
options = {}if results is None:
results = []
if options.get('reverse'):
data = data[::-1]results.append(data)
return results
# Usage remains the same
print(process_data([1, 2, 3])) # Output: [[1, 2, 3]]
print(process_data([4, 5, 6], {'reverse': True})) # Output: [[6, 5, 4]]
print(process_data([7, 8, 9], {}, [])) # Output: [[7, 8, 9]]
print(process_data([10, 11, 12])) # Output: [[10, 11, 12]]
In this refactored version, we utilize None for multiple default arguments to ensure new instances of options and results are created with each function call.
Advanced Insights
Python compiles a function into a code object upon definition, with default argument values stored in the __defaults__ attribute. These values are evaluated and shared across all function calls, leading to issues with mutable defaults.
def example(a, b=[]):
return b
print(example.__defaults__) # Output: ([],)
To avoid this, we should stick to immutable objects or None to keep default values consistent.
Using Immutable Defaults
from typing import Optional, Tuple
def add_items(item: str, items: Optional[Tuple[str, ...]] = None) -> Tuple[str, ...]:
"""
Adds an item to a tuple of items, creating a new tuple if none is provided.
Parameters:
item (str): The item to add to the tuple.
items (Optional[Tuple[str, ...]]): The initial tuple of items. Defaults to None, which is replaced by an empty tuple.
Returns:
Tuple[str, ...]: A new tuple containing the original items and the added item.
"""
if items is None:
items = ()items += (item,)
return items
# Usage
print(add_items('apple')) # Output: ('apple',)
print(add_items('banana')) # Output: ('banana',)
print(add_items('orange', ('grape',))) # Output: ('grape', 'orange')
By using immutable defaults like tuples, we ensure that the default arguments remain unchanged.
Using Tuples for Immutable Defaults
If you need a container with default values that are clearly defined, you can use tuples for clarity.
from typing import Optional, Tuple
def add_coordinates(coord: Tuple[int, int], coords: Optional[Tuple[int, int]] = (0, 0)) -> Tuple[int, int]:
"""
Adds two coordinate pairs together.
Parameters:
coord (Tuple[int, int]): A tuple containing the x and y coordinates to add.
coords (Tuple[int, int]): A tuple containing the initial x and y coordinates. Defaults to (0, 0).
Returns:
Tuple[int, int]: A tuple containing the sum of the x and y coordinates.
"""
new_coords = (coords[0] + coord[0], coords[1] + coord[1])
return new_coords
print(add_coordinates((1, 2))) # Output: (1, 2)
print(add_coordinates((3, 4))) # Output: (3, 4)
Using tuples guarantees that the default argument remains immutable and cannot be altered.
Exceptions to the Rule?
I share this insight as it has caused numerous issues in my earlier Python development and frequently arises during code reviews. The simple answer is: there are no exceptions to this rule. The goal of software development is to produce cohesive code that functions as intended. By following the refactored examples, the code remains predictable and self-explanatory.
Conclusion
We transitioned from using mutable default arguments to adopting None and if statements, recognizing the significance of argument behavior in Python. This change leads to more predictable, maintainable, and bug-free code.
In your next function definition with default arguments, remember to use None and initialize defaults inside the function for improved robustness and easier debugging.
Call to Action
I hope you found this article informative and helpful in grasping the significance of avoiding mutable default arguments in Python functions.
I welcome your feedback and encourage you to share your thoughts, questions, and suggestions. Please consider sharing this article with peers who may find it useful. Additionally, follow me on Medium for more articles on best practices in Python coding.
Follow Dr. Robinson on LinkedIn and Facebook. Visit my homepage for papers, blogs, email signups, and more!
Want the latest articles delivered straight to your inbox? Subscribe now and never miss out!
In this video, we clarify the issues associated with mutable default arguments in Python and explore effective strategies to avoid them.
This video discusses how to avoid mutable default arguments in Python, focusing on best practices for function design.