pandas.DataFrame.replace#
Values of the DataFrame are replaced with other values dynamically.
This differs from updating with .loc or .iloc , which require you to specify a location to update with some value.
Parameters to_replace str, regex, list, dict, Series, int, float, or None
How to find the values that will be replaced.
numeric, str or regex:
-
numeric: numeric values equal to to_replace will be replaced with value
-
str: string exactly matching to_replace will be replaced with value
-
regex: regexs matching to_replace will be replaced with value
list of str, regex, or numeric:
-
First, if to_replace and value are both lists, they must be the same length.
-
Second, if regex=True then all of the strings in both lists will be interpreted as regexs otherwise they will match directly. This doesn’t matter much for value since there are only a few possible substitution regexes you can use.
-
str, regex and numeric rules apply as above.
-
Dicts can be used to specify different replacement values for different existing values. For example, <'a': 'b', 'y': 'z'>replaces the value ‘a’ with ‘b’ and ‘y’ with ‘z’. To use a dict in this way, the optional value parameter should not be given.
-
For a DataFrame a dict can specify that different values should be replaced in different columns. For example, <'a': 1, 'b': 'z'>looks for the value 1 in column ‘a’ and the value ‘z’ in column ‘b’ and replaces these values with whatever is specified in value . The value parameter should not be None in this case. You can treat this as a special case of passing two lists except that you are specifying the column to search in.
-
For a DataFrame nested dictionaries, e.g., <'a': <'b': np.nan>> , are read as follows: look in column ‘a’ for the value ‘b’ and replace it with NaN. The optional value parameter should not be specified to use a nested dict in this way. You can nest regular expressions as well. Note that column names (the top-level dictionary keys in a nested dictionary) cannot be regular expressions.
-
This means that the regex argument must be a string, compiled regular expression, or list, dict, ndarray or Series of such elements. If value is also None then this must be a nested dictionary or Series.
See the examples section for examples of each of these.
value scalar, dict, list, str, regex, default None
Value to replace any values matching to_replace with. For a DataFrame a dict of values can be used to specify which value to use for each column (columns not in the dict will not be filled). Regular expressions, strings and lists or dicts of such objects are also allowed.
inplace bool, default False
Whether to modify the DataFrame rather than creating a new one.
limit int, default None
Maximum size gap to forward or backward fill.
regex bool or same types as to_replace , default False
Whether to interpret to_replace and/or value as regular expressions. If this is True then to_replace must be a string. Alternatively, this could be a regular expression or a list, dict, or array of regular expressions in which case to_replace must be None .
method
The method to use when for replacement, when to_replace is a scalar, list or tuple and value is None .
Changed in version 0.23.0: Added to DataFrame.
Object after replacement.
If regex is not a bool and to_replace is not None .
If to_replace is not a scalar, array-like, dict , or None
If to_replace is a dict and value is not a list , dict , ndarray , or Series
If to_replace is None and regex is not compilable into a regular expression or is a list, dict, ndarray, or Series.
When replacing multiple bool or datetime64 objects and the arguments to to_replace does not match the type of the value being replaced
If a list or an ndarray is passed to to_replace and value but they are not the same length.
Replace values based on boolean condition.
Simple string replacement.
Regex substitution is performed under the hood with re.sub . The rules for substitution for re.sub are the same.
Regular expressions will only substitute on strings, meaning you cannot provide, for example, a regular expression matching floating point numbers and expect the columns in your frame that have a numeric dtype to be matched. However, if those floating point numbers are strings, then you can do this.
This method has a lot of options. You are encouraged to experiment and play with this method to gain intuition about how it works.
When dict is used as the to_replace value, it is like key(s) in the dict are the to_replace part and value(s) in the dict are the value parameter.
Pandas replace() – Replace Values in Pandas Dataframe
In this post, you’ll learn how to use the Pandas .replace() method to replace data in your DataFrame. The Pandas DataFrame.replace() method can be used to replace a string, values, and even regular expressions (regex) in your DataFrame.
Update for 2023
The Quick Answer:
Table of Contents
Pandas Replace Method Syntax
The Pandas .replace() method takes a number of different parameters. Let’s take a look at them:
The list below breaks down what the parameters of the .replace() method expect and what they represent:
- to_replace= : take a string, list, dictionary, regex, int, float, etc., and describes the values to replace
- value= : The value to replace with
- inplace= : whether to perform the operation in place
- limit= : the maximum size gap to backward or forward fill
- regex= : whether to interpret to_replace and/or value as regex
- method= : the method to use for replacement
Let’s dive into how to use the method, starting by loading a sample Pandas DataFrame.
Loading Sample DataFrame
To start things off, let’s begin by loading a Pandas DataFrame. We’ll keep things simple so it’s easier to follow exactly what we’re replacing.
Let’s now dive into how to use the method, starting by looking at how to replace a single value in a given column.
Replace a Single Value in a Pandas DataFrame Column
Let’s learn how to replace a single value in a Pandas column. In the example below, we’ll look to replace the value Jane with Joan . In order to do this, we simply need to pass the value we want to replace into the to_replace= parameter and the value we want to replace with into the value= parameter.
In the code block above, we applied the .replace() method to the column directly, reassigning the column to itself. Because the two parameters are the first and second parameters, positionally, we don’t actually need to name them.
Replace Multiple Values with the Same Value in a Pandas DataFrame
Now, you may want to replace multiple values with the same value. This is also extremely easy to do using the .replace() method.
Of course, you could simply run the method twice, but there’s a much more efficient way to accomplish this. Here, we’ll look to replace London and Paris with Europe :
In the code block above, we passed in a list of values into the to_replace= parameter. This looks for both of the values in the column. Since we only passed in a single value into the value= parameter, this value is used to replace both the other values.
Now let’s look at how to replace multiple values with different ones in the following section.
Replace Multiple Values with Different Values in a Pandas DataFrame
Like the example above, you can replace a list of multiple values with a list of different ones.
In order to do this, you can pass in a list of values into the to_replace= parameter as well as a list of equal length into the value= parameter.
In the example below, we’ll replace London with England and Paris with France :
In the following section, we’ll explore how to accomplish this for values across the entire DataFrame, rather than a single column.
Replace Values in the Entire DataFrame
In the previous examples, you learned how to replace values in a single column. Similar to those examples, we can easily replace values in the entire DataFrame.
Let’s take a look at replacing the letter F with P in the entire DataFrame:
In the example above, we applied the .replace() to the entire DataFrame. We can see that this didn’t return the expected results. In this case, only entire cell values that match the conditions are replaced.
Replacing Values with Regex (Regular Expressions)
In order to replace substrings in a Pandas DataFrame, you can instruct Pandas to use regular expressions (regex). In order to replace substrings (such as in Melissa), we simply pass in regex=True :
Let’s also take a closer look at more complex regular expression replacements.
Using Pandas .replace() With More Complex Regex
We can use regular expressions to make complex replacements.
We’ll cover a fairly simple example, where we replace any four-letter word in the Name column with “Four letter name”.
The following .replace() method call does just that:
In the following section, you’ll learn how to replace values in place.
Replace Values In Place with Pandas
We can also replace values in place, rather than having to re-assign them. This is done simply by setting inplace= to True .
Let’s revisit an earlier example:
While this approach does save some memory (as it doesn’t need to create a new object), it’s often better to be consistent with how the rest of your code is formatted.
Using Dictionaries to Replace Values with Pandas replace
The Pandas .replace() method also allows you to use dictionaries to replace values. This can often be a convenient way of handling many replacements. However, it’s not my preferred approach as the behavior can often be difficult to read.
Let’s take a look at how the method can replace values:
We can see that the dictionary can be used in two different ways:
- To map values to replace so that the dictionary represents
- To map replacements from columns so that it follows the structure shown here: to_replace=
While the first approach is more concise, I would prefer using the Pandas map() method for this approach.
The second method provides more flexibility for using the method across different columns but can be a little harder to read. In these cases, I would personally just call the method twice for different columns.
Conclusion
In this post, you learned how to use the Pandas replace method to, well, replace values in a Pandas DataFrame. The .replace() method is extremely powerful and lets you replace values across a single column, multiple columns, and an entire DataFrame. The method also incorporates regular expressions to make complex replacements easier.
To learn more about the Pandas .replace() method, check out the official documentation here.
Replacing column values in a pandas DataFrame
I’m trying to replace the values in one column of a dataframe. The column (‘female’) only contains the values ‘female’ and ‘male’.
I have tried the following:
But receive the exact same copy of the previous results.
I would ideally like to get some output which resembles the following loop element-wise.
I’ve looked through the gotchas documentation (http://pandas.pydata.org/pandas-docs/stable/gotchas.html) but cannot figure out why nothing happens.
Any help will be appreciated.
16 Answers 16
If I understand right, you want something like this:
(Here I convert the values to numbers instead of strings containing numbers. You can convert them to «1» and «0» , if you really want, but I’m not sure why you’d want that.)
The reason your code doesn’t work is because using [‘female’] on a column (the second ‘female’ in your w[‘female’][‘female’] ) doesn’t mean «select rows where the value is ‘female'». It means to select rows where the index is ‘female’, of which there may not be any in your DataFrame.
You can edit a subset of a dataframe by using loc:
This should also work:
This is very compact:
Another good one:
You can also use apply with .get i.e.
Using apply to replace values from the dictionary:
Note: apply with dictionary should be used if all the possible values of the columns in the dataframe are defined in the dictionary else, it will have empty for those not defined in dictionary.
Using Series.map with Series.fillna
If your column contains more strings than only female and male , Series.map will fail in this case since it will return NaN for other values.
That’s why we have to chain it with fillna :
Example why .map fails:
For the correct method, we chain map with fillna , so we fill the NaN with values from the original column:
Alternatively there is the built-in function pd.get_dummies for these kinds of assignments:
This gives you a data frame with two columns, one for each value that occurs in w[‘female’], of which you drop the first (because you can infer it from the one that is left). The new column is automatically named as the string that you replaced.
This is especially useful if you have categorical variables with more than two possible values. This function creates as many dummy variables needed to distinguish between all cases. Be careful then that you don’t assign the entire data frame to a single column, but instead, if w[‘female’] could be ‘male’, ‘female’ or ‘neutral’, do something like this:
Then you are left with two new columns giving you the dummy coding of ‘female’ and you got rid of the column with the strings.
Замена одного или нескольких значений в столбце в DataFrame Pandas
Чтобы заменить значения в столбце на основе условия в Pandas DataFrame, вы можете использовать свойство DataFrame.loc, numpy.where() или DataFrame.where().
В этом руководстве мы рассмотрим все эти процессы на примерах программ.
Метод 1: в зависимости от условия
Чтобы заменить значения в столбце на основе условия с помощью DataFrame.loc, используйте следующий синтаксис.
В следующей программе мы заменим те значения в столбце «a», которые удовлетворяют условию, что значение меньше нуля.
Вы также можете заменить значения в нескольких значениях на основе одного условия. Передайте столбцы как кортеж в loc.
В следующей программе мы заменим те значения в столбцах «a» и «b», которые удовлетворяют условию, что значение меньше нуля.
Метод 2: с помощью where
Чтобы заменить значения в столбце на основе условия с помощью numpy.where, используйте следующий синтаксис.
В следующей программе мы воспользуемся методом numpy.where() и заменим те значения в столбце «a», которые удовлетворяют условию, что значение меньше нуля.
Метод 3
- column_name – это столбец, в котором необходимо заменить значения.
- condition – это логическое выражение, которое применяется для каждого значения в столбце.
- new_value заменяет (поскольку inplace = True) существующее значение в указанном столбце на основе условия.
В следующей программе мы будем использовать метод DataFrame.where() и заменим те значения в столбце «a», которые удовлетворяют условию, что значение меньше нуля.
В этом руководстве на примерах Python мы узнали, как заменить значения столбца в DataFrame новым значением в зависимости от условия.
Как заменить несколько значений?
Чтобы заменить несколько значений в DataFrame, вы можете использовать метод DataFrame.replace() со словарем различных замен, переданных в качестве аргумента.
Пример 1
Синтаксис для замены нескольких значений в столбце DataFrame:
В следующем примере мы будем использовать метод replace() для замены 1 на 11 и 2 на 22 в столбце a.
Пример 2
Синтаксис для замены нескольких значений в нескольких столбцах DataFrame:
В следующем примере мы воспользуемся методом replace() для замены 1 на 11 и 2 на 22 в столбце a; 5 с 55 и 2 с 22 в столбце b.
В этом руководстве на примерах Python мы узнали, как заменить несколько значений в Pandas DataFrame в одном или нескольких столбцах.