Как посмотреть типы данных в pandas

How to Check the Dtype of Column(s) in Pandas DataFrame

To check the dtypes of single or multiple columns in Pandas you can use:

Let's see other useful ways to check the dtypes in Pandas.

Step 1: Create sample DataFrame

To start, let's say that you have the date from earthquakes:

Date	Time	Depth	Magnitude Type	Type	Magnitude	Depth_int
1965-01-02 00:00:00+00:00	13:44:18	131.6	MW	Earthquake	6.0	131
1965-01-04 00:00:00+00:00	11:29:49	80.0	MW	Earthquake	5.8	80
1965-01-05 00:00:00+00:00	18:05:58	20.0	MW	Earthquake	6.2	20
1965-01-08 00:00:00+00:00	18:49:43	15.0	MW	Earthquake	5.8	15
1965-01-09 00:00:00+00:00	13:32:50	15.0	MW	Earthquake	5.8	15

How to read and convert Kaggle data to Pandas DataFrame: How to Search and Download Kaggle Dataset to Pandas DataFrame

Step 2: Get dtypes for all columns in DataFrame

To get dtypes details for the whole DataFrame you can use attribute — dtypes :

we can see several different types like:

datetime64[ns, UTC] — it's used for dates; explicit conversion may be needed in some cases
float64 / int64 — numeric data
object — strings and other

Step 3: Short explanation of dtypes in Pandas

Let's briefly cover some dtypes and their usage with simple examples. Table of the most used dtypes in Pandas:

Pandas dtype	Data Type	Description	Example	Creation
bool	bool	Boolean values – True or False	True	pd.BooleanDtype()
category	NA	Limited list of values (can be fixed)	[‘red’, ‘blue’]	pd.Categorical([1, 2, 3, 1, 2, 3])
datetime64	datetime	Datetime (conversion is needed)	2020-11-16 22:50:18.092888+0000	to_datetime(df['date'])
float64	float	Floating point numbers	80.5	df.astype('float64')
int64	int	Integer numbers	8	df.astype('int64')
object	strings	String, text and other	Red Pandas
timedelta	timedelta	Duration between two dates or times	0 days 00:00:00.000000001	pd.Timedelta(42, unit='ns')

More information about them can be found on this link: Pandas User Guide dtypes.

Pandas offers a wide range of features and methods in order to read, parse and convert between different dtypes. The most popular conversion methods are:

to_datetime(df['date'])
to_timedelta(df['timdelta'])
to_numeric(df['amount'])
df['amount'].astype('int32')

Step 4: Check if column is numeric, datetime, categorical etc

In this step we are going to see how we can check if a given column is numerical or categorical.

For this purpose Pandas offers a bunch of methods like:

is_string_dtype
is_dict_like
is_list_like
is_numeric_dtype
is_datetime64_dtype

To find all methods you can check the official Pandas docs: pandas.api.types.is_datetime64_any_dtype

To check if a column has numeric or datetime dtype we can:

for datetime exists several options like: is_datetime64_ns_dtype or is_datetime64_any_dtype :

Step 5: List all numeric/datetime columns in Pandas DataFrame

If you like to list only numeric/datetime or other type of columns in a DataFrame you can use method select_dtypes :

including

result of the operation:

excluding columns by dtype:

Step 6: Filter columns by dtype and name in Pandas DataFrame

As an alternative solution you can construct a loop over all columns. Then you can check the dtype and the name of the column.

Below we are listing all numeric column which name has word 'Depth':

As a result you will get a list of all numeric columns:

Instead of printing their names you can do something.

Step 7: Apply function on numeric columns only

To apply function to numeric or datetime columns only you can use the method select_dtypes in combination with apply .

The function below will iterate over all numeric columns and double the value:

Resources

By using DataScientYst — Data Science Simplified, you agree to our Cookie Policy.

how to check the dtype of a column in python pandas

I need to use different functions to treat numeric columns and string columns. What I am doing now is really dumb:

Is there a more elegant way to do this? E.g.

6 Answers 6

You can access the data-type of a column with dtype :

In pandas 0.20.2 you can do:

So your code becomes:

I know this is a bit of an old thread but with pandas 19.02, you can do:

Asked question title is general, but authors use case stated in the body of the question is specific. So any other answers may be used.

But in order to fully answer the title question it should be clarified that it seems like all of the approaches may fail in some cases and require some rework. I reviewed all of them (and some additional) in decreasing of reliability order (in my opinion):

1. Comparing types directly via == (accepted answer).

Despite the fact that this is accepted answer and has most upvotes count, I think this method should not be used at all. Because in fact this approach is discouraged in python as mentioned several times here.
But if one still want to use it — should be aware of some pandas-specific dtypes like pd.CategoricalDType , pd.PeriodDtype , or pd.IntervalDtype . Here one have to use extra type( ) in order to recognize dtype correctly:

Another caveat here is that type should be pointed out precisely:

2. isinstance() approach.

This method has not been mentioned in answers so far.

So if direct comparing of types is not a good idea — lets try built-in python function for this purpose, namely — isinstance() .
It fails just in the beginning, because assumes that we have some objects, but pd.Series or pd.DataFrame may be used as just empty containers with predefined dtype but no objects in it:

But if one somehow overcome this issue, and wants to access each object, for example, in the first row and checks its dtype like something like that:

It will be misleading in the case of mixed type of data in single column:

And last but not least — this method cannot directly recognize Category dtype. As stated in docs:

Returning a single item from categorical data will also return the value, not a categorical of length “1”.

So this method is also almost inapplicable.

3. df.dtype.kind approach.

This method yet may work with empty pd.Series or pd.DataFrames but has another problems.

First — it is unable to differ some dtypes:

Second, what is actually still unclear for me, it even returns on some dtypes None.

4. df.select_dtypes approach.

This is almost what we want. This method designed inside pandas so it handles most corner cases mentioned earlier — empty DataFrames, differs numpy or pandas-specific dtypes well. It works well with single dtype like .select_dtypes(‘bool’) . It may be used even for selecting groups of columns based on dtype:

Like so, as stated in the docs:

On may think that here we see first unexpected (at used to be for me: question) results — TimeDelta is included into output DataFrame . But as answered in contrary it should be so, but one have to be aware of it. Note that bool dtype is skipped, that may be also undesired for someone, but it’s due to bool and number are in different «subtrees» of numpy dtypes. In case with bool, we may use test.select_dtypes([‘bool’]) here.

Next restriction of this method is that for current version of pandas (0.24.2), this code: test.select_dtypes(‘period’) will raise NotImplementedError .

And another thing is that it’s unable to differ strings from other objects:

But this is, first — already mentioned in the docs. And second — is not the problem of this method, rather the way strings are stored in DataFrame . But anyway this case have to have some post processing.

5. df.api.types.is_XXX_dtype approach.

This one is intended to be most robust and native way to achieve dtype recognition (path of the module where functions resides says by itself) as i suppose. And it works almost perfectly, but still have at least one caveat and still have to somehow distinguish string columns.

Besides, this may be subjective, but this approach also has more ‘human-understandable’ number dtypes group processing comparing with .select_dtypes(‘number’) :

No timedelta and bool is included. Perfect.

My pipeline exploits exactly this functionality at this moment of time, plus a bit of post hand processing.

Получение типов данных столбцов в DataFrame Pandas

Чтобы получить типы данных столбцов в Pandas DataFrame, вызовите свойство dtypes, которое возвращает объект типа pandas.Series с типами данных каждого столбца в нем.

Синтаксис для использования свойства dtypes:

В следующей программе мы создали DataFrame с определенными данными и именами столбцов. Давайте получим типы данных столбцов с помощью DataFrame.dtypes.

Мы можем распечатать элементы возвращаемого значения DataFrame.dtypes, используя цикл for, как показано ниже.

В этом руководстве на примерах Python мы узнали, как получить типы данных столбца в DataFrame с помощью свойства dtypes.

How to get & check data types of Dataframe columns in Python Pandas

In this article we will discuss different ways to fetch the data type of single or multiple columns. Also see how to compare data types of columns and fetch column names based on data types.

Use Dataframe.dtypes to get Data types of columns in Dataframe

In Python’s pandas module Dataframe class provides an attribute to get the data type information of each columns i.e.

It returns a series object containing data type information of each column. Let’s use this to find & check data types of columns.

Suppose we have a Dataframe i.e.

Contents of the dataframe are,

Let’s fetch the Data type of each column in Dataframe as a Series object,

Index of returned Series object is column name and value column of Series contains the data type of respective column.

Get Data types of Dataframe columns as dictionary

We can convert the Series object returned by Dataframe.dtypes to a dictionary too,

Get the Data type of a single column in Dataframe

We can also fetch the data type of a single column from series object returned by Dataframe.dtypes i.e.

Check if data type of a column is int64 or object etc.

Using Dataframe.dtypes we can fetch the data type of a single column and can check its data type too i.e.

Check if Data type of a column is int64 in Dataframe

Check if Data type of a column is object i.e. string in Dataframe

Get list of pandas dataframe column names based on data type

Suppose we want a list of column names whose data type is np.object i.e string. Let’s see how to do that,

We basically filtered the series returned by Dataframe.dtypes by value and then fetched index names i.e. columns names from this filtered series.

Get data types of a dataframe using Dataframe.info()

Dataframe.info() prints a detailed summary of the dataframe. It includes information like

Name of columns
Data type of columns
Rows in dataframe
non null entries in each column

Let’s see an example,

It also gives us detail about data types of columns in our dataframe.

Complete example is as follows,

Output:

Advertisements

Thanks for reading.

Terms of Use

Disclaimer

To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.

Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen.

Читать:

No pyvenv cfg file pycharm как исправить