Pandas: change data type of Series to String

Question

I use Pandas 'ver 0.12.0' with Python 2.7 and have a dataframe as below:

df = pd.DataFrame({'id' : [123,512,'zhub1', 12354.3, 129, 753, 295, 610],
                    'colour': ['black', 'white','white','white',
                            'black', 'black', 'white', 'white'],
                    'shape': ['round', 'triangular', 'triangular','triangular','square',
                                        'triangular','round','triangular']
                    },  columns= ['id','colour', 'shape'])

The id Series consists of some integers and strings. Its dtype by default is object. I want to convert all contents of id to strings. I tried astype(str), which produces the output below.

df['id'].astype(str)
0    1
1    5
2    z
3    1
4    1
5    7
6    2
7    6

1) How can I convert all elements of id to String?

2) I will eventually use id for indexing for dataframes. Would having String indices in a dataframe slow things down, compared to having an integer index?

Not sure why you get that output as `astype` works fine for me, at least in version 0.13.1, maybe 0.12.0 has a bug? In answer to your second point, yes it is likely to be slower as string comparison will not be faster than integer comparison but I would profile this first, also it depends on the size — EdChum, Mar 06 '14 at 17:41
you've set the column, right? df['id'] = df['id'].astype(str) — Andy Hayden, Mar 06 '14 at 17:51
@Andy Hayden, yes I do the appointment, but it is the output that I thought was unexpected. — Zhubarb, Mar 06 '14 at 18:48
It only returns the 1st character of each Series element as I put in the question under `df['id'].astype(str)` — Zhubarb, Mar 07 '14 at 08:55
For anyone wondering why the accepted answer isn't working in later versions of pandas, I have added a new answer to the question to reflect the current documentation. — rocksNwaves, Mar 05 '20 at 20:35

rocksNwaves · Accepted Answer · 2021-06-02T13:21:32.900

219

A new answer to reflect the most current practices: as of now (v1.2.4), neither astype('str') nor astype(str) work.

As per the documentation, a Series can be converted to the string datatype in the following ways:

df['id'] = df['id'].astype("string")

df['id'] = pandas.Series(df['id'], dtype="string")

df['id'] = pandas.Series(df['id'], dtype=pandas.StringDtype)

edited Jun 02 '21 at 13:21

answered Mar 05 '20 at 20:34

rocksNwaves

5,331
4
38
77

1

When i try this I get `data type "string" not understood` – thentangler Sep 21 '22 at 16:11
First solution works technically, but replaces content. Copies the strings of one row into all others. particular. Second solution doesn't work technically. Third throws error ```TypeError: Expected an instance of StringDtype, but got the class instead. Try instantiating 'dtype'.``` Will look at specifying data type when I load the data. – Simone Mar 10 '23 at 13:52
The third solution misses brackets. The following worked ```df['col'] = pandas.Series(df['col'], dtype = pd.StringDtype()) ``` – Simone Mar 10 '23 at 14:30

score 125 · Answer 2 · edited Mar 07 '14 at 09:34

125

You can convert all elements of id to str using apply

df.id.apply(str)

0        123
1        512
2      zhub1
3    12354.3
4        129
5        753
6        295
7        610

Edit by OP:

I think the issue was related to the Python version (2.7.), this worked:

df['id'].astype(basestring)
0        123
1        512
2      zhub1
3    12354.3
4        129
5        753
6        295
7        610
Name: id, dtype: object

edited Mar 07 '14 at 09:34

Zhubarb

11,432
18
75
114

answered Mar 06 '14 at 17:36

Amit

19,780
6
46
54

4

Thank you, I will try this when I am next to a computer and accept your answer. do you know why `astype(str )` does not work? – Zhubarb Mar 06 '14 at 18:49
2

@Zhubarb - I just tried it, I am getting same result with `df['id'].astype(str)` – Amit Mar 06 '14 at 18:53
the same result that I posted (undesired) or the result that you got with `df.id.apply(str)` (desired)? – Zhubarb Mar 07 '14 at 08:57
4

I believe it should be `.astype('str')` instead of `.astype(str)` – Alex Klibisz Aug 06 '16 at 19:15
I just tried with `df['id'].astype(str)` and I am getting SettingWithCopyWarning: warning. How to get rid of this warning? – Ernest S Kirubakaran Sep 06 '17 at 08:37
2

@ErnestSKirubakaran - Read the previous comment, try with `.astype('str')` – Amit Sep 25 '17 at 11:40
I'm trying the df['id'].astype(basestring) and I'm receiving an error NameError: name 'basestring' is not defined. – DataGirl Nov 24 '21 at 16:13
1

Also I believe `apply` isn’t really vectorized way of converting to str. – MMK Jan 18 '22 at 13:41
This answer needs to be updated, as the type to specify is now `"string"` and`'str'` is no longer accepted – PJ_ Apr 12 '22 at 20:35

score 70 · Answer 3 · answered Oct 03 '17 at 10:15

70

You must assign it, like this:-

df['id']= df['id'].astype(str)

answered Oct 03 '17 at 10:15

Rishil Antony

725
5
5

score 8 · Answer 4 · answered Jun 28 '19 at 19:43

8

Personally none of the above worked for me. What did:

new_str = [str(x) for x in old_obj][0]

answered Jun 28 '19 at 19:43

manesioz

798
1
9
21

Ya true the other ones did not change anything. I know those should've worked but something is off and it did not so I guess this can also be a solution – Vengenzz Vicky Sep 15 '22 at 06:44

score 6 · Answer 5 · answered Mar 27 '20 at 10:00

You can use:

df.loc[:,'id'] = df.loc[:, 'id'].astype(str)

This is why they recommend this solution: Pandas doc

TD;LR

To reflect some of the answers:

df['id'] = df['id'].astype("string")

This will break on the given example because it will try to convert to StringArray which can not handle any number in the 'string'.

df['id']= df['id'].astype(str)

For me this solution throw some warning:

> SettingWithCopyWarning:  
> A value is trying to be set on a copy of a
> slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

score 5 · Answer 6 · answered Sep 07 '21 at 11:19

5

There are two possibilities:

Use .astype("str").astype("string"). As seen here
Use .astype(pd.StringDtype()). From the official documentation

answered Sep 07 '21 at 11:19

Rafael Ortega

81
1
4

score 3 · Answer 7 · answered Sep 10 '20 at 13:58

3

For me it worked:

 df['id'].convert_dtypes()

see the documentation here:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.convert_dtypes.html

answered Sep 10 '20 at 13:58

drGabriel

548
6
5

score 2 · Answer 8 · answered Jan 18 '22 at 03:23

2

use pandas string methods ie df['id'].str.cat()

answered Jan 18 '22 at 03:23

Jimmy Obonyo Abor

7,335
10
43
71

score 2 · Answer 9 · answered Sep 09 '22 at 17:24

2

If you want to do dynamically

df_obj = df.select_dtypes(include='object')
df[df_obj.columns] = df_obj.astype(str)

answered Sep 09 '22 at 17:24

Ozzy Black

53
6

score 1 · Answer 10 · edited Jan 28 '23 at 20:32

1

Your problem can easily be solved by converting it to the object first. After it is converted to object, just use "astype" to convert it to str.

obj = lambda x:x[1:]
df['id']=df['id'].apply(obj).astype('str')

edited Jan 28 '23 at 20:32

ggorlen

44,755
7
76
106

answered May 13 '20 at 14:51

shekhar chander

600
8
14

score -2 · Answer 11 · answered Dec 01 '21 at 17:56

-2

for me .to_string() worked

df['id']=df['id'].to_string()

answered Dec 01 '21 at 17:56

Yachika paul

1

Pandas: change data type of Series to String

11 Answers11

Linked

Related