Pandas: change data type of Series to String

Question

I use Pandas 'ver 0.12.0' with Python 2.7 and have a dataframe as below:

df = pd.DataFrame({'id' : [123,512,'zhub1', 12354.3, 129, 753, 295, 610],
                    'colour': ['black', 'white','white','white',
                            'black', 'black', 'white', 'white'],
                    'shape': ['round', 'triangular', 'triangular','triangular','square',
                                        'triangular','round','triangular']
                    },  columns= ['id','colour', 'shape'])

The id Series consists of some integers and strings. Its dtype by default is object. I want to convert all contents of id to strings. I tried astype(str), which produces the output below.

df['id'].astype(str)
0    1
1    5
2    z
3    1
4    1
5    7
6    2
7    6

1) How can I convert all elements of id to String?

2) I will eventually use id for indexing for dataframes. Would having String indices in a dataframe slow things down, compared to having an integer index?

Not sure why you get that output as `astype` works fine for me, at least in version 0.13.1, maybe 0.12.0 has a bug? In answer to your second point, yes it is likely to be slower as string comparison will not be faster than integer comparison but I would profile this first, also it depends on the size — EdChum, Mar 06 '14 at 17:41
you've set the column, right? df['id'] = df['id'].astype(str) — Andy Hayden, Mar 06 '14 at 17:51
@Andy Hayden, yes I do the appointment, but it is the output that I thought was unexpected. — Zhubarb, Mar 06 '14 at 18:48
It only returns the 1st character of each Series element as I put in the question under `df['id'].astype(str)` — Zhubarb, Mar 07 '14 at 08:55
For anyone wondering why the accepted answer isn't working in later versions of pandas, I have added a new answer to the question to reflect the current documentation. — rocksNwaves, Mar 05 '20 at 20:35

rocksNwaves · Accepted Answer · 2021-06-02T13:21:32.900

125

A new answer to reflect the most current practices: as of now (v1.2.4), neither astype('str') nor astype(str) work.

As per the documentation, a Series can be converted to the string datatype in the following ways:

df['id'] = df['id'].astype("string")

df['id'] = pandas.Series(df['id'], dtype="string")

df['id'] = pandas.Series(df['id'], dtype=pandas.StringDtype)

edited Jun 02 '21 at 13:21

answered Mar 05 '20 at 20:34

rocksNwaves

3,720
2
25
55

3

This needs to get higher in the search results for this kind of question. Everything else I tried was from out of date advice! Thank you! – Liz Z Mar 16 '21 at 21:10
@LizZ My pleasure! My answer is relatively new, but I assume it will work its way up to the top eventually. – rocksNwaves Mar 16 '21 at 21:15
2

This solution worked and the others did not. This should go as the accepted answer instead ! – lqope54 Aug 12 '21 at 12:28

score 123 · Answer 2 · edited Mar 07 '14 at 09:34

123

You can convert all elements of id to str using apply

df.id.apply(str)

0        123
1        512
2      zhub1
3    12354.3
4        129
5        753
6        295
7        610

Edit by OP:

I think the issue was related to the Python version (2.7.), this worked:

df['id'].astype(basestring)
0        123
1        512
2      zhub1
3    12354.3
4        129
5        753
6        295
7        610
Name: id, dtype: object

edited Mar 07 '14 at 09:34

Zhubarb

9,913
18
70
106

answered Mar 06 '14 at 17:36

Amit

18,038
6
44
52

4

Thank you, I will try this when I am next to a computer and accept your answer. do you know why `astype(str )` does not work? – Zhubarb Mar 06 '14 at 18:49
2

@Zhubarb - I just tried it, I am getting same result with `df['id'].astype(str)` – Amit Mar 06 '14 at 18:53
the same result that I posted (undesired) or the result that you got with `df.id.apply(str)` (desired)? – Zhubarb Mar 07 '14 at 08:57
4

I believe it should be `.astype('str')` instead of `.astype(str)` – Alex Klibisz Aug 06 '16 at 19:15
I just tried with `df['id'].astype(str)` and I am getting SettingWithCopyWarning: warning. How to get rid of this warning? – Ernest S Kirubakaran Sep 06 '17 at 08:37
2

@ErnestSKirubakaran - Read the previous comment, try with `.astype('str')` – Amit Sep 25 '17 at 11:40
I'm trying the df['id'].astype(basestring) and I'm receiving an error NameError: name 'basestring' is not defined. – DataGirl Nov 24 '21 at 16:13
Also I believe `apply` isn’t really vectorized way of converting to str. – MMK Jan 18 '22 at 13:41
This answer needs to be updated, as the type to specify is now `"string"` and`'str'` is no longer accepted – PeJota Apr 12 '22 at 20:35

score 67 · Answer 3 · answered Oct 03 '17 at 10:15

67

You must assign it, like this:-

df['id']= df['id'].astype(str)

answered Oct 03 '17 at 10:15

Rishil Antony

695
5
5

score 5 · Answer 4 · answered Jun 28 '19 at 19:43

5

Personally none of the above worked for me. What did:

new_str = [str(x) for x in old_obj][0]

answered Jun 28 '19 at 19:43

manesioz

723
1
8
19

score 4 · Answer 5 · answered Mar 27 '20 at 10:00

You can use:

df.loc[:,'id'] = df.loc[:, 'id'].astype(str)

This is why they recommend this solution: Pandas doc

TD;LR

To reflect some of the answers:

df['id'] = df['id'].astype("string")

This will break on the given example because it will try to convert to StringArray which can not handle any number in the 'string'.

df['id']= df['id'].astype(str)

For me this solution throw some warning:

> SettingWithCopyWarning:  
> A value is trying to be set on a copy of a
> slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

score 4 · Answer 6 · answered Sep 07 '21 at 11:19

4

There are two possibilities:

Use .astype("str").astype("string"). As seen here
Use .astype(pd.StringDtype()). From the official documentation

answered Sep 07 '21 at 11:19

Rafael Ortega

71
4

score 2 · Answer 7 · answered Sep 10 '20 at 13:58

2

For me it worked:

 df['id'].convert_dtypes()

see the documentation here:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.convert_dtypes.html

answered Sep 10 '20 at 13:58

Gabriel

326
2
4

shekhar chander · Answer 8 · 2020-05-17T09:07:21.317

1

Your problem can easily be solved by converting it to the object first. After it is converted to object, just use "astype" to convert it to str.

obj = lambda x:x[1:]
df['id']=df['id'].apply(obj).astype('str')

edited May 17 '20 at 09:07

answered May 13 '20 at 14:51

shekhar chander

424
6
13

score 1 · Answer 9 · answered Jan 18 '22 at 03:23

1

use pandas string methods ie df['id'].str.cat()

answered Jan 18 '22 at 03:23

Jimmy Obonyo Abor

6,529
9
42
70

score 0 · Answer 10 · answered Dec 01 '21 at 17:56

0

for me .to_string() worked

df['id']=df['id'].to_string()

answered Dec 01 '21 at 17:56

Yachika paul

1

Pandas: change data type of Series to String

10 Answers10

Linked

Related