12

Suppose I have a DataFrame with 100k rows and a column name. I would like to split this name into first and last name as efficiently as possibly. My current method is,

def splitName(name):
  return pandas.Series(name.split()[0:2])

df[['first', 'last']] = df.apply(lambda x: splitName(x['name']), axis=1)

Unfortunately, DataFrame.apply is really, really slow. Is there anything I can do to make this string operation nearly as fast as a numpy operation?

Thanks!

duckworthd
  • 14,679
  • 16
  • 53
  • 68
  • 5
    If you have pandas 0.8.1 or above, it looks like you should be able to do `series.str.split()`. Docs here: http://pandas.pydata.org/pandas-docs/stable/basics.html#vectorized-string-methods – Thomas K Oct 10 '12 at 22:36

1 Answers1

23

Try (requires pandas >= 0.8.1):

splits = x['name'].split()
df['first'] = splits.str[0]
df['last'] = splits.str[1]
Wes McKinney
  • 101,437
  • 32
  • 142
  • 108
  • 1
    Perfect! Didn't know about this addition. – duckworthd Oct 12 '12 at 19:11
  • Interestingly, this question is identical to [this later one](http://stackoverflow.com/questions/17116814/pandas-how-do-i-split-text-in-a-column-into-multiple-columns) but the response has no mention of `Series.split()`. Has it been removed from `pandas`? – LondonRob Feb 07 '14 at 15:25
  • 9
    It is now available as `Series.str.split()` – joris Apr 09 '14 at 15:01