1

I need to replace the specific values in every row of pandas df with another value. My data looks like this:

time        log
1         whats the weather look like today
2         what is the weather look like today
3         whats for lunch
4         what’s for lunch

I need to replace whats to be what is and what’s to be what is also. The desired output:

time        log
1         what is the weather look like today
2         what is the weather look like today
3         what is for lunch
4         what is for lunch

I have tried:

new_df = df.log.str.replace("^whats", "what is").str.replace("^what’s", "what is")

This took care of whats but not the other case and the outcome is not a pandas df and I need it to be pandas df.

Chique_Code
  • 1,422
  • 3
  • 23
  • 49
  • 3
    `df['log'] = df.log.str.replace("^whats|what‚Äôs", "what is")`? – Quang Hoang Feb 09 '21 at 19:14
  • it does return the dataframe back but it doesn't take care of the replacement. – Chique_Code Feb 09 '21 at 19:22
  • What do you mean *it doesn't take care of the replacement*? It does update your `df` to match the expected output. – Quang Hoang Feb 09 '21 at 19:29
  • I can still see `what‚Äôs` values and I need them to be replaced with `what is`... `whats` have been successfully replaced. – Chique_Code Feb 09 '21 at 19:31
  • 2
    I have a feeling that there might be some problem with what exactly the characters are. Because as I have just updated, it works for me. Can you find the give rows if you search for the string? – My Work Feb 09 '21 at 19:35
  • Hm. I ran `testing = df[df['log'].str.startswith("what‚Äôs")]` and it returned 0 rows back, though it is in there, I exported to csv and see the values... – Chique_Code Feb 09 '21 at 19:42
  • Sorry, what do you mean by that? – Chique_Code Feb 09 '21 at 19:44
  • 1
    Copied the text in your example and the replacement method - it works for me. – wwii Feb 09 '21 at 19:54
  • 1
    Are you sure you only have string there and no byte code? Or can you post the original data, the ones which make the trouble? What I mean is that what you see as `‚Äôs` might be totally different thing which is just shown like this, hence if you copy it out as it is here, all works, but in your dataset the actual character is not `‚Äôs`. Your finding suggest that, for me it find the row with `‚Äôs`. – My Work Feb 09 '21 at 20:35
  • Interesting. Perhaps I need to convert it to utf-8? I will try that and see if your code works for this scenario. In any case, I accepted the answer, because seems like what I am facing here is out of the scope of the question. Thank you for the help! – Chique_Code Feb 09 '21 at 21:47
  • `df['log'].apply(lambda x: x.decode("utf-8"))` returns `AttributeError: 'str' object has no attribute 'decode'` – Chique_Code Feb 09 '21 at 21:55
  • That should fail on strings (see https://stackoverflow.com/questions/28583565/str-object-has-no-attribute-decode-python-3-error) which you have there. Try it specifically on the cell, not on the entire column. – My Work Feb 10 '21 at 06:35

1 Answers1

2

What you are getting is a Pandas Series, if you want to get a DataFrame, just use

new_df = pd.DataFrame(df.log.str.replace("^whats", "what is").str.replace("^what’s", "what is"))

And as was pointed out by @Quang Hoang, you can search using the pandas OR and search either for whats or what’s:

new_df = pd.DataFrame(df.log.str.replace("^whats|what’s", "what is"))

Full code:

import pandas as pd

df = pd.DataFrame({"time": [1,2,3,4],
    "log": ['whats the weather look like today', 'what is the weather look like today', 'whats for lunch', 'what’s for lunch']
             })
new_df = pd.DataFrame(df.log.str.replace("^whats", "what is").str.replace("^what’s", "what is"))

and the results are:

print(df)
   time                                  log
0     1    whats the weather look like today
1     2  what is the weather look like today
2     3                      whats for lunch
3     4                   what’s for lunch

print(new_df)
                                   log
0  what is the weather look like today
1  what is the weather look like today
2                    what is for lunch
3                    what is for lunch
My Work
  • 2,143
  • 2
  • 19
  • 47