Pandas not able to open file with umlauts in path

Question

Try to read the content from file hello.txt at path C:/temp/Kälte/Grad°/:

Message; Content

Greeting; Hello World!

path = 'C:/temp/Kälte/Grad°/hello.txt'

# OKAY
data = ""
with open(path) as f:
    data = f.read()
print("file content:\n" + data)

# OKAY
import csv
with open(path, 'r') as csvfile:
    print("reading csv file:\n"+path)
    reader = csv.reader(csvfile, delimiter=';')
    for row in reader:
        print(row)
     
# FAILURE
import pandas as pd
print("reading csv file with pandas:\n"+path)
dataframe = pd.read_csv(path, sep=';',names=["Message","Content"],header=1)
print (dataframe)

I remember the exact same problem from several years ago, and I wonder why nobody resolves such a basic issue in pandas. Doesn't pandas call python's open internally (because then it should work, shouldn't it)? Or am I doing it wrong?

Answer 1

This appears to be a recurrence of this issue, but the solutions there didn't help on my set-up.

Here's a manual solution if there's no other option:

path = r'C:/temp/Kälte/Grad°/hello.txt'

import pandas as pd
import csv

with open(path, 'r') as fp:
    reader = csv.reader(fp, delimiter=';')
    cols = next(reader)
    df = pd.DataFrame(list(reader), columns=cols)

print(df)

    Message        Content
0  Greeting   Hello World!