0

When I try to read csv file I get an error pandas.errors.ParserError: Error tokenizing data. C error

I have a set of different csv files and see this error occurs across different files

list_ = []
for file_ in allFiles:
    try:
        df = pd.read_csv(file_, index_col=None, header=None, sep=',')
        list_.append(df)
        except pd.errors.EmptyDataError:
            continue

I am trying to see if I can skip reading the entire csv file if this error (pandas.errors.ParserError: Error tokenizing data. C error) occurs while reading a csv file

scott martin
  • 1,253
  • 1
  • 14
  • 36

1 Answers1

0

Request you to try with

data = pd.read_csv('file1.csv', error_bad_lines=False)

Do note that using error_bad_lines=False will cause the offending lines to be skipped try this ... If it help else i will try to edit and solve it further

Puneet Sinha
  • 1,041
  • 1
  • 10
  • 23
  • also ... i saw some help in https://stackoverflow.com/questions/18039057/python-pandas-error-tokenizing-data – Puneet Sinha Jan 29 '19 at 09:20
  • thanks for the comment, yes I have tried that option as well but it dint help. I am trying to see a way how I could delete these erroneous files – scott martin Jan 29 '19 at 09:21
  • yes I have tried the suggestions in that link. As I read as I am reading a bunch of files and these errors throw up randomly so I am trying to find a way to skip these erroneous files to overcome this problem.. – scott martin Jan 29 '19 at 10:15