Reading a csv (text) file with pkgutil.get_data

Question

so I'm writing some code which needs to pull configuration / data from CSV files, which are packaged with the application. From what I understand using pkgutil is the 'right' way to do this. So what I'm trying to do is:

import pkgutil
MatFile = pkgutil.get_data('impy.implosions', 'LILAC_Materials.csv')

which works fine and gives me the file's bytes. But I can't figure out how to feed this into csv.reader in a clean way. I found this old question but its solution would be something like:

MatFile = io.StringIO(MatFile)
dataReader = csv.reader(MatFile , delimiter=',')

which doesn't work since StringIO expects a str. The complementary functionality in io would be BytesIO, but then that doesn't help me since csv.reader can't handle it. It seems like this should have a simple solution, but I'm not familiar with handling byte data in python. Thanks!

Answer 1

In Python 3, the csv module's classes expect you to pass them an iterable that yields Unicode strings. If you are have your data as a single byte strings, you first need to decode the data, then split it up into lines.

Here's a quick bit of code that should work:

MatFile = pkgutil.get_data('impy.implosions', 'LILAC_Materials.csv')
dataReader = csv.reader(MatFile.decode('utf-8').splitlines(), delimiter=',')

I'm guessing the file was encoded with UTF-8 (or ASCII, which is a subset). If you know otherwise swap in the appropriate encoding in the decode call. str.splitlines takes care of breaking up the single string into a list of lines, which is perfectly acceptable as input to csv.reader.