dumping column data from data frames to list in python

Question

import numpy as np
import pandas as pd


def ExtractCsv( Start,End):
    lsta,lstb,lstc,lstd= list(),list(),list(),list()
    j=0;
    for i in range(Start,End+1):

        f1 = pd.read_csv('C:/Users/sanilm/Desktop/Desktop_backup/opc/csv/Newfolder/fault_data.tar/fault_data/test/healthy'+ str(i)+'.csv');

        listc=list(f1['c'])
        listd=list(f1['d'])
        liste=list(f1['e'])
        listf=list(f1['f'])

        lsta.append(listc)
        lstb.append(listd)
        lstc.append(liste)
        lstd.append(listf)

    print(lsta)
    return f1

f1=ExtractCsv(1,3)

input csv file there are 3 files:
a   b   c   d   e   f
1   10  2901.1  13.915  39.812  6.2647
1   10  2906.1  13.368  42.083  12.945
1   10  2805.3  12.951  42.261  13.398
1   10  3049.2  14.101  43.499  15.237
1   10  2854.8  13.978  42.699  9.1297

expected output: [2901.1, 2906.1, 2805.3, 3049.2, 2854.8, 2860.9, 2992.9, 2867.1, 2947.6, 2679.4, 2891.2, 2853.8, 2896.4, 3114.6, 3155.3, 2930.2, 2810.0, 2903.5]

but the output am getting is [[2901.1, 2906.1, 2805.3, 3049.2, 2854.8], [2860.9, 2992.9, 2867.1, 2947.6, 2679.4, 2891.2], [2853.8, 2896.4, 3114.6, 3155.3, 2930.2, 2810.0, 2903.5]]

any suggestions on how can i achieve my expected output

Use `lsta.extend(listc)` instead of `append`. – hpaulj Aug 07 '18 at 05:31 — hpaulj, Aug 07 '18 at 05:31

score 0 · Accepted Answer · answered Aug 07 '18 at 04:42

0

It looks like you just want to flatten your results.

Maybe try this: Making a flat list out of list of lists in Python

From link above (but there you can look up more information):

flat_list = [item for sublist in l for item in sublist]

answered Aug 07 '18 at 04:42

franiis

1,378
1
18
33

jezrael · Answer 2 · 2018-08-07T04:54:21.080

In loop you can create list of DataFrames and then concat together.

Also if many columns in file is possible add parameter usecols to read_csv for read only specified columns names:

def ExtractCsv(Start,End):
    dfs = []
    for i in range(Start,End+1):
        path = 'C:/Users/sanilm/Desktop/Desktop_backup/opc/csv/Newfolder/fault_data.tar/fault_data/test/healthy'+ str(i)+'.csv'
        f1 = pd.read_csv(path, usecols=['c','d','e','f'])
        dfs.append(f1)

    return pd.concat(dfs, ignore_index=True)

df = ExtractCsv(1,3)

Last if need extract some column to list:

lsta = df['c'].tolist()

dumping column data from data frames to list in python

2 Answers2