1

I have a large dataframe with many rows and columuns.

An example of the structure is:

a = np.random.rand(6,3)
df = pd.DataFrame(a)

I'd like to split the DataFrame into seperate data frames each consisting of 3 rows.

agf1997
  • 2,668
  • 4
  • 21
  • 36

2 Answers2

2

You can use numpy.split() method:

In [8]: df = pd.DataFrame(np.random.rand(9, 3))

In [9]: df
Out[9]:
          0         1         2
0  0.899366  0.991035  0.775607
1  0.487495  0.250279  0.975094
2  0.819031  0.568612  0.903836
3  0.178399  0.555627  0.776856
4  0.498039  0.733224  0.151091
5  0.997894  0.018736  0.999259
6  0.345804  0.780016  0.363990
7  0.794417  0.518919  0.410270
8  0.649792  0.560184  0.054238

In [10]: for x in np.split(df, len(df)//3):
    ...:     print(x)
    ...:
          0         1         2
0  0.899366  0.991035  0.775607
1  0.487495  0.250279  0.975094
2  0.819031  0.568612  0.903836
          0         1         2
3  0.178399  0.555627  0.776856
4  0.498039  0.733224  0.151091
5  0.997894  0.018736  0.999259
          0         1         2
6  0.345804  0.780016  0.363990
7  0.794417  0.518919  0.410270
8  0.649792  0.560184  0.054238
MaxU - stand with Ukraine
  • 205,989
  • 36
  • 386
  • 419
  • doesn't this create a list with the values rather then separate dataframes? Is there a way to do something similar with pd? – agf1997 Nov 05 '16 at 00:23
  • no, it creates a list of DFs. Alternatively you can put into a dictionary, naming those DFs as you wish. How are you going to dynamically name your new DFs? – MaxU - stand with Ukraine Nov 05 '16 at 00:26
2

you can use groupby

g = df.groupby(np.arange(len(df)) // 3)

for n, grp in g:
    print(grp)

          0         1         2
0  0.278735  0.609862  0.085823
1  0.836997  0.739635  0.866059
2  0.691271  0.377185  0.225146
          0         1         2
3  0.435280  0.700900  0.700946
4  0.796487  0.018688  0.700566
5  0.900749  0.764869  0.253200

to get it into a handy dictionary

mydict = {k: v for k, v in g}
piRSquared
  • 285,575
  • 57
  • 475
  • 624
  • g is a `DataFrameGroupBy object` how would I use that? How does one index into a specific group (e.g. the 3rd group of 3 values)? – agf1997 Nov 05 '16 at 00:37
  • @pi followup ... is there an easy way to set the index values of `mydict[>0]` to 0, 1, 2? Thanks so much! – agf1997 Nov 05 '16 at 00:56