1

I have a pandas dataframe that requires a lot of conditionals to calculate specific values and it is taking quite a lot of time to run. I was wondering if anyone could suggest a better way of handling this. The code is as follows:

for value in value_range:
del final_list[:]
if value>=340 and value<=640:
    for time in df_1.index:
        if (value >= 340 and value <= 360):
            rad = band_correction_interpolation(360, df_2, time, st0[str(value)], ft0['value'])
            final_list.append(rad)
        elif (value > 360 and value < 380):
            rad = interpolation(df_1, df_2, value, 360, 380, time, st0, ft0)
            final_list.append(rad)
        elif (value > 380 and value <= 410):
            if value == 410:
                rad = interpolation(df_1, df_2, value, 380, 440, time, st0, ft0)
            else:
                rad = interpolation(df_1, df_2, value, 380,410, time, st0, ft0)
            final_list.append(rad)
        df_final[str(value)] = final_list          
elif (value > 640 and value<= 2500):
    for time in df_1.index:
        if (value > 640 and value < 840):
            rad = interpolation(df_1, df_2, value, 640, 840, time, st0, ft0)
            final_list.append(rad)
        elif (value >= 840 and value <= 1000):
            rad = band_correction_interpolation(840, df_2, time, st0[str(value)], ft0['value'])
            final_list.append(rad)
        elif (value >= 1640 and value <=2500):
            rad = band_correction_interpolation(1640, df_2, time, st0[str(value)], ft0['value'])
            final_list.append(rad)
    df_final[str(value)] = final_list

Unfortunately, this is part of a bigger program and it wouldn't really make sense to show an output or the input data. The gist of it is that, I need to apply mathematical calculations with different parameters based on what is in 'value'.

Brain_overflowed
  • 426
  • 1
  • 5
  • 21
  • You need to shorten this question. A lot. Absolutely no need to show all your code here to get point across. Please read: https://stackoverflow.com/help/mcve – JohnE Oct 13 '17 at 16:36
  • That said, I would use `numpy.select()` in a case like this and it will be much faster. Also could do nested `numpy.where()` altho that leads to much less readable code IMO. Also could try numba. – JohnE Oct 13 '17 at 16:37
  • Can I use numpy.select() directly on a pandas dataframe? I am very new to pandas and numpy, I have never used it before but I will look into it @JohnE – Brain_overflowed Oct 13 '17 at 17:34
  • this might be a good starting point: https://stackoverflow.com/questions/19913659/pandas-conditional-creation-of-a-series-dataframe-column/19913845#19913845 Also from that link note all the linked answer to the right of it – JohnE Oct 13 '17 at 17:42
  • The link you referred to has a far simpler condition set...I needed help because of the complexity and sheer number of parameters that needs checking. Thanks anyways for trying to help from what you know @JohnE – Brain_overflowed Oct 18 '17 at 22:26
  • Just FYI for anyone else, using np.select increased my runtime by approximately 4min – Brain_overflowed Oct 19 '17 at 15:37

0 Answers0