Evaluate string in column of dataframe with a variable

Question

I have a dataframe of the following format:

Name           Factor     Expression   Year 
Hydro          0.075            <10    2010  
Hydro          0.075            >10    2010  
Hydro          0.075            <10    2011  
Hydro          0.075            >10    2011
Hydro          0.075            <10    2012

And the following variable: i=3.

I would like to filter the dataframe where the Expression column evaluates as true, when the variable i is on the left hand side of the string in the column of expression.

For example, the first row would evaluate to true as 3<10.

The resulting dataframe that I would like is:

Name           Factor     Expression   Year 
Hydro          0.075            <10    2010  
Hydro          0.075            <10    2011  
Hydro          0.075            <10    2012

Thank you for any help.

Answer 1

Pandas has a safer version of eval that supports a limited number of operations. Luckily, > and < work, and you can use this along with string concatenation:

i = '3'
idx = pd.eval(i + df.Expression)
df.loc[idx]

    Name  Factor Expression  Year
0  Hydro   0.075        <10  2010
2  Hydro   0.075        <10  2011
4  Hydro   0.075        <10  2012

---

As @coldspeed noted, the above approach only works on DataFrames that are < 100 rows*, which isn't ideal. He also proposed the following solution:

df[[pd.eval(f"{i}{j}") for j in df['Expression']]]

_{*The above limitation is discussed more in depth in the following question: AttributeError: 'PandasExprVisitor' object has no attribute 'visit_Ellipsis', using pandas eval}