Python: Better way to write nested for loops and if statements

Question

I am trying to find a more Pythonic way of doing the below.

for employee in get_employees:
    for jobs in employee['jobs']:
        for nemployee in employee_comps:
            if nemployee['employee_id'] == employee['id']:
                for njob in nemployee['hourly_compensations']:
                    if njob['job_id'] == jobs['id']:
                        njob['rate'] = jobs['rate']

It works but seems clunky. I'm new to Python, if there is another thread that will help with this please direct me there!

I'm a bit confused by the third loop. What is nemployee, employee_comps, njob? Can you elaborate more on the problem you're solving? I don't quite understands the structure of the dicts/lists you're working on. — Huy Ngo, Aug 24 '20 at 02:40
Looking at what you are trying to do this seems to be the logical way. You could do some preprocessing of the data so you could avoid the "search the entire array to find a match" logic, then use dictionaries to avoid the search loops. But since you don't break out of loops when the match is made, I have to assume each loop could have multiple matches. — RufusVS, Aug 24 '20 at 02:42
Does this answer your question? [In python is there an easier way to write 6 nested for loops?](https://stackoverflow.com/questions/1280667/in-python-is-there-an-easier-way-to-write-6-nested-for-loops) — Ilya Berdichevsky, Aug 24 '20 at 02:42
This seems like the kind of task that would be better-served using a database, by doing JOIN/WHERE type operations. Or, at least, a DataFrame that can serve as an in-memory pseudo-database. [`pandas`](https://pypi.org/project/pandas/) is the foremost python dataframe library, and is probably worth looking into, if you can structure your data into a format that's friendly to it. — Green Cloak Guy, Aug 24 '20 at 02:51

score 0 · Answer 1 · answered Aug 24 '20 at 03:00

The code you have is very clean and pythonic, I would suggest staying with that.

If you want it in one line, this should work, but I don't have data to test it on, so I'm not sure.

[[njob.update({njob['rate']: jobs['rate']}) for njob in nemployee['hourly_compensations'] if njob['job_id'] == jobs['id']] for employee in get_employees for jobs in employee['jobs'] for nemployee in employee_comps if nemployee['employee_id'] == employee['id']]

score 0 · Accepted Answer · answered Aug 24 '20 at 03:10

The main comment I would make about the code is that you are free to change the order of the outer three for loops because the operation that you are performing does not depend on the order that you loop over these (as you are not breaking out of any loops when finding a match), and that given that this is the case, there is no point in doing the jobs loop only to reach an if statement inside it that is independent of the value of jobs. It would be more efficient to put the jobs loop inside the other two, so that it can also be inside the if, i.e. the loop is only performed for those combinations of values of employee and nemployee where the if condition evaluates True.

Beyond this but less importantly, where there are consecutive for statements (over independent iterables) after doing this rearrangement, you could replace them with a single loop over an itertools.product iterator to reduce the depth of nesting of for loops if you wish (reducing it from four to two explicit loops):

from itertools import product

for employee, nemployee in product(get_employees, employee_comps):
    if nemployee['employee_id'] == employee['id']:
        for jobs, njob in product(employee['jobs'],
                                  nemployee['hourly_compensations']):
            if njob['job_id'] == jobs['id']:
                njob['rate'] = jobs['rate']

Thank you! This is what I was looking for! – Kia Martinez Aug 24 '20 at 11:23 — Kia Martinez, Aug 24 '20 at 11:23

Python: Better way to write nested for loops and if statements

2 Answers2