Nested for loop stop if null value using Scrapy

Question

I use a nested for loop to get data of Weekdays. If one of the days is 'null' the loop stops at that day and doesn't get the rest of the days. I believe that I need to use some 'if' statement at some point but it still doesn't work for me after many tries.

Here are the code I use before trying if statements:

    try:

        Sunday = []
        Monday = []
        Tuesday = []
        Wednesday = []
        Thursday = []
        Friday = []
        Saturday = []

        for i in data[1][2]:
            for j in i[1]:
                for x in str(j[1]).split(', '):
                    if i[0] == 7:
                        Sunday.append(x)
                    elif i[0] == 1:
                        Monday.append(x)  
                    elif i[0] == 2:
                        Tuesday.append(x)
                    elif i[0] == 3:
                        Wednesday.append(x)  
                    elif i[0] == 4:
                        Thursday.append(x)  
                    elif i[0] == 5:
                        Friday.append(x)  
                    elif i[0] == 6:
                        Saturday.append(x)                                 
                                                                                                                                                                                                                                                        
    except:

If i[1] value are null at any day of the weekdays, the scraper doesn't get that day nor the days after it. I tried to use if statement to check if j is null and continue but still does not work for me.

score 0 · Answer 1 · answered May 06 '22 at 21:34

0

Do you check the lenght of i before accessing the second element? Maybe you can add a guard like:

for i in data[1][2]:
    if len(i) > 1 and i[0] and i[1]:
        for j in i[1]:
            # ...

the same consideration can be done for data.

answered May 06 '22 at 21:34

LordAlucard90

1
1

The issue is in the second loop for j in i[1]: and I tried this if statement but no luck. – Ahmed Maher May 07 '22 at 12:44

score 0 · Answer 2 · answered May 13 '22 at 04:35

0

I used nested for loops in a list comprehension. That solved the issue

            try:
                Sunday = [x for i in data[1][2] for j in str(i[1])]
            except:
                Sunday = []
            try:        
                Monday = [x for i in data[1][3] for j in str(i[1])]
            except:
                Monday = []
            .........

answered May 13 '22 at 04:35

Ahmed Maher

103
1
10

You shouldn't use bare except clauses: https://stackoverflow.com/questions/54948548/what-is-wrong-with-using-a-bare-except – Tzane May 13 '22 at 12:54
I understand. I try with a solution for my use case and I will share it here if I applied a good one. – Ahmed Maher May 17 '22 at 06:54

Nested for loop stop if null value using Scrapy

2 Answers2