How to handle skipped items in a for loop

Question

I have been using the Amazon Product API with bottlenose and when iterating over the XML response I have encountered null type errors.

I think I have half managed to handle them however it won't extract the other information if it encounters this error thus showing fewer results than there are.

Is there a way to handle this properly so all the information is extracted and the error is ignored?

price_list = [{}]
    for i in price_search:
      lnp = i.LowestNewPrice.FormattedPrice.text
      qty_n = i.TotalNew.text
      qty_u = i.TotalUsed.text
      int_qty_u = int(qty_u)
    if int_qty_u > 0:
      lup = i.LowestUsedPrice.FormattedPrice.text
    else:
        continue
    price_list.append({'Lowest New Price': lnp, 'Lowest Used Price': lup, 'Quantity New': qty_n, 'Quantity Used': qty_u})

In this instance it is specifically the LowestUsedPrice, if an item doesn't have this tag then the error is raised.
I am new to Python and coding so struggling along as best I can...

Use a try block around the potential error raiser code and give it some treatment in the except clause (or do nothing to ignore). Use the raised error exception name in the except clause. — progmatico, Oct 14 '18 at 19:08
Ok, I will have a go at this. Is there a way I could find out which tag is raising the error in this exception? — Dan, Oct 14 '18 at 19:35

MAP · Accepted Answer · 2018-10-14T19:43:29.457

1

I believe you have a bad indentation problem. Python defines a block by its indentation. Your if/else structure is outside the for loop. This is what you might be looking for:

price_list = [{}]
for i in price_search:
  lnp = i.LowestNewPrice.FormattedPrice.text
  qty_n = i.TotalNew.text
  qty_u = i.TotalUsed.text
  int_qty_u = int(qty_u)
  if int_qty_u > 0:
      lup = i.LowestUsedPrice.FormattedPrice.text
  else:
      continue
  price_list.append({'Lowest New Price': lnp, 'Lowest Used Price': lup, 'Quantity New': qty_n, 'Quantity Used': qty_u})

Other than that, use a try-except clause to handle exceptional values or cases and return the program to a valid state. An example could be:

 if int_qty_u > 0:
      try:
          lup = i.LowestUsedPrice.FormattedPrice.text
      except: #we catch any exception that could happend
          lup = '<null>' #just to put a string

for sake of completition i would do a try-except on all the for's block:

price_list = [{}]
for i in price_search:
  try:
      lnp = i.LowestNewPrice.FormattedPrice.text
      qty_n = i.TotalNew.text
      qty_u = i.TotalUsed.text
      int_qty_u = int(qty_u)
      if int_qty_u > 0:
          lup = i.LowestUsedPrice.FormattedPrice.text
      else:
         continue
  except:
      lnp,qty_n,qty_u,int_qty_u='null','null','null',-1 #multiple assignment in a bad case
  price_list.append({'Lowest New Price': lnp, 'Lowest Used Price': lup, 'Quantity New': qty_n, 'Quantity Used': qty_u})

edited Oct 14 '18 at 19:43

answered Oct 14 '18 at 19:10

MAP

397
2
8

Thanks, the indentation problem was just me trying to paste it properly into the post question box :/ – Dan Oct 14 '18 at 19:30
Yeah @Dan, it gets pretty hard to format code, speacially of python. I updated the answer with an example of the try-except. – MAP Oct 14 '18 at 19:43
This has really helped clear things up a lot, that try block should cover if any of the other tags have null values as well I guess? – Dan Oct 14 '18 at 19:54
it should cover those other tags as well. For example if ' i ' does have a valid value for LowestUsedPrice tag but not for TotalNew tag then the code will still work although you don't know which sentence produced the exception (unless you introduce more code you don't know which tag has null values, but you might only be interested in providing a solution with the except clause ). – MAP Oct 14 '18 at 20:15
1

I would just add that OP (@Dan) should learn more about exceptions and try blocks before he starts applying a catch-all except clause like this "everywhere". See [here](https://stackoverflow.com/questions/4990718/about-catching-any-exception/4992124#4992124) for example. – progmatico Oct 14 '18 at 21:08
Ok, so the code above works well enough and I have done some research into the try clause block as mentioned above and narrowed it down to catching only AttributeError. I have run into a strange problem though whereby if I change the page number of the results, the values that aren't null are being overwritten with null. The error tells me that the specific tag doesn't exist when it clearly does as I'm looking at it. Is there a way I can go through the error step by step and find where the problem is? – Dan Oct 16 '18 at 20:54

score 0 · Answer 2 · answered Oct 14 '18 at 19:41

continue goes to the next iteration of the loop, so you skip the rest of the loop body. Instead of that, you should assign a default value to the variable.

if int_qty_u > 0:
    lup = i.LowestUsedPrice.FormattedPrice.text
else:
    lup = "some default value"

score 0 · Answer 3 · answered Oct 14 '18 at 20:07

You could also try adding a check if the item has the tag LowestUsedPrice:

price_list = [{}]

for i in price_search:
    lnp = i.LowestNewPrice.FormattedPrice.text
    qty_n = i.TotalNew.text
    qty_u = i.TotalUsed.text
    int_qty_u = int(qty_u)
if int_qty_u > 0 and i.LowestUsedPrice != None:
    lup = i.LowestUsedPrice.FormattedPrice.text
else:
    lup = 'null'

price_list.append({'Lowest New Price': lnp, 'Lowest Used Price': lup, 'Quantity New': qty_n, 'Quantity Used': qty_u})

How to handle skipped items in a for loop

3 Answers3