Overpass a custom exception but print all the other exceptions

Question

I am running the following try-except code:

try:
    paths = file_system_client.get_paths("{0}/{1}/0/{2}/{3}/{4}".format(container_initial_folder, container_second_folder, chronological_date[0], chronological_date[1], chronological_date[2]), recursive=True)
    list_of_paths=["abfss://{0}@{1}.dfs.core.windows.net/".format(storage_container_name, storage_account_name)+path.name for path in paths if ".avro" in path.name]

except Exception as e:
    if e=="AccountIsDisabled":
        pass
    else:
        print(e)

I want neither to print the following error if my try-except fells upon it nor to stop my program execution if I fell upon this error:

"(AccountIsDisabled) The specified account is disabled.
RequestId:3159a59e-d01f-0091-5f71-2ff884000000
Time:2020-05-21T13:09:03.3540242Z"

I just want to overpass it and print any other error/exception (eg. TypeError, ValueError, etc) that will occur.

Is this feasible in Python 3?

Please note that the .get_paths() method belongs to the azure.storage.filedatalake module which enables direct connection of Python with Azure Data Lake for path extraction.

I am giving the note to pinpoint that the Exception I am trying to bypass is not a built-in Exception.

[Update] In sort after following the proposed attached answers I modified my code to this:

import sys
from concurrent.futures import ThreadPoolExecutor
from azure.storage.filedatalake._models import StorageErrorException
from azure.storage.filedatalake import DataLakeServiceClient, DataLakeFileClient

storage_container_name="name1" #confidential
storage_account_name="name2" #confidential
storage_account_key="password" #confidential 
container_initial_folder="name3" #confidential
container_second_folder="name4" #confidential

def datalake_connector(storage_account_name, storage_account_key):
    global service_client

    datalake_client = DataLakeServiceClient(account_url="{0}://{1}.dfs.core.windows.net".format("https", storage_account_name), credential=storage_account_key)
    print("Client successfuly created!")

    return datalake_client

def create_list_paths(chronological_date, 
                      container_initial_folder="name3", 
                      container_second_folder="name4", 
                      storage_container_name="name1", 
                      storage_account_name="name2"
                      ):
    list_of_paths=list()
    print("1. success")
    paths = file_system_client.get_paths("{0}/{1}/0/{2}/{3}/{4}".format(container_initial_folder, container_second_folder, chronological_date[0], chronological_date[1], chronological_date[2]), recursive=True)
    print("2. success")
    list_of_paths=["abfss://{0}@{1}.dfs.core.windows.net/".format(storage_container_name, storage_account_name)+path.name for path in paths if ".avro" in path.name]
    print("3. success")
    list_of_paths=functools.reduce(operator.iconcat, result, [])

    return list_of_paths

service_client = datalake_connector(storage_account_name, storage_account_key)
file_system_client = service_client.get_file_system_client(file_system=storage_container_name)

try:
    list_of_paths=[]
    executor=ThreadPoolExecutor(max_workers=8)
    print("Start path extraction!")
    list_of_paths=[executor.submit(create_list_paths, i, container_initial_folder, storage_container_name, storage_account_name).result() for i in date_list]

except:
    print("no success")
    print(sys.exc_info())

Unfortunately the StorageErrorException cannot be handled for a reason, I am still getting the following stdout:

you can have multiple `except`-statements to match against different types of exceptions. You can match multiples at the same time too (https://stackoverflow.com/questions/6470428/catch-multiple-exceptions-in-one-line-except-block). As long as you can access the actual exception type, you'll be fine. Try and find out where that particular exception is defined and import it. — Filip Allberg, May 21 '20 at 13:28
No, what you did in the update is wrong. You should revert it. — CristiFati, May 21 '20 at 14:23
@CristiFati Check the last part of my edit! About the StorageErrorException — NikSp, May 21 '20 at 14:24
Yes, because you get it before the *try* / *except* block :) you should look at the stcktrace which contains the line that raised it. — CristiFati, May 21 '20 at 14:25
@CristiFati please check my latest update on my question. Sorry for insisting on this. If we solve it I will build you a statue :) — NikSp, May 21 '20 at 14:46
Ohhh you saw it on my stctrace.....Really don't know why it's there — NikSp, May 21 '20 at 15:57
@CristiFati Maybe because I don't have my Azure Subscription active, Azure cannot let me to interpret this error with Python. I don't that my inactive subscription is an issue, although I cannot find any other plausible solution to this. It's really weird. — NikSp, May 21 '20 at 16:15
No, you have the code locally. Print some additional info about it. Add: `except: print(sys.exc_info())` — CristiFati, May 21 '20 at 17:12
@CristiFati if I don't define a StorageExceptionError I get the following error ```NameError: name 'StorageErrorException' is not defined``` So the ```print(sys.exc_info()) ``` don't even get printed... If set a class StorageErrorException and I ```raise StorageErrorException``` inside a try statement then I get the error without Python pass-ing it. — NikSp, May 21 '20 at 18:50
Ok agreee with you. In that scenario even when I clear all the results of my Jupyter notebook I get the ```NameError: name 'StorageErrorException' is not defined``` when using ```try:.... except:....``` — NikSp, May 21 '20 at 19:18
you can't possibly get the name error as you removed all `StorageErrorException` references. Place all your code, not only parts leaving people to guess what the missing code is. — CristiFati, May 21 '20 at 19:41
@CristiFati I have added more code I hope it will help. Sorry for not revealing extra information about some variables since they are confidential. If you have an Azure Data lake account use yours. — NikSp, May 21 '20 at 20:33
I didn't say `except StorageErrorException:`. I said `except:`. — CristiFati, May 21 '20 at 20:39
@CristiFati Output of ```except: print(sys.exc_info())``` is: ```(, StorageErrorException('(AccountIsDisabled) The specified account is disabled.\nRequestId:0b47c2e9-601f-0094-49b0-2f2a5f000000\nTime:2020-05-21T20:41:39.4946101Z'), )``` — NikSp, May 21 '20 at 20:42
@CristiFati I uploaded a new stdout... supporting my previous comment — NikSp, May 21 '20 at 20:55
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/214367/discussion-between-cristifati-and-niksp). — CristiFati, May 21 '20 at 21:17

CristiFati · Accepted Answer · 2020-05-21T22:21:12.183

3

Listing [Python 3.Docs]: Compound statements - The try statement.

There are several ways of achieving this. Here's one:

try:
    # ...
except StorageErrorException:
    pass
except:
    print(sys.exc_info()[1])

Note that except: is tricky because you might silently handle exceptions that you shouldn't. Another way would be to catch any exception the code could raise explicitly.

try:
    # ...
except StorageErrorException:
    pass
except (SomeException, SomeOtherException, SomeOtherOtherException) as e:
    print(e)

Quickly browsing [MS.Docs]: filedatalake package and the sourcecode, revealed that StorageErrorException (which extends [MS.Docs]: HttpResponseError class) is the one that you need to handle.

Might want to check [SO]: About catching ANY exception.

Related to the failure of catching the exception, apparently there are 2 having the same name:

azure.storage.blob._generated.models._models_py3.StorageErrorException (currently imported)
azure.storage.filedatalake._generated.models._models_py3.StorageErrorException

I don't know the rationale (I didn't work with the package), but given the fact the package raises an exception defined in another package when it also defines one with the same name, seems lame.
Anyway importing the right exception solves the problem.

As a side note, when dealing with this kind of situation, don't only import the base name, but work with the fully qualified one:

import azure.storage.filedatalake._generated.models.StorageErrorException

edited May 21 '20 at 22:21

answered May 21 '20 at 13:30

CristiFati

38,250
9
50
87

nice, for some reason I remembered python didn't allow multiple except clauses, that's definitely the best way to do it! – Adam.Er8 May 21 '20 at 13:32
It looks so balanced, let's try it out! – NikSp May 21 '20 at 13:45
I guess that the AccountIsDisabled here is the class defined by @Adam.Er8 – NikSp May 21 '20 at 13:47
O, so the exception isn't defined by the 3rd party package? Looking into it. – CristiFati May 21 '20 at 13:53
StorageErrorException does not work for a reason. This is what I tried to express in my updated answer. :( – NikSp May 21 '20 at 14:13
I agree that the StorageErrorException is the one to handle but I cannot pass it as I upload it to my answer – NikSp May 21 '20 at 14:21
What do you mean by "*I cannot pass it as I upload it to my answer*"? – CristiFati May 21 '20 at 14:24

score 0 · Answer 2 · answered May 21 '20 at 13:25

0

you want to compare the type of the exception, change your condition to:

if type(e)==AccountIsDisabled:

example:

class AccountIsDisabled(Exception):
    pass

print("try #1")
try:
    raise AccountIsDisabled
except Exception as e:
    if type(e)==AccountIsDisabled:
        pass
    else:
        print(e)

print("try #2")
try:
    raise Exception('hi', 'there')
except Exception as e:
    if type(e)==AccountIsDisabled:
        pass
    else:
        print(e)

Output:

try #1
try #2
('hi', 'there')

answered May 21 '20 at 13:25

Adam.Er8

12,675
3
26
38

1

Thank you for the answer, let me attach it to my code and let you know about the result!\ – NikSp May 21 '20 at 13:28
Not very idiomatic – Filip Allberg May 21 '20 at 13:32
The good think is that it worked...so it's a solution. And probably if I want more exceptions to overpass I can follow the same approach. The only this that still confuses me is how Python can understand which error is in a multiple exception logic. So If I want to overapss two custom Exceptions how python gets that this is an AccountIsDisabled exception and not a ValueError. I write my thoughts. Your solution works in any case. – NikSp May 21 '20 at 13:37

Overpass a custom exception but print all the other exceptions

2 Answers2