You shouldn't be using Application.Current.Shutdown at all. The proper way to exit from an application is to let the user close the main window. Using Shutdown is a bad idea for many reasons - you'll lose the ability to abort the shutdown, for example (the typical scenario being a form saying "You have unsaved changes").
EDIT:
The OP made it clear that his application has no main window, so using Application.Current.Shutdown is in fact appropriate. However, it should only be called after all the critical background work is safely finished (or cancelled).
Task and BackgroundWorker by default use a background thread (a thread pool thread, in fact), which doesn't prevent the application from exiting. When the process exits, all the background threads simply die - no exception, no logging, no finalization. Do not use background threads for critical tasks. However, note that the alternative would be that the threads wouldn't exit at all and the process would continue running - you have to handle the graceful shutdown of all background tasks yourself.
In other words, when the user presses Exit in your context menu, cancel any running background work, wait for it to finish properly and only call Shutdown after all that has successfully completed.