It is not totally true that finally will always be executed. See this answer from Haacked:
Two possibilities:
- StackOverflowException
- ExecutingEngineException
The finally block will not be executed
  when there's a StackOverflowException
  since there's no room on the stack to
  even execute any more code. It will
  also not be called when there's an
  ExecutingEngineException, which is
  very rare.
In fact, for any sort of asynchronous exception (like StackOverflowException, OutOfMemoryException, ThreadAbortException) the execution of a finally block  is not guaranteed.
However, these exceptions are exceptions you usually cannot recover from, and in most cases your process will exit anyway.
In fact, there is also at least one other case where finally is not executed as described by Brian Rasmussen in a now deleted question:
The other case I am aware of is if a
  finalizer throws an exception. In that
  case the process is terminated
  immediately as well, and thus the
  guarantee doesn't apply.
The code below illustrates the problem
static void Main(string[] args) {
   try {
      DisposableType d = new DisposableType();
      d.Dispose();
      d = null;
      GC.Collect();
      GC.WaitForPendingFinalizers();
   } catch {
      Console.WriteLine("catch");
   } finally {
      Console.WriteLine("finally");
   }
}
public class DisposableType : IDisposable {
   public void Dispose() {
   }
   ~DisposableType() {
      throw new NotImplementedException();
   }
}
A reliable try/catch/finally will have to use Constrained Execution Regions (CER). An example is provided by MSDN:
[StructLayout(LayoutKind.Sequential)]
struct MyStruct
{
    public IntPtr m_outputHandle;
}
sealed class MySafeHandle : SafeHandle
{
    // Called by P/Invoke when returning SafeHandles
    public MySafeHandle()
        : base(IntPtr.Zero, true)
    {
    }
    public MySafeHandle AllocateHandle()
    {
        // Allocate SafeHandle first to avoid failure later.
        MySafeHandle sh = new MySafeHandle();
        RuntimeHelpers.PrepareConstrainedRegions();
        try { }
        finally
        {
            MyStruct myStruct = new MyStruct();
            NativeAllocateHandle(ref myStruct);
            sh.SetHandle(myStruct.m_outputHandle);
        }
        return sh;
    }
}
An excellent source of information is the following article:
Reliability Best Practices