I think you misunderstand the trow here. As you have written it, it leaves the connection opened because the commit transaction; is not executed. The begin transaction; (increments the @@TRANCOUNT counter), but throw does not automatically decrese it! (no automatic rollback).
When throw is executed it tries to find a catch statement if not found only error is shown and the the rest is terminated.
To quote the MSDN:
If a TRY...CATCH construct is not available, the statement batch is
terminated. The line number and procedure where the exception is
raised are set. The severity is set to 16.
In your simple case you could extend it like this:
BEGIN TRY
begin transaction;
throw 50001, N'Exception', 1;
END TRY
BEGIN CATCH
IF <boolean_expression_for_commit>
commit transaction;
ELSE
rollback transaction;
END CATCH
That will depend on your usecase. Both commit and rollback decrement the @@TRANCOUNT counter. You will get your transaction closed based on the <boolean_expression_for_commit>.
EDIT The OP wanted to know more about XACT_ABORT
If the XACT_ABORT is set to ON the TROW causes rollback on the whole transaction even when CATCH is missing. That will decrese the transaction counter and close the open transaction.
There is a catch however. If a developer wants to create a log in a CATCH then rollback is performed also on the CATCH block! (no log is created)
Things can get weird when SET XACT_ABORT OFF (the default). It may or may not leave the transaction opened based on severity of the error. If the error is considered sereve enough it will still rollback. In your case a simple THROW is not severe enough.
Note: That is also a reason why now THROW should be used instread of RAISERROR. The THROW follows the XACT_ABORT setting, RAISEERROR ignores it.