I have seen numerous other questions similar to this but did not find my answer there.
My problem was that I was creating threads with the following flow:
private void btn_Click(object sender, EventArgs e)
{
    service.GetCount(
        (count, ex) =>
        {
            if (ex != null)
                return;
            for (int i = 0; i < count; i++)
            {
                service.Get(onItemReceived, i);
            }
        }
    );
}
public void GetCount(Action<int, Exception> callback)
{
    var callingThread = TaskScheduler.FromCurrentSynchronizationContext();
    Func<int> action = () =>
    {
        return client.GetCount(); // Synchronous method, could take a long time
    };
    Action<Task<int>> completeAction = (task) =>
    {
        Exception ex = (task.Exception != null) ? task.Exception.InnerException : task.Exception;
        if (callback != null)
            callback(task.Result, ex);
    };
    Task.Factory.StartNew(action).ContinueWith(completeAction, callingThread);
}
public void Get(Action<object, Exception> callback, int index)
{
    var callingThread = TaskScheduler.FromCurrentSynchronizationContext();
    Func<object> action = () =>
    {
        return client.Get(index); // Synchronous method, could take a long time
    };
    Action<Task<object>> completeAction = (task) =>
    {
        Exception ex = (task.Exception != null) ? task.Exception.InnerException : task.Exception;
        if (callback != null)
            callback(task.Result, ex);
    };
    Task.Factory.StartNew(action).ContinueWith(completeAction, callingThread);
}
In this way, each of my service's asynchronous methods would callback the thread they were originally called on (generally the UI thread). So I am simulating how the await/async keywords work (I cannot use .NET 4.5).
The problem with this pattern is that I get inexplicably locked to the UI thread after the first call to "ContinueWith". So in this case if I attempt to spawn 5 threads to each process a synchronous function Get, they will execute 1 by 1 instead of in parallel and they will block the UI thread while doing so, even if I try specifying TaskCreationOptions.LongRunning.
This never happens with my first call to Task.Factory.StartNew, it only happens to subsequent calls from within the first callback.