One very very important difference between those is that Thread.Sleep() is blocking thread for a duration, means thread won't be available to thread pool, while Task.Delay() creates a task and immediately release thread, so that other tasks can use that thread.
Thread.Sleep may easily lead to thread starvation.
To demonstrate (notice I have changed return type to Task):
static void Main(string[] args)
{
Task.WaitAll(Enumerable.Range(0, 1000).Select(o => doTask()).ToArray());
Console.WriteLine("1");
Task.WaitAll(Enumerable.Range(0, 1000).Select(o => doTask2()).ToArray());
Console.WriteLine("2");
}
public async static Task doTask() => await Task.Delay(TimeSpan.FromSeconds(1));
public async static Task doTask2() => await Task.Factory.StartNew(() => Thread.Sleep(TimeSpan.FromSeconds(1)));
You will see 1 pretty quickly, but it takes a lot of time (several minutes on my 8 core CPU) for 2 to appears.
Rule of thumbs: using threads - use Thread.Sleep, using tasks - stick to Task.Delay.