This unit test does what you're describing. It does the same thing both with and without calling Dispose to show what happens without calling Dispose.
Both methods create a file, open the file, write to it, then open it again and write to it again.
The first method throws an IOException because the StreamWriter wasn't disposed. The file is already open and can't be opened again.
The second one disposes before trying to reopen the file, and that works without an exception. 
[TestClass]
public class DisposableTests
{
    [TestMethod]
    [ExpectedException(typeof(IOException))]
    public void DoesntDisposeStreamWriter()
    {
        var filename = CreateFile();
        var fs = new StreamWriter(filename);
        fs.WriteLine("World");
        var fs2 = new StreamWriter(filename);
        fs2.WriteLine("Doesn't work - the file is already opened.");
    }
    [TestMethod]
    public void DisposesStreamWriter()
    {
        var filename = CreateFile();
        var fs = new StreamWriter(filename);
        fs.WriteLine("World");
        fs.Dispose();
        var fs2 = new StreamWriter(filename);
        fs2.WriteLine("This works");
        fs2.Dispose();
    }
    private string CreateFile()
    {
        var filename = Guid.NewGuid() + ".txt";
        using (var fs = new StreamWriter(filename))
        {
            fs.WriteLine("Hello");
        }
        return filename;
    }
}
You likely wouldn't see the problem with memory use because that's not specifically what IDisposable addresses. All objects use memory, but most aren't disposable. The garbage collector reclaims memory from objects that are no longer referenced (like an object that's created in a method but goes out of scope when the method ends.)
IDisposable is for scenarios where the class holds onto some resource that isn't garbage collected. Another example is SqlConnection. It's not about memory, it's about the connection to the SQL Server. There are only so many available. (It will eventually get released, but not in a predictable manner - that's a digression.) 
The exact reasons why classes might be IDisposable vary. They often have nothing in common. IDisposable doesn't care what the reason is. It just means that there's something in the class that needs to be cleaned up.