Include is an extension method which is used by EF to enable Eager Loading of your Entities.
In the first case, by specifying Include("Artist"), when you retrieve Album entities from your Albums set, you are instructing LINQ to also retrieve the associated Artist entity (usually associations are through foreign keys in the database, although you can associate in your model as well).
In the second case, you aren't pulling through any related entities when Albums are fetched.
Note that as of Entity Framework 4.1 and onwards, there is a preferred mechanism for using Include which takes a lambda, so the string associations become strongly typed, i.e.:
var album = storeDB.Albums.Include(alb => alb.Artist).ToList();
Just remember to import System.Data.Entity when using the lambda extension.