C# - Odd Null Reference Exception during testing, why does this happen?

Question

This references my last question which appears to have been abandoned. I am experiencing an odd "bug" if you will with C# and MS VS 2015. To reproduce the error, follow the steps:

1. Open console app project and copy paste code below.
2. Set a break point here:
3. First run code past break point, it works! :D
4. Then run code again but this time STOP at the break point and DRAG the executing statement cursor INTO the if statement from here: to here:

Hit Continue and an NRE exception is thrown. Why does this happen? Is it just me? What is the technical explination for this?

CODE:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace testapp
{
    class Program
    {
        static void Main(string[] args)
        {
            FILECollection randomCollection = new FILECollection();
            // Fill with junk test data:
            for(int i = 0; i<10; i++)
            {
                FILE junkfile = new FILE() { fileName = i.ToString(), folderName = i.ToString(), fileHashDigest = new byte[1] };
                randomCollection.Add(junkfile);
            }

            if (true)
            {
                Console.WriteLine("testing this weird exception issue...");
                FILE test;
                test = new FILE();
                test.fileName = "3";
                test.folderName = "3";
                test.fileHashDigest = new byte[1];

                FILE exists = randomCollection.Where(f => f.fileName == test.fileName &&
                                              f.fileHashDigest.SequenceEqual(test.fileHashDigest)).First();
            }
        }
    }


    public class FILE
    {
        public FILE() { _fileName = "";}
        private string _fileName;
        public string fileName
        {

            get
            {
                    if (false)
                        return this._fileName.ToUpper();
                    else
                        return this._fileName;
            }
            set
            {

                    if (false)
                        this._fileName = value.ToUpper();
                    else
                        this._fileName = value;
            }
        }
        public string folderName { get; set; }
        public byte[] fileHashDigest { get; set; }
    }

    public class FILECollection : IEnumerable<FILE>, ICollection<FILE>
    {
        private HashSet<FILE> svgHash;
        private static List<FILE> PreallocationList;
        public string FileName = "N/A";

        /// <summary>
        /// Default Constructor, will not 
        /// preallocate memory.
        /// </summary>
        /// <param name="PreallocationSize"></param>
        public FILECollection()
        {
            this.svgHash = new HashSet<FILE>();
            this.svgHash.Clear();
        }

        /// <summary>
        /// Overload Constructor Preallocates
        /// memory to be used for the new 
        /// FILE Collection.
        /// </summary>
        public FILECollection(int PreallocationSize, string fileName = "N/A", int fileHashDigestSize = 32)
        {
            FileName = fileName;
            PreallocationList = new List<FILE>(PreallocationSize);
            for (int i = 0; i <= PreallocationSize; i++)
            {
                byte[] buffer = new byte[fileHashDigestSize];
                FILE preallocationSVG = new FILE()
                {
                    fileName = "",
                    folderName = "",
                    fileHashDigest = buffer
                };
                PreallocationList.Add(preallocationSVG);
            }
            this.svgHash = new HashSet<FILE>(PreallocationList);
            this.svgHash.Clear(); // Capacity remains unchanged until a call to TrimExcess is made.
        }

        /// <summary>
        /// Add an FILE file to 
        /// the FILE Collection.
        /// </summary>
        /// <param name="svg"></param>
        public void Add(FILE svg)
        {
            this.svgHash.Add(svg);
        }

        /// <summary>
        /// Removes all elements 
        /// from the FILE Collection
        /// </summary>
        public void Clear()
        {
            svgHash.Clear();
        }


        /// <summary>
        /// Determine if the FILE collection
        /// contains the EXACT FILE file, folder, 
        /// and byte[] sequence. This guarantees 
        /// that the collection contains the EXACT
        /// file you are looking for.
        /// </summary>
        /// <param name="item"></param>
        /// <returns></returns>
        public bool Contains(FILE item)
        {
            return svgHash.Any(f => f.fileHashDigest.SequenceEqual(item.fileHashDigest) &&
                                    f.fileName == item.fileName &&
                                    f.folderName == item.folderName);
        }

        /// <summary>
        /// Determine if the FILE collection 
        /// contains the same file and folder name, 
        /// byte[] sequence is not compared. The file and folder
        /// name may be the same but this does not guarantee the 
        /// file contents are exactly the same. Use Contains() instead.
        /// </summary>
        /// <param name="item"></param>
        /// <returns></returns>
        public bool ContainsPartially(FILE item)
        {
            return svgHash.Any(f => f.fileName == item.fileName &&
                                    f.folderName == item.folderName);
        }

        /// <summary>
        /// Returns the total number
        /// of FILE files in the Collection.
        /// </summary>
        public int Count
        { get { return svgHash.Count(); } }

        public bool IsReadOnly
        { get { return true; } }

        public void CopyTo(FILE[] array, int arrayIndex)
        {
            svgHash.CopyTo(array, arrayIndex);
        }

        public bool Remove(FILE item)
        {
            return svgHash.Remove(item);
        }

        public IEnumerator<FILE> GetEnumerator()
        {
            return svgHash.GetEnumerator();
        }

        IEnumerator IEnumerable.GetEnumerator()
        {
            return svgHash.GetEnumerator();
        }
    }
}

I think either I am debugging in a terribly wrong way, or Microsoft should take a look at this. It's like future code is breaking current code...which is impossible!

Answer 1

OK here's my best guess..

First, as I mentioned in the comments, the exception doesn't occur if you comment out the line FILE exists = randomCollection.Where(f => f.fileName == test.fileName && f.fileHashDigest.SequenceEqual(test.fileHashDigest)).First()‌​;

Second, I noticed the same behavior can be reproduced with the following code:

if (true)
{
    object o;
    o = new object();
    Func<bool> m = () => o == null;
}

i.e. the cause seems to be related to the variable being used in a lambda expression. So, looking at the same code snippet above in ILSpy I get the following:

Program.<>c__DisplayClass0_0 <>c__DisplayClass0_ = new Program.<>c__DisplayClass0_0();
<>c__DisplayClass0_.o = new object();
Func<bool> func = new Func<bool>(<>c__DisplayClass0_.<Main>b__0);

so my best guess is that the NullReferenceException refers to <>c__DisplayClass0_ intance being null - and I'm therefore inclined to believe that the stepping through the if(true) actually skipped the first line where <>c__DisplayClass0_ is instantiated