Code is behaving differently in Release vs Debug Mode

Question

We have some unit tests that fail when run in Release mode vs debug mode. If I attach a debugger in release mode the tests pass. There is way too much code to publish here so I am really just looking for best practices in debugging Release mode issues. I have checked for:

DEBUG and RELEASE preprocessor directives but I did not find any.
Conditional Methods

SOLUTION: In this case it is because I was comparing floating point variables for equality. I could not change the floats to decimal without a major refactoring so I added an extension method:

public static class FloatExtension
{
    public static bool AlmostEquals(this float f1, float f2, float precision)
    {
        return (Math.Abs(f1 - f2) <= precision);
    }

    public static bool AlmostEquals(this float f1, float f2)
    {
        return AlmostEquals(f1, f2, .00001f);
    }

    public static bool AlmostEquals(this float? f1, float? f2)
    {
        if (f1.HasValue && f2.HasValue)
        {
            return AlmostEquals(f1.Value, f2.Value);
        }
        else if (f1 == null && f2 == null)
        {
            return true;
        }
        return false;
    }
}

Couple of questions. 1. What sort of failures are you getting to give the question some 'flavour'? 2. Have you checked for conditional methods? — Tim Lloyd, Jan 24 '11 at 20:59
The main problem is the Equals method is returning false. However if I take every statement individually they all return true. If I attempt to attach a debugger the problem goes away. — Shaun Bowe, Jan 24 '11 at 21:02
@stefan I am comparing some float values in the IsEqual method. — Shaun Bowe, Jan 24 '11 at 21:05

Mark Byers · Answer 1 · 2011-01-24T21:26:06.323

8

One thing that might cause the behaviour that you are seeing is an error that causes a race condition. Attaching a debugger can change the timing of the code such that the race condition is no longer triggered.

To fix it, use synchronization appropriately whenever you have multiple threads accessing data.

I am comparing some float values in the IsEqual method.

That sounds like a very bad idea. You should not compare floats for equality because floating point calcualtions are not 100% precise and you can get representation and rounding errors. Compare to see whether they are sufficiently close together. For calculations involving money, you probably want to use the decimal type instead.

edited Jan 24 '11 at 21:26

answered Jan 24 '11 at 21:01

Mark Byers

811,555
193
1,581
1,452

Thanks for your answer, however this particular portion of the application is single threaded. – Shaun Bowe Jan 24 '11 at 21:06
It must be the floating point numbers. I will make some changes and validate. – Shaun Bowe Jan 24 '11 at 21:23

score 4 · Accepted Answer · edited May 23 '17 at 12:09

4

Since it seems to be floating point related there are so many things that can go wrong. See: C# - Inconsistent math operation result on 32-bit and 64-bit and Double precision problems on .NET

There are so many things that can be trashed with floating points. And comparing floats for equality is a general no-no. You chould check the difference smaller than a reasonably epsilon.

edited May 23 '17 at 12:09

Community

1
1

answered Jan 24 '11 at 21:10

stefan

2,886
21
27

See also http://stackoverflow.com/questions/2342396/why-does-this-floating-point-calculation-give-different-results-on-different-mach/2343351#2343351 – Eric Lippert Jan 24 '11 at 23:07

score 3 · Answer 3 · answered Jan 24 '11 at 21:09

Questions you should ask yourself -

Is my code threaded? Timing differences will affect output
Is someone calling Debug.Assert() with an expression that side effects?
What objects implement IDisposable() and do some do so in such a way that changes state?
Are you P/Invoking into unmanaged code?

Number 3 is a very likely bad-boy in this case. Garbage collection may be very different in debug and release and you may find that when an object is garbage collected is affecting the outcome of a later unit test.

And FYI, if you're using NUnit and TestDriven.NET - the two run tests in different orders.

+1 for mentioning differences in garbage collection. In debug mode, objects tend to stay alive longer (e.g. until the end of a method) in order to support debugging, while in release mode, objects arre often collected sooner. — stakx - no longer contributing, Jan 24 '11 at 21:13

score 3 · Answer 4 · answered Jan 24 '11 at 21:42

This is often the case as the debug build is not optimized by default, and even if you enable it, the behavior when debugging is very different. You can disable the "Optimize code" from the project settings for all assemblies on the Properties->Build tab.

There are certainly other changes that can cause differences, like you mention Conditional Methods are one. These I've found to rarely be the cause of issues, for me it's almost always the optimizer.

Classic gotcha's of the optimizer include methods that get 'inlined' so that they fail to appear on a call stack. This causes problems when using System.Diagnostics.StackFrame classes to determine the current execution point. Similarly this will affect the result of MethodBase.GetCurrentMethod or other functions/behavior that rely on the executing method.

Then there are of course many things I've seen the optimizer do that I simply cannot explain at all. One such example was documented and discussed in a post 'HashDerivedBytes - replacing Rfc2898DeriveBytes, but why?' but I've never solved the mystery. I only know that the optimizer just flat broke Rfc2898DeriveBytes when used to generate a series of derived bytes. Oddly enough this only broke when the bytes generated were not evenly divisible by the size of the hash algorithm used (20) and only produced incorrect results after the first 20 bytes.

The fact is that optimizations adversely affecting code is not a new thing for compilers. Most of the old-school C++ developers will tell you that straight away and then, as I did, go into some long drawn out story about how they worked around it ;)

So if a build with Optimization ON is started under a debugger, will some optimization be suppressed, possibly causing the behaviour described in the original question? An example of this would be very helpful. — Govert, May 09 '12 at 14:31
12 years later and it was the StackFrame that was the issue. — ScoMo, Aug 23 '23 at 15:19

score 0 · Answer 5 · answered Jan 24 '11 at 21:05

As Mark suggests, this is usually a result of a timing-related issue, often a race condition or synchronization problem.

One common way to handle this sort of problem is to use "print" statements in the affected areas to show you what's going on. If the print statements (Console.WriteLine, Response.Write, logging, or whatever) make the problem go away, store the values in global variables and print the globals once the problem has shown up.

The most recent time this has happened to me was in code that was reading from a serial port. The debugging activity caused just enough of a change in timing to affect how bytes from the serial port were buffered, which changed how the buffer was getting parsed. Since the print statements changed the timing, I had to store the data up to output later.

score 0 · Answer 6 · answered Oct 25 '11 at 11:14

Just to add my two cents to this, I recently found that I had a date comparison in an sql procedure that the testing called. The dates were all auto-generated prior in the test procedure and values were inserted into the DB, and so occasionally they were exactly the same (when using RunTests) causing a null to be returned on a table join. Not what I was expecting. Obviously, in debug mode, since I'm slowly progressing through it, there will be a difference in the auto-generated times which meant that I never bumped into the error. I resolved this by inserting

Threading.Thread.Sleep(520)

wherever there would definitely be a delay between actions. Problem fixed.

Code is behaving differently in Release vs Debug Mode

6 Answers6

Linked