3

I am trying to turn off denormal number support in matlab, so that basically any two computations that would result in a denormal number would instead just result in zero (DAZ, FTZ)

I've researched several sites include the one below, but I haven't found anything about doing this.

http://blogs.mathworks.com/cleve/2014/07/21/floating-point-denormals-insignificant-but-controversial-2/

Veridian
  • 3,531
  • 12
  • 46
  • 80

1 Answers1

0

I've never heard of such an option in Matlab. It would likely require deep manipulation of a lot of the floating-point math, effectively requiring a new datatype to be supported if this were to be an easily toggle-able option in Matlab. You could write your own mex C code to do this (more here and here) for an individual function.

And of course you can get something like this with one line of Matlab – here's an example:

a = [1e-300 1e-310 1e-310];
b = [1e-301 1e-311 1e-310];
x = a-b;
x(abs(x(:)) < realmin(class(x))) = 0;

where realmin is the smallest normalized floating-point number. However, the floating point math is still performed using the extended denormal/subnormal values in a. It's just the output that's clipped to zero.

Unless you're doing this for fun an experimentation, or possibly running code on an embedded platform, I'd really recommend against disabling denormals as a form of optimization. Instead, focus on why your values are so small and how you might rescale your problem to avoid the issue entirely.

Community
  • 1
  • 1
horchler
  • 18,384
  • 4
  • 37
  • 73
  • Thanks, not for fun though. It is experimentation. Basically I would love if matlab gave the user the ability to control all of the aspects of floating point calculations, they are still not there yet though. – Veridian Apr 08 '15 at 00:20
  • @starbox: I agree that it would be nice, and even helpful sometimes for debugging or comparing against C code. I just think that it would be quite prohibitive for them to implement (and test) this. It would also be an option that they would not want users turning on carelessly (or forgetting that it's enabled). You can always try putting in a [service request](http://www.mathworks.com/company/aboutus/contact_us/?s_tid=gn_cntus) with The MathWorks. – horchler Apr 08 '15 at 00:31
  • @starbox: One problem is that disabling denorms would cascade into all the library functions used by Matlab, with results that would effect non-denormal inputs to those library functions in many cases. That might be what you're hoping for, but it's not a generally useful computation model. – Stephen Canon Apr 08 '15 at 17:14
  • @StephenCanon, Agreed in the general case it wouldn't be useful. But breaking functions down into the arithmetic operations would be necessary, something I'm willing to do for my purposes. Oh well. Is there a better tool than MATLAB at controlling floating point operations and equipped with a GUI for displaying images and graphs? (e.g. MAPLE?) – Veridian Apr 08 '15 at 18:21
  • 1
    In my bespoke backpropagation code for training neural nets, I hit the realmin limit when gradients get small (using Adam SGD variant). Therefore my algorithms incur a perfomance cost. Usually this happens in areas that are very performance critical, so manually checking matrices for numbers below realmin is a performance cost also. I wish Matlab would take care of this efficiently behind the scenes and just set those elements to zero. You can't get away from vanishing gradients all of the time in SGD. – rnoodle Mar 22 '18 at 20:54