How to count Multiply-Adds operations?

Question

Given a defined Convolutional Neural Network, is there a function or method on how to compute the number of Multiply-Adds operations?

The term was introduced MobileNetV2 paper : https://arxiv.org/pdf/1801.04381.pdf.

We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as actual latency, and the number of parameters.

BiBi · Answer 1 · 2018-06-28T12:36:19.393

7

Counting the Multiply-Add operations is equivalent to calculating the FLOPs of a model. This can be achieved using the profiler from tensorflow.

flops = tf.profiler.profile(graph,\
     options=tf.profiler.ProfileOptionBuilder.float_operation())
print('FLOP = ', flops.total_float_ops)

Be sure to look at the caveats explained in this answer. Basically:

The number of FLOPs calculated might include initialisation operations like Multiply-Add from initialising your weights with a Gaussian distribution for instance, you need to freeze the graph to rule those possibly irrelevant Multiply-Add operations,
The Multiply-Add calculations from TensorFlow are approximate. An issue has been opened on this matter here.

edited Jun 28 '18 at 12:36

answered Jun 28 '18 at 12:29

BiBi

7,418
5
43
69

1

Any advice on how to get it to work on TF 2.0 models? – FlashDD Nov 05 '19 at 08:37
For TF 2.0 & Keras model, see: https://stackoverflow.com/questions/49525776/how-to-calculate-a-mobilenet-flops-in-keras – driedler Jan 22 '20 at 14:25

score 0 · Answer 2 · answered Jun 28 '18 at 10:08

Im not sure about a function, but this website should help you keep track of MulAdd operations, The only caveat is you might need the prototxt file, but for popular architectures MulAdd operations are already listed

-https://dgschwend.github.io/netscope/quickstart.html

Hope it helps.

How to count Multiply-Adds operations?

2 Answers2