3

i am trying to reach the single threaded FP peak performance for my nehalem cpu to detect the performance anomalies of my application, but i can't seem to reach it. The clock speed is 3.2 GHz, and i want to achieve the peak FP performance of the cpu without using SSE instructions and multi-threading.

As i understand it single precision FP addition and multiplication can be done in parallel each clock cycle, yielding a maximum performance of 2 * 3.20 = 6.4 GFLOPS/sec.

However i am not able to reach this performance with a simple piece of code:

int iterations = 1000000;
int flops_per_iteration = 2;
int num_flops = iterations * flops_per_iterations;

for(int i=0; i<iterations; i++)
{
    a[i] = i; 
    b[i] = i*2;
    c[i] = i*3;
}

tick(&start_time);

for(int i = 0; i < iterations; i++){
    a[i] *= b[i];
    c[i] += b[i];
}

time = tock(&start_time);

printf("Performance: %0.4f GFLOPS \n", flops/(time*pow(10,-3)*pow(10,9)));

This piece of code gives me a performance of: ~1.5 GFLOPS instead of 6.4 GFLOPS.

Anybody has any other example that can approach the peak performance without using MT and SSE, or has any idea my code doesn't?

Thanks in advance

* Update: Added Assembly Code of the hot loop: *

Address Assembly
Block 17:
0x4013a5    movssl  (%rdi,%rax,4), %xmm2
0x4013aa    movssl  (%r8,%rax,4), %xmm0
0x4013b0    movssl  (%rsi,%rax,4), %xmm1
0x4013b5    mulss %xmm2, %xmm0
0x4013b9    addss %xmm1, %xmm2
0x4013bd    movssl  %xmm0, (%r8,%rax,4)
0x4013c3    movssl  %xmm2, (%rsi,%rax,4)
0x4013c8    inc %rax
0x4013cb    cmp %rcx, %rax
0x4013ce    jb 0x4013a5 <Block 17>
Ricky
  • 1,673
  • 2
  • 19
  • 23
  • have you tried the same thing without arrays? – Karoly Horvath Mar 05 '12 at 12:10
  • What is your compiler and its options? – osgx Mar 05 '12 at 12:16
  • i didn't try it without the arrays actually, i could give it a go. I only compile with -O2 at the moment – Ricky Mar 05 '12 at 12:17
  • Have you tried to time the operations with other functions e.g. `clock()` or `getTimeOfDay` ? – Tudor Mar 05 '12 at 12:31
  • these functions use getTimeOfDay, i verified the time from the timer against the #cycles reported by intel VTune, and the timing is correct – Ricky Mar 05 '12 at 12:33
  • so.. have you tried it without the array, so we know it's not a memory bandwith issue? – Karoly Horvath Mar 05 '12 at 13:19
  • when i alter the code to: `float x = 0.5; #pragma unroll(128) for(int i=0; i 6.4 GFLOPS. Is this possible? – Ricky Mar 05 '12 at 13:36
  • possible duplicate of [Why is my application not able to reach core i7 920 peak FP performance](http://stackoverflow.com/questions/9500264/why-is-my-application-not-able-to-reach-core-i7-920-peak-fp-performance) – Paul R Mar 05 '12 at 13:44
  • @DanLeakin, 6.66 GFLOPS is possible because you use numbers like `1000000` in your calculations. To get proper GFLOPS, divide it by `1024*1024*1024`. – Evgeny Kluev Mar 05 '12 at 13:52
  • @EvgenyKluev: Nonsense. You get "proper" GFlops if you calculate by using 1 G = 10^9 and nothing else. 1 GHz are _exactly_ 10^9 Hz. You confuse this with memory sizes like 2^30 Byte, which are close to 10^9 byte and are often, but technically wrong are called GByte. The correct term is actually GiByte - but this has _nothing_ to do with frequencies. See http://en.wikipedia.org/wiki/Binary_prefix – Gunther Piez Mar 05 '12 at 14:09
  • and the nehalem only has one FADD execution unit right, so it can never get above 3.2 GFLOPS if i'm correct? – Ricky Mar 05 '12 at 14:20
  • Right. My mistake. Sorry. Nehalem has only 1 FADD unit and one execution unit to convert from integer. You should not get more than 3.2 GFLOPS without SSE. – Evgeny Kluev Mar 05 '12 at 14:29

3 Answers3

3

To give a performance of 6.4 GFLOPS, your CPU should perform 10 instructions in one clock. Or 7 instructions if unrolled. This is just not possible. You cannot get more than 4 instructions/clock on this processor.

Evgeny Kluev
  • 24,287
  • 7
  • 55
  • 98
  • thanks, that already explains a lot indeed. Is there any way to approach this peak performance with another piece of code? – Ricky Mar 05 '12 at 13:03
  • Yes, you can get more FLOPS if you do more arithmetic operations than memory reads/writes. For example, `a[i] *= (b[i]*b[i]); c[i] = (a[i]+c[i]+b[i]);` – Evgeny Kluev Mar 05 '12 at 13:08
  • 1
    @DanLeakin Have you tried looking at [this question](http://stackoverflow.com/questions/8389648/how-to-achieve-4-flops-per-cycle)? After some tweaking, the OP managed to get his code to run at 1 add + 1 mul per cycle. – Mysticial Mar 05 '12 at 16:48
2

Howe large is your L3 cache? 4 MB? So you may give a little more headroom for the cache. Try decreasing the working size by 50%.

However, the "parallelism" in the FP operation does basically mean that a FP operation can be triggered while others are still being processed and not finished. But you will hardly manage to get real parallelism without either

  • using a multithreaded approach and/or
  • using SSE registers.
Haymo Kutschbach
  • 3,322
  • 1
  • 17
  • 25
  • L3 cache is 8 MB. I tried reducing the problem size but that doesn't really improve the performance – Ricky Mar 05 '12 at 12:21
  • I don't really want to exploit parallelism in this case, i only want to understand why i am not able to reach the theoretical peak performance for FP operations using a single thread and no sse instructions. Thanks for your reaction though! – Ricky Mar 05 '12 at 12:22
  • Could you post your (optimized) machine instructions for the hot loop? It will not only consist out of * and +. Moreover, the loading of registers and moving of data. They introduce costs as well. – Haymo Kutschbach Mar 05 '12 at 12:30
  • Thanks, I added the assembly of the hot loop to my initial post. But since the iterations are independent the compiler should be able to pipeline this code and perform the multiplication and addition in a pipelined fashion right? – Ricky Mar 05 '12 at 12:42
2

Shoudln't you use loop unrolling to fill up the CPU pipeline ?

Joel Falcou
  • 6,247
  • 1
  • 17
  • 34
  • I tried this using `#pragma unroll(x)` with several values for x, but this only improves the performance a little – Ricky Mar 05 '12 at 12:51