Why does gcc take a long time to compile a C code if it has a big array in the extern block?
#define MAXNITEMS 100000000
int buff[MAXNITEMS];
int main (int argc, char *argv[])
{
return 0;
}
Asked
Active
Viewed 121 times
0
Mustafa
- 109
- 1
- 7
-
What is long? Long compared to what? Do you compile in debug or release? – Juraj Blaho Oct 03 '13 at 07:59
-
just try to compile it with MAXNITEMS=1000 then try to compile it as it is and compare them. (I meant "Long" compared to the compilation time when using small arrays) – Mustafa Oct 03 '13 at 08:00
-
You do realize that's a 400MiB array on systems where `int` takes four bytes, right? – Daniel Kamil Kozar Oct 03 '13 at 08:03
-
@Daniel. Yes, but array reservation in memory happen at runtime not at compile time, is not it? In comilation we are just generating the correspondent machine code. – Mustafa Oct 03 '13 at 08:04
-
@Mustafa seems correct, just compiled his code, and the executable size is ~6.5kb – Nikhil Oct 03 '13 at 08:06
-
1Wrong. When declared in a global scope, arrays are initialized to zero and placed in the data section of the resulting executable (talking about ELF executables in this case, though I'd expect it to be similar with COFF). Therefore, no initialization or "reservation" at runtime is needed - they're ready to use. Interestingly, running `gcc -v` reveals that `gcc` itself does not take a long time to run : it's `ld`. – Daniel Kamil Kozar Oct 03 '13 at 08:07
-
@Daniel I have expected something like that to be the reason. But the generated ELF file is only of 4.5kb size? Why? – Mustafa Oct 03 '13 at 08:14
-
There is no compile time difference on my system. What version of what compiler do you use on what operating system on what kind of hardware and disk? Me: Ubuntu 13.04 running in a VM on core i5 2.5 GHz with 2 GB RAM and SSD, using gcc 4.7.3. – jxh Oct 03 '13 at 08:36
-
1@DanielKamilKozar: That's not correct. If the array isn't assigned any values, the compiler just puts the necessary size into a special .bss segment and remembers the offset for it in this segment. The code above does *not* create a 400MB executable. See http://stackoverflow.com/questions/93039/where-are-static-variables-stored-in-c-c – Aaron Digulla Oct 03 '13 at 08:41
-
@jxh I am using gcc version 4.4.5 on (Debian 4.4.5-8) Target: i486-linux-gnu – Mustafa Oct 03 '13 at 08:42
-
@Mustafa: Someone that can actually observe the same problem that you do needs to work with you. Good luck. – jxh Oct 03 '13 at 08:44
-
@Mustafa: how often did you time the compile? – Aaron Digulla Oct 03 '13 at 08:44
-
@Aaron. I have not understand exactly what you mean? If you mean how many time I tried it. Then I have tried it 100 of times. If you mean how long does it take to compile, then it takes 4 sec – Mustafa Oct 03 '13 at 08:51
-
@Mustafa: Some people think it's enough to run code once to find out how long it takes to execute :-) – Aaron Digulla Oct 03 '13 at 08:57
-
@DanielKamilKozar, Aaron is correct. The `.bss` section is made specifically so that the executable wouldn't contain a huge amount of 0s when the operating system can allocate that easily at run-time. The .bss is still initialized to zero, so the behavior is the same. – Shahbaz Oct 03 '13 at 08:59
-
@Mustafa, can you give us the exact command you use to compile the code? – Shahbaz Oct 03 '13 at 09:01
-
@Shahbaz, it is "#gcc test.c" and the latency is only in the linking stage because if I say: "#gcc -c test.c" it takes no time to generate the object code – Mustafa Oct 03 '13 at 10:00
-
If I initialize the array it takes approximatly 10 sec to compile. The generated a.out is of 400 MB size. That makes sense. But if it is not initialized, I do not know what is the reason of this latency – Mustafa Oct 03 '13 at 10:10
-
`.bss` is the memory segment that is *NOT* guaranteed to be initialized, as opposed to the `.data` segment, which holds initialized data. The fact that Linux initializes the whole `.bss` to zero does not mean that this is how it should operate. GCC cleverly uses this, though it's possible to disable this behaviour with `-fno-zero-initialized-in-bss` for OSes that don't initialize their `.bss` with zeros. – Daniel Kamil Kozar Oct 03 '13 at 13:20
1 Answers
0
I suspect a bug somewhere. There is no reason for the compile to take longer, no matter how big the array is since the compiler will just write an integer into the .bss segment since you never assign a value to an element in it. Proof:
.file "big.c"
.comm buff,4000000000000000000,32
.text
.globl main
.type main, @function
main:
.LFB0:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset 6, -16
movq %rsp, %rbp
.cfi_def_cfa_register 6
movl %edi, -4(%rbp)
movq %rsi, -16(%rbp)
movl $0, %eax
popq %rbp
.cfi_def_cfa 7, 8
ret
.cfi_endproc
.LFE0:
.size main, .-main
.ident "GCC: (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3"
.section .note.GNU-stack,"",@progbits
As you can see, the only thing left of the array in the assembly is .comm buff,4000000000000000000,32.
I suggest you gcc with -S to see the assembler code. Maybe your version of GCC has bug. I tested with GCC 4.7.3 and the compile times here are the same, no matter which value I use.
Community
- 1
- 1
Aaron Digulla
- 321,842
- 108
- 597
- 820
-
Yes, you r right. I am not able to find a reasonable reason for this latency apart from a specific bug. Having in mind that I have done this on a Virtual Machine, does it make any change? – Mustafa Oct 03 '13 at 10:01
-
Possibly. Some other VM on the same host might have been very busy, for example clogging the disk I/O. – Aaron Digulla Oct 03 '13 at 10:03
-
-
RAM is just one of many factors. What does the host OS do while you compile? Maybe your virus scanner is busy with something. If you really want to find out, then you need to run gcc with a profiler while monitoring your VM and host OS. – Aaron Digulla Oct 03 '13 at 10:11
-
True. But my Hosting OS is idle, this experiment I have been trying for long time. Currently my CPU usage is less than 20% – Mustafa Oct 03 '13 at 10:13
-
If I initialize the array it takes approximatly 10 sec to compile. The generated a.out is of 400 MB size. That makes sense. But if it is not initialized, I do not know what is the reason of this latency – Mustafa Oct 03 '13 at 10:14
-
As I said: It's probably impossible say by just thinking about it. Break out your tools and measure the performance to see where the time goes. – Aaron Digulla Oct 03 '13 at 10:15
-
But my question is: what is it doing more if the matrix is huge? why for small matrix it takes no time? If it is relevant to some background activity going on my PC, then why compiling the small matrix takes no time? – Mustafa Oct 03 '13 at 10:28
-
@Mustafa, if it's due to a bug, then who knows! It could be the compiler is by mistake allocating memory for that array, initializing it with zero only to later decide to skip it and put it in `.bss`. – Shahbaz Oct 03 '13 at 11:35
-