The CPU temperature is solely determined by the CPU usage in a unit
time. [...] So as the CPU usage percentage or the CPU
frequency decreases, the CPU usage in a unit time decreases, and the
CPU temperature decreases.
Power draw (i.e. heat generated) is not solely determined by CPU utilization, although this but also depends on what instructions the CPU is performing. In a digital, synchronous CMOS circuit (such as your processor), the power consumption can be computed as:
P = C x V^2 x f
Where C is the capacitance of the digital circuit (changes based on what instructions are being executed), V is the voltage of the CPU, and f is the clock frequency. Some instructions draw more power than others, so we will assume it's fixed here (i.e. running the same programs that do some appreciable work other than idling). As a side effect of this, CPU temperature will decrease at idle (just NOPs) even at the same clock frequency.
Note, however, that the power consumed by the CPU is still directly related to the frequency and voltage. Halving the frequency will drop the power consumption to 50%, while halving the voltage will drop power consumption to 25% of it's original value. This has a huge effect on heat generation, even if we want to perform the same amount of work (recall that power is work per unit time; see below).
Higher usage means higher temperature. Lower usage means lower temperature.
Yes, this is true. When your computer is idle, it's often doing "nothing" (i.e. NOP instructions, in a low power state, or simply not doing power intensive instructions). When it's doing something, like rendering graphics, it uses a lot more components in the CPU (like the ALU, FPU, MIU), generating more heat.
Underclocking your CPU frequency with cpufreq will only affect the percentage of CPU usage without lowering the temperature.
No, this is false. See the equation above. Underclocking will cause programs to execute in a longer timespan, but the power consumed by the circuit will decrease. CMOS power consumption is directly related to the number of logic switches per unit time.
This is very intuitive given the definition of power, which is simply work per unit time, or the rate at which we perform work/computations. If we run the same program through to completion at a given frequency f, and then compare running it at frequency f/2, in the latter case, although we've taken twice as long to execute the program, we've done the same amount of work - and thus, the power consumed by the CPU during this time will be half.
The CPU will therefore operate at a lower temperature, even though it takes longer to perform the same amount of work, as it now has more time to dissipate the heat in the CPU. Underclocking also allows operating the CPU at a lower voltage (undervolting), further decreasing the power consumption with no effect on work.
