My CPU is undervolted and even then I made sure to get a cooler. This is because the less heat your CPU runs at the lesser power it uses.
Not me. Using Intel G2020, no point buying a separate CPU cooler.The same applies for i3 CPU.
Intel CPUs are no longer like nuclear reactors back in 2003/4 (back then, stock CPU cooler was not enough to prevent thermal throttling at stock speeds.)
I consider that warm.
Which CPU are you using? Doing any overclocking?

If your CPU runs at 50W at 100% load, it dissipates 50W of heat which is about 26 degree celcius / hour. ( According to This calculator)
Suppose you manage to chill your CPU to 0 degree celcius, it will still be using 50W of power at 100% load, which will translate to the same amount of heat being output.
How does your CPU suddenly become more efficient when you cool it a few more degrees? Assuming it is not running at throttling temps eg. 100deg celcius)
Stock cooler sucks. I switched from stock to CM Hyper 212+ and I see like 15C difference load on i7-2600 @ 1.22v. Totally worth it for a $49 purchase. You only need AIO liquid coolers if doing high overclocks.

But even if the CPU cooling fan is quieten by using aftermarket cooler, the noise will come from elsewhere like the power supply. Unfortunately I already bought the FSP Hexa 500 which is quite noisy. My HIS brand Radeon HD 6850 graphics card is also noisy with the blue fan spinning at 3400 RPM.
But then the Arctic Freezer 11 LP should lower down the CPU temperature and in term lower down the chassis interior temperature ...... maybe I should still buy it.
Erm, lower CPU temp does not equal lower chassis temp. If the heat from the chassis not exhausted properly the interior temp will be higher even though the CPU temp will be lower. Graphics card fan you can control with Catalyst Control Centre so that shouldn't be a problem. I find mine at 50% the noise is about equal to the intel stock cooler so I just leave it there.