CPUs are relatively small, and engineers are constantly trying to make them smaller and get more transistors in the same surface.
Why aren't CPUs bigger? If an approximately 260mm2 die can hold 758 million transistors (AMD Phenom II x4 955). Then a 520mm2 should be able to hold double the amount of transistors and technically double the clock speed or cores. Why isn't this done?
There are a lot of technical concerns (path lengths get too long and you lose efficiency, electrical interference causes noise), but the primary reason is simply that that many transistors would be too hot to adequately cool. That's the whole reason they're so keen to reduce the die size - it allows for performance increases at the same thermal levels.
Generally you're right: In the short term, increasing parallelization is not only viable but the only way to go. In fact, multi-cores, as well as caches, pipelining and hyper-threading are exactly what you propose: speed gain through increased chip area use. Of course, shrinking geometries does not collide with increasing die area use. However, die yield is a big limiting factor.
Die yield grows in inverse proportion to die size: large dies are simply more likely to "catch" wafer errors. If a wafer error hits a die, you can throw it away. Die yield obviously affects die cost. So there's an optimal die size in terms of costs vs. profits per die.
The only way to produce significantly larger dies is to integrate fault tolerant and redundant structures. This is what Intel tries to do in their Terra-Scale project (UPDATE: and what is already practiced in every-day products as Dan points out).
Several of the answers given here are good answers. There are technical issues in increasing the size of the CPU and it will lead to a lot more heat to deal with. However all of them are surmountable given strong enough incentives.
I would like to add what I believe is a central issue: economics. CPUs are made in wafers like this, with a large number of CPUs per wafer. The real manufacturing cost is per wafer, so if you double the area of a CPU you can only fit half as many on a wafer, so the per-CPU price doubles. Also, not all of the wafer always comes out perfect, there can be errors. So doubling the area doubles chance of a defect in any specific CPU.
Therefore from the economic standpoint the reason they are always making things smaller is to get better performance/mm^2, which is the determining factor in price/performance.
TL;DR: In addition to the other reasons mentioned doubling the area of a CPU more than doubles the cost.
Adding more transistors to a processor doesn't automatically make it faster.
Increased path length == slower clock rate.
Adding more transistors will increase the path length. Any increase has to be used valuable or it'll cause an increase in cost, heat, energy, but a decrease in performance.
You can of course always add more cores. Why don't they do this? Well, they do.
Your general assumption is wrong. A CPU with a double sized die does not mean it can operate with double speed. This would only add more space for adding more cores (see some Intel manycore chips with 32 or 64 cores) or larger caches. But most of the current software can not make use of more than 2 cores.
Therefore the increased die size increases the price massively without a gain of the same height. This one of the (simplified) reasons CPUs are as they are.
In Electronics SMALLER = FASTER 3GHz needs to be much smaller than 20MHz The larger the interconnections, the greater the ESR and the slower the speed.
Doubling the amount of transistors doesn't double the clock speed.
The cost of producing the raw wafers is a factor. Monocrystalline silicon is not free, and the refining process is somewhat expensive. So using more of your raw material increases cost.
Big living things, artificial or not, like dinosaurs, are loosers. The ratio area / volume is not fair for their survival : too many constraints about energy - every form - in and out.
Think of a CPU as a network of connected nodes (transistors). In order to provide more capabilities the number of nodes and the paths between them increase to a degree, but that increase is linear. So one generation of a CPU might have a million nodes, the next might have 1.5 million. With miniaturization of the circuit, the number of nodes and paths are condensed into a smaller footprint. The current fabrication processes are down to 30 nanometers.
Let's say that you need five units per node and five units distance between two nodes. End to end, in a straight line you can create a bus of 22222 nodes in 1 CM of space. You can make a matrix of 493 million nodes in a square CM. The design of the circuit is what contains the CPU's logic. Doubling the space is not what increases the speed, it just would enable the circuit to have more logical operators. Or in the case of multi-core CPUs to allow the circuit to handle more work in parallel. Increasing the footprint would actually decrease the clock speed because the electrons would have to travel longer distances through the circuit.
