I'm trying to wrap my head around the differences between different types of processors when it comes to AI training/inference.
As far as I understand: Large AI/NN systems basically need a HUGE amount of matrix multiplications to happen (multiplications/accumulations).
- CPU has a huge drawback because of the van Neumann Bottleneck (1 ALU accessing memory at a time).
- GPU partially solves this by throwing a bunch of extra CUDA/tensor cores into the equation, but the Van Neumann bottleneck is still fundamentally present
- Google's TPU solves this bottleneck by connecting ALUs in a matrix. As far as I can tell all other NPU's function with this same logic.
It is clear to me how a TPU can achieve a lot more operations per clock cycle due to the matrix of processors.
-Now 2 startups: Mythic and Synthic have brought new Analog Matrix processors to the market that are supposed to be even faster/ more efficient.
Now, I've heard that analog systems can be more energy efficient due to the fact that physical signals are inherently infinitessimaly precise, but wouldn't this effect be corrupted by noise? And wouldn't it only be more efficient for high accuracy calculations?
My question is thus: How do the analog matrix processors (by e.g. Mythic) improve upon classic TPU's?
