Carat: Unlocking Value-Level Parallelism for Multiplier-Free GEMMs

In recent years, hardware architectures optimized for general matrix multiplication (GEMM) have been well studied to deliver better performance and e.ciency for deep neural networks. With trends towards batched, low-precision data, e.g., FP8 format in this work, we observe that there is growing untapped potential for value reuse. We propose a novel computing paradigm, value-level parallelism, whereby unique products are computed only once, and di.erent inputs subscribe to (select) their products via temporal coding. Our architecture, Carat, employs value-level parallelism and transforms multiplication into accumulation, performing GEMMs with e.cient multiplier-free hardware. Experiments show that, on average, Carat improves iso-area throughput and energy e.ciency by 1.02. and 1.06. over a systolic array and 3.2. and 4.3. when scaled up to multiple nodes.