Memory systems for AI
LLM inference is increasingly limited by the memory capacity of the accelerator rather than by its compute. At Huawei, I work on adding that capacity with high bandwidth flash.
-
FLINTHPCA 2027
Stores LLM weights in HBF, next to HBM. A hardware burst-buffer controller coalesces and pipelines HBF reads, using the existing NAND flash buffers, to keep bandwidth high. A phantom-plane refresh mechanism moves refresh off the critical inference path, and a read-only flash translation layer maps the weights to HBF locations with a compact table.





