Comment by jhj

Not really, it's just adding some data transposition (coalescing individual bytes from the data words together) and an option to use a LZ/dictionary-type compressor to compress redundant things. But an LZ-type compressor doesn't make much sense on NN weights I think since it is not as redundant as most text data with many repeats, and also the space of possible dictionary matches is pretty small since unless the data is highly sparse, there may not be many repetitions that you can leverage to avoid the dictionary overhead.

If you add an LZ-type compressor and have this be in the critical path for inference, then decompression will be a lot slower. It would be best to fuse decompression with the compute kernels (e.g., a GEMM that performs decompression on each tile before the arithmetic), and the simpler the decompression routine, the easier this will be.