Extreme Compression of Per-Layer Embeddings via Cross-Layer Latent Vector Quantization
Scaling embedding capacity alongside Transformer computation is an emerging trend in language models. This trend makes high compression rates increasingly useful. Hence, we study extreme compression of Per-Layer Embeddings (PLE), which fol…
