Maarten Grootendorst

Demystifying the compression of large language models Maarten Grootendorst · Follow Published in Towards Data Science · 20 min read · 6 hours ago — As their name suggests, Large Language Models (LLMs) are often too large to run on consumer hardware. These models may exceed billions of parameters and generally need GPUs with large amounts of VRAM to speed up inference. As such, more and more research has been focused on making these models