Arun Nanda

Running large AI models on edge devices Arun Nanda · Follow Published in Towards Data Science · 9 min read · Sep 7, 2024 — Image created using Pixlr AI models, particularly Large Language Models (LLMs), need large amounts of GPU memory. For example, in the case of the LLaMA 3.1 model, released in July 2024, the memory requirements are: The 8 billion parameter model needs 16 GB memory in 16-bit floating point weights The