Abrar Anwar, Josh Welsh, and Yan Chang

Listen to this article ReMEmbR combines LLMs, VLMs, and retrieval-augmented generation to enable robots to reason and take action. | Source: NVIDIA Vision-language models, or VLMs, combine the powerful language understanding of foundational large language models with the vision capabilities of vision transformers (ViTs) by projecting text and images into the same embedding space. They can take unstructured multimodal data, reason over it, and return the output in a structured format. Building on a broad