Using Ollama, Llama3, Continue, and Open WebUI to bring a safe, local, open source, and free virtual assistant experience
—
I tell people all the time that I use local LLMs to optimize my daily workflow, including a locally hosted ChatGPT-like UI along with free coding autocomplete integration (similar to Copilot) into VSCode, and the most common answer that I get is “Wow, that sounds like a pain to set up.” And I typically rebuke that it only really took me a few minutes!
From there, to help this person, I send them a collection of links, instructions, and tutorials, all just to cover something that can be accomplished so quickly. This will take you through the entire process from end to end in the time it takes for you to finish your coffee.
This tutorial will be divided into two sections, one for the web UI and one for code assist (since not everyone will be interested in that portion). If you are interested in the VSCode integration please complete the Web UI section first, and then continue!
Why Local?
If you have business secrets and proprietary code or are using private keys and PII you do not want that information to become…