Ollama
Open Source LLM for AI
For full documentation visit ollama.com.
What is Ollama and why I use it as my main LLM for AI
Ollama is an open-source platform that simplifies running large language models (LLMs) and multimodal models (like Llama, Qwen, and DeepSeek) locally on any own machine. It handles the heavy technical lifting and allows for the use of AI without cloud subscription fees or data privacy concerns.
It is widely supported and compatible with most anything available allowing for one LLM to be used across many applications reducing overhead. I run multiple Ollama LLMs on my servers and workstations allowing for independent AI instances.
Why Use Ollama
-
Complete Privacy: Because the AI runs entirely on your local hardware, your data never leaves your machine.
-
No API or Token Fees: Once set up, you can run prompts as often as you want without paying per-word charges like you do with cloud-based AI providers.
-
Offline Capability: After downloading the models, you can run your AI assistant in air-gapped environments or while traveling without an internet connection.
-
Easy Integration: Ollama serves a local REST API (usually at http://localhost:11434), making it easy to plug local AI into development environments, coding assistants, or Python scripts.
How It Works
Instead of manually downloading model weights and configuring complex dependencies, Ollama packages the setup and model weights into a single installable app. You simply use terminal commands like ollama run llama3.2 to download and interact with a model instantly.
Highly Compatibility
It can be downloaded directly for macOS, Windows, or Linux from the Ollama Download Page. Once installed, it operates in the background, making it ready to use with just a few CLI commands.
How I Generally Use Ollama
It can easily be installed and ran as a CLI, Docker container or both. I typically have Ollama CLI installed with a Docker container to run it using Docker Compose vs Docker Desktop as Docker Compose supports GPU passthrough where as Docker Desktop does not. It also provides the LLM support for most everything I typically use.
Great For Testing New Models
Generally, I use it to download and test out the latest models available. Being able to download and install the latest models quickly and easily, makes it ideal. This allows me to find models useful for specific things such as coding, embedding, etc. Currently, I use many models for different purposes based on their respective capabilities such as Gemma4, Ministral, Mistral, and others.
AI Technologies
AI Models
More AI Tech
---
title: Ollama
text: Open Source LLM for AI
image: /assets/svg/ollama3.svg
link:
target:
---