Ollama is an Open Source project by Meta that provides a platform for creating, running, and managing multiple different Large Language Models or LLM locally on a machine. Ollama acts as a bridge for LLM Model communication.
Ollama can be compared to Docker, Docker also allows to download of multiple images from the Docker Repository and runs in containers.
The latest versions of Ollama can be downloaded from the official website.
Jeffrey Morgan and Michael Chiang are the founders of Ollama
Ollama is available for Macintosh, Linux, and Windows operating systems. its command line Interface allows easy download, run, and manage Large Language Models locally.
Ollama meaning is to play with a rubber ball using the buttocks
Ollama can run on CPU
Ollama can run on Intel CPU also, but it should be avoided. Ollama is designed to give better performance but requires Nvidia or AMD GPUs, for development purposes, Ollama can run on CPU only, but it will not give better performance, even with 16 core CPU.
Enhanced Privacy
Ollama once downloaded the Lange Language Model from the internet, doesn't require a network connection. so all the communication and sensitive information never leaves your secure environment, on the other hand, Large Language Models provided by Other vendors collect data to train Models.
Zero Operational Cost
Large Language Models like Open AI's Chat GPT, or Google AI support millions of users simultaneously, which requires high processing power to run, and in return, they charge some amount based on consumption to support infrastructure, whereas Ollama runs on the local machine, handling a single user does not require that much high computation, hence the operation cost is none.
Ollama Model Library
Currently, Ollama supports the below Large Language Models.
| Model | Parameters | Size | Download |
|---|---|---|---|
| Llama 3.2 | 3B | 2.0GB | ollama run llama3.2 |
| Llama 3.2 | 1B | 1.3GB | ollama run llama3.2:1b |
| Llama 3.1 | 8B | 4.7GB | ollama run llama3.1 |
| Llama 3.1 | 70B | 40GB | ollama run llama3.1:70b |
| Llama 3.1 | 405B | 231GB | ollama run llama3.1:405b |
| Phi 3 Mini | 3.8B | 2.3GB | ollama run phi3 |
| Phi 3 Medium | 14B | 7.9GB | ollama run phi3:medium |
| Gemma 2 | 2B | 1.6GB | ollama run gemma2:2b |
| Gemma 2 | 9B | 5.5GB | ollama run gemma2 |
| Gemma 2 | 27B | 16GB | ollama run gemma2:27b |
| Mistral | 7B | 4.1GB | ollama run mistral |
| Moondream 2 | 1.4B | 829MB | ollama run moondream |
| Neural Chat | 7B | 4.1GB | ollama run neural-chat |
| Starling | 7B | 4.1GB | ollama run starling-lm |
| Code Llama | 7B | 3.8GB | ollama run codellama |
| Llama 2 Uncensored | 7B | 3.8GB | ollama run llama2-uncensored |
| LLaVA | 7B | 4.5GB | ollama run llava |
| Solar | 10.7B | 6.1GB | ollama run solar |
