How to use LLMs like Deepseek locally

Cover Image for How to use LLMs like Deepseek locally
Arfath Ahmed Syed
Arfath Ahmed Syed

Large language models (LLMs) such as DeepSeek are transforming the way we interact with AI—enabling everything from natural language conversations to code generation and complex reasoning. While many services host these models in the cloud, running an LLM locally gives you full control, enhanced privacy, and often faster response times (depending on your hardware). This guide will walk you through setting up and running DeepSeek (or similar LLMs) on your own machine. 

Introduction

Cloud-based AI services have their benefits, but they also come with drawbacks: data privacy concerns, recurring API costs, and dependency on external servers. Running LLMs like DeepSeek locally means all processing happens on your machine. You get the freedom to tweak configurations, optimize performance for your hardware, and ensure that your data remains private.

In this guide, we’ll cover the essential prerequisites, installation steps, and best practices to get a DeepSeek-like model up and running on your local system.

Prerequisites

Before you start, ensure your machine meets these basic requirements:

Software:

  • Operating System: Linux (Ubuntu 20.04 or later), macOS, or Windows Subsystem for Linux (WSL2)
  • Python Version: 3.8 or later

Hardware:

  • For Smaller Models (e.g., 1.5B – 7B parameters): A modern CPU with at least 8–16 GB RAM can suffice; a GPU (e.g., NVIDIA RTX 3060 with 8–12 GB VRAM) will boost performance.
  • For Larger Models (e.g., 14B and above): More RAM (32 GB or more) and a GPU with higher VRAM (RTX 3080/4090 or equivalent) are recommended.
  • Basic Tools: Git, pip, and a terminal emulator
  • Optional: Docker (if you prefer a containerized deployment) and a Web UI tool like Open WebUI or Chatbox for a graphical interface.

Step 1: Installing a Local Runtime with Ollama

One of the simplest ways to run DeepSeek-like LLMs locally is by using Ollama—a tool that manages the download and execution of models on your machine.

1. Install Ollama:

On macOS or Linux, open your terminal and run:

curl -fsSL https://ollama.com/install.sh | sh

Then verify the installation:

ollama --version


2. Start the Ollama Service:

Ensure the Ollama service is running. On a Mac, you can start a you might use:

ollama serve


Conclusion

Running DeepSeek-like LLMs locally empowers you with full control over your AI workflows, enhances data privacy, and eliminates dependency on external APIs. Whether you’re experimenting with an 8B model for casual chat or pushing a larger model for advanced coding and reasoning tasks, local deployment lets you tailor the system to your exact needs.

By following this guide, you can set up a local runtime using Ollama, download and run a model, and even integrate a user-friendly GUI via Docker-based Open WebUI or Chatbox. As hardware continues to improve and LLMs become more accessible, running your own AI models locally is becoming an increasingly viable option for developers, researchers, and enthusiasts alike.