The computational landscape of Artificial Intelligence (AI) is undergoing a significant bifurcation. While hyperscalers continue to train trillion-parameter models in massive data centers, a concurrent revolution is occurring at the edge. The capability to run high-fidelity Large Language Models (LLMs) on consumer-grade hardware has transitioned from a theoretical possibility to a practical reality, driven by advances in quantization techniques and efficient inference runtimes. This report details the architectural design and implementation of a robust, self-hosted AI platform that mirrors the functionality of commercial APIs while retaining absolute data sovereignty.

The objective of this infrastructure is to provide a unified, internet-accessible interface for local LLMs. This system must serve two distinct consumer types: human users interacting via a rich chat interface (Open WebUI), and programmatic clients consuming an OpenAI-compatible API (LM Studio). To achieve this while mitigating the severe security risks associated with exposing local hardware to the public internet, we employ a zero-trust networking architecture.

This report serves as a definitive implementation guide for DevOps engineers and system architects. It integrates LM Studio as the high-performance inference engine, Docker for containerized application management, Tailscale for secure, NAT-traversing networking, and Caddy as an intelligent reverse proxy and security gateway. By the conclusion of this document, the reader will possess a production-ready blueprint for a "Personal AI Cloud" that is secure, scalable, and accessible from anywhere in the world.

The proposed stack is composed of four distinct layers, each selected for its specific strengths in the modern DevOps ecosystem:

The interaction model is designed to minimize latency while maximizing security. External requests originating from the public internet are encrypted and routed via Tailscale's relay servers directly to the local host. Caddy intercepts these requests, terminating the TLS connection. Based on the request path, Caddy routes traffic either to the Docker container hosting Open WebUI or directly to the LM Studio API port, enforcing Basic Authentication for the latter to prevent unauthorized API consumption.

Before deploying the software stack, one must validate the underlying hardware. Self-hosting LLMs is memory-bound rather than compute-bound for single-user scenarios.

VRAM Considerations:

The primary constraint is Video Random Access Memory (VRAM). The model weights must fit entirely within VRAM to achieve acceptable token generation speeds (tokens per second, or tok/s).

System Memory (RAM): While GPU offloading is preferred, LM Studio can offload layers to the system CPU and RAM if VRAM is exhausted. However, this incurs a significant performance penalty, often dropping speeds from 50+ tok/s to <5 tok/s. A minimum of 32 GB system RAM is recommended to support the operating system, Docker containers, and partial model offloading.⁹

2. Infrastructure Foundation: Operating System and Networking

This section establishes the bedrock of our deployment. While LM Studio and Docker are cross-platform, Linux (specifically Ubuntu LTS) is the recommended host operating system for this architecture due to its superior handling of container networking and background service management.¹⁰

NVIDIA Driver Installation:

For LM Studio to utilize the GPU, the proprietary NVIDIA drivers must be installed. On a fresh Ubuntu installation:

Bash

sudo apt update && sudo apt upgrade -y
sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers autoinstall

Verify the installation with nvidia-smi. The output must show the Driver Version and CUDA Version. Crucially, the CUDA version displayed here is the maximum supported version; LM Studio includes its own bundled CUDA runtime libraries, but they rely on the host driver being present and compatible.

Open WebUI is distributed as a Docker image. Using Docker ensures that the complex Python dependencies and frontend frameworks required by the UI do not interfere with the host system.

Installation Strategy:

Do not use the docker.io package from standard repositories, as it is often outdated. Install from the official Docker repository to ensure support for the latest Compose features.

Bash

# Set up Docker's apt repository.
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install Docker packages.
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

User Permissions:

To avoid running Docker commands as root (which can mess up file permissions for persistent volumes), add the current user to the docker group:

Bash

sudo usermod -aG docker $USER
newgrp docker

Tailscale creates a secure mesh network (Tailnet) that overlays the public internet. It allows devices to communicate as if they were on the same physical LAN, regardless of their actual location.

Why Tailscale? Traditional remote access requires opening ports (Port Forwarding) on the home router. This exposes the device to port scanners and botnets. Tailscale uses NAT traversal techniques (STUN/TURN) to punch through firewalls securely, establishing encrypted WireGuard tunnels.¹

Installation:

Bash

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

Upon running tailscale up, a URL is generated. Visit this URL to authenticate the machine and add it to your Tailnet.

DNS Configuration (MagicDNS): To use SSL certificates effectively, "MagicDNS" must be enabled in the Tailscale Admin Console. This assigns a stable hostname (e.g., gpu-server.tailnet-name.ts.net) to the machine, which is resolvable from anywhere inside the Tailnet.¹¹

3. The Inference Runtime: Deep Dive into LM Studio

LM Studio has evolved from a simple GUI application into a robust server platform capable of headless operation via its Command Line Interface (CLI), lms. This section details the configuration of the inference engine.⁵

While LM Studio offers an AppImage for Linux, the CLI workflow is preferred for server deployments. The CLI tool lms allows for scripting, automation, and service management.

Bootstrapping the CLI:

If you have the GUI installed, the CLI might already be available or can be bootstrapped. For a purely headless server, use the installer script:

Bash

curl -fsSL https://lmstudio.ai/install.sh | bash

After installation, verify the binary is in your path:

Bash

lms --help

The output should list subcommands like server, ls, load, and get.¹³

The effectiveness of the entire stack depends on the model loaded. LM Studio natively supports the GGUF format, which is optimized for running on CPUs and consumer GPUs via Apple Metal or CUDA.

Searching and Downloading:

Use the lms get command to query the Hugging Face hub.

Bash

lms get llama-3-8b-instruct

The interface will present a list of available quantization levels.

Listing Local Models:

Once downloaded, models reside in the internal cache. To see available models and—crucially—their "Model Key" or path:

Bash

lms ls

This command returns the identifier required to load the model programmatically.¹³

The lms server command spins up a local HTTP server that mimics the OpenAI API specification. This compatibility is vital, as it allows tools designed for OpenAI (like Open WebUI) to plug into LM Studio seamlessly.

The Binding Address (--host): By default, lms server start binds to 127.0.0.1 (localhost).⁵

CORS (Cross-Origin Resource Sharing): Web-based interfaces (like Open WebUI running in a browser) often strictly enforce CORS. Even though Open WebUI runs server-side, enabling CORS on LM Studio is a best practice to prevent connection rejections during complex fetch operations or client-side plugin interactions.¹⁴

Command Construction:

The robust command to start the server manually is:

Bash

lms server start --port 1234 --cors

However, this is a foreground process. For a production report, we must configure this as a background daemon.

Relying on a terminal window to keep the server running is fragile. We must create a Systemd service to manage the LM Studio process, ensuring it starts on boot and restarts on failure.¹⁰

Service File Creation:

Create a file at /etc/systemd/system/lmstudio.service.

Ini, TOML

[Unit]
Description=LM Studio Inference Server
After=network-online.target
Wants=network-online.target


Type=simple
User=your_username
# Set the home directory to ensure lms finds the model cache
Environment="HOME=/home/your_username"
WorkingDirectory=/home/your_username

# Pre-start command to ensure a specific model is loaded into memory
# This uses the 'load' subcommand to prep the GPU before the server listens
# --gpu max: Offload all layers to GPU
# --context-length: Set to 8192 or model max
# --identifier: Sets the API name (e.g., "local-model")
ExecStartPre=/usr/local/bin/lms load <model_key_from_lms_ls> --gpu max --context-length 8192 --identifier local-llm

# Main process: The HTTP server
ExecStart=/usr/local/bin/lms server start --port 1234 --cors

# Restart logic
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Key Configuration Details:

Enabling the Service:

Bash

sudo systemctl daemon-reload
sudo systemctl enable lmstudio
sudo systemctl start lmstudio

Check status with systemctl status lmstudio to ensure the model loaded successfully and the port is listening.¹⁰

4. The Presentation Layer: Open WebUI via Docker

Open WebUI (formerly Ollama WebUI) provides a polished, ChatGPT-like interface. It supports chat history, user accounts, RAG (document upload), and web search.

A critical challenge in containerizing Open WebUI on Linux is establishing connectivity to the LM Studio service running on the host.

Create a directory for the project:

Bash

mkdir ~/ai-stack
cd ~/ai-stack
touch docker-compose.yml

docker-compose.yml Content:

YAML

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: always
    ports:
      - "3000:8080"  # Expose Container Port 8080 to Host Port 3000
    environment:
      # Connection to LM Studio
      # We use the /v1 suffix because LM Studio is OpenAI-compatible
      - OPENAI_API_BASE_URL=http://host.docker.internal:1234/v1
     
      # API Key: LM Studio doesn't strictly enforce it locally, but the client requires a value.
      - OPENAI_API_KEY=lm-studio
     
      # Security settings
      - WEBUI_AUTH=true  # Enable login screen
      - ENABLE_SIGNUP=true # Enable for initial setup, set to false later for security
     
      # UI Customization
      - DEFAULT_MODELS=local-llm # Matches the identifier set in LM Studio Service
     
    volumes:
      - open-webui-data:/app/backend/data
     
    # CRITICAL: Enable host resolution on Linux
    extra_hosts:
      - "host.docker.internal:host-gateway"

volumes:
  open-webui-data:

Environment Variable Analysis:

Launch the container:

Bash

docker compose up -d

Monitor the logs to ensure it connects to the database and initializes:

Bash

docker compose logs -f

Once running, navigate to http://localhost:3000 in a local browser. Create the first admin account. Go to Settings > Connections and verify that the connection to http://host.docker.internal:1234/v1 is verified (green checkmark). If it fails, check the extra_hosts configuration and ensure the LM Studio service is active.²⁰

5. The Zero-Trust Gateway: Tailscale Funnel & Caddy

At this stage, we have a working local stack: Open WebUI on port 3000 talk to LM Studio on port 1234. Now, we must expose this securely to the internet.

Tailscale offers two exposure modes:

The user request specifies "accessible over the Internet," which implies Funnel. However, exposing internal ports directly to the public web is risky. Therefore, we will use Caddy as an intermediary to secure the traffic.

Enabling Funnel:

Bash

sudo tailscale funnel 443 on

This command reserves the https://<node-name>.<tailnet>.ts.net domain for this machine and routes traffic to it.²

Caddy is chosen over Nginx because of its automatic TLS integration. Caddy can interface directly with the Tailscale socket to provision certificates for .ts.net domains automatically.⁴

Installation on Host: Install Caddy using the official.deb packages (as detailed in section 2.3). Do not run Caddy in Docker unless necessary, as mounting the Tailscale socket into Docker introduces permission complexities.²² Running it on the host is cleaner for this specific architecture.

The Caddyfile is the configuration map. It needs to handle two scenarios:

Generating Passwords:

Bash

caddy hash-password --plaintext "my-secret-api-password"
# Output: $2a$14$.... (hash string)

The Configuration (/etc/caddy/Caddyfile):

Code snippet

# Define the Tailscale domain. Caddy detects the.ts.net suffix
# and attempts to fetch the cert from the local Tailscale daemon.
machine-name.tailnet-name.ts.net {

    # 1. API Protection Block
    # Match any request starting with /v1 (OpenAI standard path)
    handle_path /v1/* {
        # Security: Require Username/Password for API access
        basic_auth {
            # Format: <username> <hashed_password>
            api_user $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG
        }

        # Proxy to LM Studio on the host
        # We rewrite the path because handle_path strips the prefix,
        # but LM Studio EXPECTS /v1. So we use 'handle' or re-add it.
        # BETTER APPROACH: Use 'handle /v1*' to keep the path intact.
    }
   
    # Correction on API Block Logic:
    handle /v1* {
        basic_auth {
            api_user $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG
        }
        reverse_proxy localhost:1234 {
            # Essential for some API clients to respect the proxy
            header_up Host {upstream_hostport}
        }
    }

    # 2. Chat Interface Block (Default Fallback)
    handle {
        # Proxy to Docker container
        reverse_proxy localhost:3000
    }

    # Logging for observability
    log {
        output file /var/log/caddy/access.log
    }
}

Understanding the Logic:

Applying Changes:

Bash

sudo systemctl reload caddy

6. End-to-End Integration and Testing

We now have the complete pipeline:

Internet -> Tailscale Funnel -> Caddy (Auth/Route) ->

To test the API endpoint, we use curl. This simulates an external app (like a VS Code extension or a custom Python script) connecting to your backend.

Test 1: Unauthenticated (Should Fail)

Bash

curl -I https://machine-name.tailnet-name.ts.net/v1/models

Expected Result: HTTP/2 401 Unauthorized. This confirms Caddy is protecting your API.²³

Test 2: Authenticated (Should Succeed)

Bash

curl -X GET https://machine-name.tailnet-name.ts.net/v1/models \
  -u "api_user:my-secret-api-password"

Expected Result: HTTP/2 200 OK and a JSON list of models, including local-llm.

Test 3: Chat Completion

Bash

curl -X POST https://machine-name.tailnet-name.ts.net/v1/chat/completions \
  -u "api_user:my-secret-api-password" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "local-llm",
    "messages": [
      { "role": "user", "content": "Hello, world!" }
    ]
  }'

Expected Result: A JSON response containing the generated text "Hello! How can I help you today?".

7. Advanced Security and Maintenance

While Open WebUI has built-in auth, for enterprise-grade security, consider disabling the native auth and using an SSO provider or simply relying on Tailscale identity if switching to "Serve" mode.

To prevent Denial of Service (DoS) attacks on your GPU, configure rate limiting in Caddy.

Code snippet

handle /v1* {
    rate_limit {
        zone api_zone 10mb
        events 10
        window 1m
    }
    #... rest of config
}

Note: Rate limiting requires the rate-limit module for Caddy, which may require a custom build. Alternatively, use Fail2Ban on the host to monitor Caddy access logs for 401 errors.

GGUF models are updated frequently. Create a cron job or a simple script to update your model:

Bash

#!/bin/bash
# update_model.sh
systemctl stop lmstudio
lms get llama-3-8b-instruct --force
systemctl start lmstudio

8. Troubleshooting Guide

This architecture represents a mature, sovereign alternative to cloud AI APIs. By leveraging Tailscale Funnel, we have eliminated the most dangerous aspect of self-hosting—opening firewall ports—while maintaining global accessibility. Caddy provides the necessary security layer, wrapping a raw internal API in HTTPS and Basic Authentication. Docker and LM Studio decouple the application logic from the inference runtime, allowing for independent scaling and updates.

The result is a robust, personal AI cloud that is secure by design, privacy-preserving by default, and capable of delivering commercial-grade LLM experiences on your own terms.