Skip to content
Cortex

Installation Guide 🚀

Quick Install

Download and install Cortex for your platform:

PlatformDownload LinkInstallation Steps
macOSDownload1. Open the .pkg file
2. Follow the installer wizard
LinuxDownloadRun: curl -s https://raw.githubusercontent.com/menloresearch/cortex/main/engine/templates/linux/install.sh | sudo bash -s
WindowsDownload1. Run the installer
2. Follow the setup wizard

Verify Your Installation

After installing, verify Cortex is working:

Terminal window
cortex -v

System Requirements

Before installing, ensure your system meets these minimum requirements:

Essential Requirements

  • CPU: Intel Haswell (2013+) or AMD Excavator (2015+) with AVX2 support
  • RAM: 8GB minimum (16GB recommended)
  • Storage: 10GB free space
  • OS: macOS 12+, Windows 10+, or Ubuntu 20.04+/Debian 11+

GPU Support (Optional)

For accelerated performance:

  • VRAM: 6GB minimum (8GB+ recommended)
  • NVIDIA: Driver 470.63.01+ and CUDA 12.0+
  • AMD: Driver 22.7.1+

Platform-Specific Instructions

macOS Installation

  1. Download the installer package from the links above
  2. Open the downloaded .pkg file by double-clicking it
  3. Follow the installation wizard to complete setup
  4. Verify installation by opening Terminal and running:
    Terminal window
    cortex -v

Installation Locations

  • Binary: /usr/local/bin/cortex, /usr/local/bin/cortex-server
  • Data: /Users/<username>/cortexcpp
  • Config: /Users/<username>/.cortexrc
  • Uninstaller: /usr/local/bin/cortex-uninstall.sh

Uninstall

Run the uninstaller script:

Terminal window
sudo sh /usr/local/bin/cortex-uninstall.sh
PlatformStable ReleaseBeta ReleaseNightly Build
macOSDownloadDownloadDownload
LinuxDownloadDownloadDownload
WindowsDownloadDownloadDownload

Minimum Requirements

Before installing Cortex, ensure your system meets these requirements:

Operating Systems

  • macOS: macOS 12 (Monterey) or later
  • Linux: Ubuntu 20.04+, Debian 11+, or latest Arch Linux
    • Other distributions may work with Docker or binary files
  • Windows: Windows 10 or later

Hardware Requirements

CPU

  • Intel: Haswell (Q2 2013) or newer; Tiger Lake (Q3 2020)+ for Celeron/Pentium
  • AMD: Excavator (Q2 2015) or newer

RAM

  • 8GB: Up to 3B parameter models (int4 quantization)
  • 16GB: Up to 7B parameter models (int4 quantization)
  • 32GB: Up to 13B parameter models (int4 quantization)

GPU (Optional)

  • 6GB VRAM: 3B parameter models (int4) with full-speed inference
  • 8GB VRAM: 7B parameter models (int4) with full-speed inference
  • 12GB VRAM: 13B parameter models (int4) with full-speed inference

NVIDIA Requirements:

  • NVIDIA driver 470.63.01+
  • CUDA Toolkit 12.0+ (Cortex will automatically install required CUDA runtime)

Storage

  • At least 10GB free space for models

Installation Options

Cortex offers four installation methods:

  • Network Installers: Minimal downloaders that fetch packages during installation (requires internet)
  • Local Installers: Complete packages for offline installation
  • Docker Images: Pre-built containers or build-it-yourself options
  • Binary Files: Direct executable files without package management

Updating Cortex

Terminal window
sudo cortex update

Building from Source

macOS Build

Prerequisites

  • CMake 3.10+
  • gcc/g++ 9+
  • ninja-build
  • make-gnu

Build Steps

Terminal window
git clone https://github.com/menloresearch/cortex.cpp.git
cd cortex.cpp
git submodule update --init


cd engine
make configure-vcpkg


# For Mac Silicon (M1/M2)
make build CMAKE_EXTRA_FLAGS="-DCORTEX_CPP_VERSION=latest -DCMAKE_BUILD_TEST=OFF -DMAC_ARM64=ON -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"


# For Mac Intel
make build CMAKE_EXTRA_FLAGS="-DCORTEX_CPP_VERSION=latest -DCMAKE_BUILD_TEST=OFF -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"


# Test the build
./build/cortex -h

Working with Cortex

After installing Cortex, you can start using it immediately. Here’s how to perform common tasks:

Using Cortex in Docker

1. Check Available Engines

Terminal window
curl --request GET --url http://localhost:39281/v1/engines \
  --header "Content-Type: application/json"

2. Download Models

First, monitor events:

Terminal window
websocat ws://localhost:39281/events

Then download a model:

Terminal window
curl --request POST --url http://localhost:39281/v1/models/pull \
  --header 'Content-Type: application/json' \
  --data '{"model": "tinyllama:gguf"}'

3. Start and Use a Model

Terminal window
# Start model
curl --request POST --url http://localhost:39281/v1/models/start \
  --header 'Content-Type: application/json' \
  --data '{"model": "tinyllama:gguf"}'


# Query the model
curl --request POST --url http://localhost:39281/v1/chat/completions \
  --header 'Content-Type: application/json' \
  --data '{
    "frequency_penalty": 0.2,
    "max_tokens": 4096,
    "messages": [{"content": "Tell me a joke", "role": "user"}],
    "model": "tinyllama:gguf",
    "presence_penalty": 0.6,
    "stop": ["End"],
    "stream": true,
    "temperature": 0.8,
    "top_p": 0.95
  }'

Troubleshooting

Common Issues

Permission Errors on Linux/macOS

Terminal window
# Fix permissions for data directory
sudo chown -R $(whoami):$(whoami) ~/cortexcpp

Docker Container Won’t Start

Terminal window
docker logs cortex
docker system info  # Check available resources

GPU Not Detected

  • Ensure NVIDIA drivers are properly installed
  • Verify CUDA Toolkit installation
  • For Docker: check that nvidia-container-toolkit is configured correctly

Release Channels

Cortex offers three release channels:

  • Stable: Latest released version, recommended for most users
  • Beta: Release candidates for upcoming stable versions (tagged as vx.y.z-rc1 on GitHub)
  • Nightly: Latest development builds from the dev branch (available on Discord)

Different channels use different naming conventions and file locations:

ChannelApp NameBinary Name (Linux/Mac)Binary Name (Windows)
Stablecortexcppcortex, cortex-servercortex.exe, cortex-server.exe
Betacortexcpp-betacortex-beta, cortex-server-betacortex-beta.exe, cortex-server-beta.exe
Nightlycortexcpp-nightlycortex-nightly, cortex-server-nightlycortex-nightly.exe, cortex-server-nightly.exe