Skip to main content

Installation

warning

🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.

Cortex.cpp Installation​

Cortex.cpp offers four installer types​

  • Network Installers download a minimal system and require an internet connection to fetch packages during installation.
  • Local Installers include all necessary packages, enabling offline installation without internet access.
  • Dockerfile Installers are used to build a Docker image for Cortex.
  • Binary files without package management.

Cortex.cpp supports three channels​

  • Stable: The latest stable release on github.
  • Beta: The release candidate for the next stable release, available on github release with the tag vx.y.z-rc1
  • Nightly: The nightly build of the latest code on dev branch, available on discord.

Download URLs​

TypeVersionOperating SystemDownload Link
LocalStableWindowsDownload
LocalStableLinuxDownload
LocalStableMacOSDownload
info

For other versions, please look at cortex.cpp repo or each installer page.

Minimum Requirements​

OS​

  • MacOS 12 or later
  • Windows 10 or later
  • Linux: Ubuntu 20.04 or later, Debian 11 or later (For other distributions, please use the Dockerfile installer or binary files, we have not tested on other distributions yet.)

Hardware​

CPU​

info
  • Cortex.cpp supports a processor that can handle AVX2. For the full list, please see here.
  • We support older processors with AVX, AVX-512 and non-AVX, though this is not recommended.
Intel CPU​
  • Haswell processors (Q2 2013) and newer.
  • Tiger Lake (Q3 2020) and newer for Celeron and Pentium processors.
AMD CPU​
  • Excavator processors (Q2 2015) and newer.

RAM​

info

We support DDR2 RAM as the minimum requirement but recommend using newer generations of RAM for improved performance.

  • 8GB for running up to 3B models (int4).
  • 16GB for running up to 7B models (int4).
  • 32GB for running up to 13B models (int4).

GPU​

info

Having at least 6GB VRAM when using NVIDIA, AMD, or Intel Arc GPUs is recommended.

  • 6GB can load the 3B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
  • 8GB can load the 7B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
  • 12GB can load the 13B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
info
  • NVIDIA driver version 470.63.01 or higher. Note: Users must install the driver by themselves.
  • CUDA Toolkit version 12.0 or higher. Note: Cortex.cpp will automatically detect and install the required version of cudart to the user's machine.

Disk​

  • At least 10GB for app storage and model download.