Skip to content

Use Ollama

Hangar serves GGUF Repos to Ollama the way huggingface.co does. Ollama signs its requests with the key pair in ~/.ollama, and Hangar needs that key for every Repo unless the Repo is public and the Instance allows anonymous reads (security.anonymous_read, off by default). Register the key once:

hangar ollama key register

or paste ~/.ollama/id_ed25519.pub under Settings, Ollama keys. Then this works with no Modelfile:

ollama pull hangar.example.com/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M
ollama run hangar.example.com/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M
  • The tag after : picks a quant, case-insensitively, or a full GGUF file name. Without a tag (Ollama's latest), Hangar picks Q4_K_M if the Repo has it, otherwise the closest quant.
  • The chat template comes from the Repo's template file, or is matched from the GGUF's own chat template. system, params and LICENSE files in the Repo become the system prompt, parameters and licence.
  • A matching mmproj-*.gguf projector is included for vision models.
  • The quant must be in the Arsenal. For a Partial mirror without it, ollama pull fails with file does not exist: Ollama does not show the message Hangar sends with it, which names the Pull to run. Pull the quant first with hangar pull <repo>:<quant> --server-only, or use hangar ollama install, which checks for you. If an admin turned on Pull-through, the request starts the Pull and ollama pull asks you to try again when it finishes.

Install with the hangar CLI

hangar ollama install runs ollama pull for you, after two checks:

hangar ollama install bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M

  1. It checks that the quant is in the Arsenal. Without a quant, it checks the one Ollama asks for as latest. If the quant is missing, it says so, prints the command that pulls it into the Instance, and asks whether to run it:

    bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M is not in the Arsenal, so Ollama would fail with "file does not exist". Pull it into the Instance first:
  hangar pull bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M --server-only
Pull it now? [y/N]

    Answer y to pull it and continue. Any other answer stops with exit code 3 and leaves the command for you to run. To pull without the question, add --pull.

  2. It checks that your Ollama key is registered and offers to register it. Add --yes to register it without the question.

A quant the Repo does not have stops the command with the list of quants it does have.

HTTPS or HTTP

Ollama uses HTTPS unless you pass --insecure. Use HTTPS with a trusted certificate (see TLS and reverse proxies), or for a plain-HTTP Instance on your network:

ollama pull --insecure hangar.lan:8080/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M

hangar ollama install adds --insecure for http:// Instances.

Private Repos

With your key registered, Hangar gives Ollama a short-lived token for each Repo you can read, private Repos included.

Limits

  • ollama push to Hangar is not supported. Use hangar import --from-ollama to bring models you have in Ollama into the Arsenal.
  • Ollama must reach Hangar on the host the model name gives, at the root of that host. Ollama refuses redirects to other hosts, so Hangar always streams Ollama downloads itself, even on S3 in redirect mode.