> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ngram.space/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect private inference

> Expose an authenticated model endpoint to a worker without moving the Entity to the model host.

The inference gateway sits in front of Ollama. A remote Entity worker reaches it through an authenticated route, while identity, memory, and tools stay with the worker.

```mermaid theme={"theme":"github-light-default"}
flowchart LR
  E[Entity worker] -->|Authenticated HTTPS| G[Inference gateway]
  G --> O[Ollama]
  E --> M[(Entity memory)]
```

Use this topology when you want the worker and model host to live on different machines. A [hosted API route](/deployment/hosted-inference-testing) does not need this gateway.

## Configure the gateway

From the application checkout on the model host:

```bash theme={"theme":"github-light-default"}
uv run ngram gateway setup
```

The wizard coordinates the bearer token, tunnel configuration, local environment, and startup. Review each generated setting for your host and domain.

The gateway host uses `INFERENCE_GATEWAY_TOKEN`. The worker uses the same value in `NGRAM_INFERENCE_GATEWAY_TOKEN`, along with the public gateway root in `NGRAM_INFERENCE_BASE_URL`.

## Operate the stack

```bash theme={"theme":"github-light-default"}
uv run ngram gateway on
uv run ngram gateway status
```

The wrapper manages the gateway stack through the supplied platform script. By default the local gateway binds to `127.0.0.1:8010`; the tunnel handles remote connectivity.

To stop the gateway while preserving Ollama:

```bash theme={"theme":"github-light-default"}
uv run ngram gateway off --leave-ollama-running
```

To stop and restart it with the same tunnel name:

```bash theme={"theme":"github-light-default"}
uv run ngram gateway restart
```

Use the [CLI reference](/reference/cli) for tunnel name, host, port, and configuration overrides.

## Cloudflare Access

When the route also requires Cloudflare Access service authentication, the worker accepts `NGRAM_CF_ACCESS_CLIENT_ID` and `NGRAM_CF_ACCESS_CLIENT_SECRET`. Supply both privately; a partial pair is rejected.

The Access credentials and the gateway bearer token serve different checks. Keep both sides aligned and do not put them in a shell's public manifest.

## Diagnose each hop

Confirm the local model endpoint, local gateway, tunnel, and remote worker in that order. A reachable tunnel with a rejected bearer token is an authentication problem; an authenticated gateway that cannot reach Ollama is a backend problem.

Use gateway status and worker logs to identify the failing hop before changing model IDs or memory configuration. Hosted providers use different health and catalog paths, so do not use a gateway `/health` assumption for every API provider.