Skip to main content
The inference gateway sits in front of Ollama. A remote Entity worker reaches it through an authenticated route, while identity, memory, and tools stay with the worker. Use this topology when you want the worker and model host to live on different machines. A hosted API route does not need this gateway.

​
Configure the gateway

From the application checkout on the model host:
The wizard coordinates the bearer token, tunnel configuration, local environment, and startup. Review each generated setting for your host and domain. The gateway host uses INFERENCE_GATEWAY_TOKEN. The worker uses the same value in NGRAM_INFERENCE_GATEWAY_TOKEN, along with the public gateway root in NGRAM_INFERENCE_BASE_URL.

​
Operate the stack

The wrapper manages the gateway stack through the supplied platform script. By default the local gateway binds to 127.0.0.1:8010; the tunnel handles remote connectivity. To stop the gateway while preserving Ollama:
To stop and restart it with the same tunnel name:
Use the CLI reference for tunnel name, host, port, and configuration overrides.

​
Cloudflare Access

When the route also requires Cloudflare Access service authentication, the worker accepts NGRAM_CF_ACCESS_CLIENT_ID and NGRAM_CF_ACCESS_CLIENT_SECRET. Supply both privately; a partial pair is rejected. The Access credentials and the gateway bearer token serve different checks. Keep both sides aligned and do not put them in a shell’s public manifest.

​
Diagnose each hop

Confirm the local model endpoint, local gateway, tunnel, and remote worker in that order. A reachable tunnel with a rejected bearer token is an authentication problem; an authenticated gateway that cannot reach Ollama is a backend problem. Use gateway status and worker logs to identify the failing hop before changing model IDs or memory configuration. Hosted providers use different health and catalog paths, so do not use a gateway /health assumption for every API provider.