Request Options

The LLMaaS Gateway accepts a number of request headers that change how an individual request is handled — without touching your account defaults or the model parameters in the body. These options are applied per request, so you can opt in only where they help.

Response Caching

The gateway can cache responses and replay them for repeated requests, avoiding a round-trip to the model and cutting both cost and latency. It supports two lookup paths:

Direct (hash) matching — deterministic, exact-match replay. The request is normalized and hashed; an identical request is served instantly, with no embeddings required.
Semantic (similarity) matching — embedding-based lookup that serves a cached answer when a new request is close enough to a previous one, even if the wording differs.

Caching is opt-in per request: send an x-bf-cache-key header to scope a request to a cache partition. Without it, the request bypasses the cache entirely. The key also isolates entries from each other — a request under one key can never be served a response cached under another.

Only direct matching is available

On the LLMaaS Gateway only Direct (hash) matching is currently enabled. Semantic (similarity) matching is not available yet — an identical (normalized) request is required for a cache hit.

On a cache miss, the gateway returns the model's response immediately and stores it in the background, so the first request is never delayed by a cache write. Send the same request again to observe a hit. Cached entries expire after a time-to-live (TTL) and persist across gateway restarts.

Activating caching

Caching activates the moment a request carries an x-bf-cache-key header — no separate enable step is needed. Pick any string as the key; it defines the cache partition. Identical requests under the same key hit the cache, while a different key keeps entries isolated.

curl https://ai.ewcs.ch/v1/chat/completions \
  -H "Authorization: Bearer sk-bf-..." \
  -H "Content-Type: application/json" \
  -H "x-bf-cache-key: my-feature" \
  -d '{
    "model": "ew/glm-5.2",
    "messages": [{"role": "user", "content": "Summarize this in one sentence."}]
  }'

Send the same request again — the second response is served from cache. To scope another feature or tenant separately, use a different key.

In opencode, set the header through the provider's options.headers object (see Setting request headers in opencode):

"headers": {
  "x-bf-cache-key": "opencode"
}

Cache header reference

Every cache behavior can be overridden per request via headers. The x-bf-cache-key header is the only one required to engage caching; the rest are optional overrides.

Header	Context key (Go)	Value	Effect
`x-bf-cache-key`	`CacheKey`	string	Scope this request to a cache partition. Required (or `default_cache_key`) for caching to engage.
`x-bf-cache-ttl`	`CacheTTLKey`	duration string or seconds	Override TTL for this request. Invalid values are ignored.
`x-bf-cache-threshold`	`CacheThresholdKey`	float (0–1)	Override the semantic similarity threshold. Clamped to `[0,1]`.
`x-bf-cache-type`	`CacheTypeKey`	`direct` or `semantic`	Limit lookup to a single path.
`x-bf-cache-no-store`	`CacheNoStoreKey`	`true`	Skip writing the response (still serves cached hits).

Direct-only mode

With only Direct (hash) matching available, x-bf-cache-threshold and x-bf-cache-type: semantic have no effect — every request uses direct matching. x-bf-cache-key, x-bf-cache-ttl, and x-bf-cache-no-store remain functional.

Request & Response Logging

The gateway logs every LLM request and its response, which lets operators inspect traffic, troubleshoot issues, and monitor usage. Log entries are retained for a limited time and are visible to platform operators.

By default the full message content (prompts and generated responses) is recorded alongside metadata. If a request contains sensitive data and you do not want its content persisted, you can disable content logging for that single request by sending the x-bf-disable-content-logging: true header. Only metadata (timestamps, model, token counts, status) is then retained — the message content is not stored.

Header is per-request

The header affects only the request it is sent on. It does not change the gateway's global logging configuration, and it does not disable logging of metadata.

Setting request headers in opencode

opencode passes custom request headers through the provider's options.headers object. In the ew provider block from the OpenCode guide, add a headers entry to options:

"options": {
  "baseURL": "https://ai.ewcs.ch/v1",
  "apiKey": "{file:~/.config/opencode/ew-key}",
  "headers": {
    "x-bf-disable-content-logging": "true"
  }
}

Every request opencode sends through this provider now carries the header, so content logging is suppressed for that client. The same headers object can carry other gateway headers — for example x-bf-cache-key to opt a request into response caching (see Response Caching).