> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akumi.eu/llms.txt
> Use this file to discover all available pages before exploring further.

# Cache

> Reuse stored responses for repeated requests to cut cost and latency.

The Cache service returns a stored response for a repeated request instead of calling a model again, cutting both cost and latency. When a request you have served before comes in again, the platform replays the earlier answer and skips the model call entirely.

You will find Cache under **Services > Cache**.

## Turning Cache on

Use the **Enabled** switch in the top right of the page. While it is off, nothing is cached and nothing is served from cache. Once on, caching happens automatically; you do not need to change your requests.

## What gets cached

Caching is **automatic but deterministic-only**: a request is cached only when its result is reproducible, which means either `temperature: 0` or a fixed `seed`. Requests with `temperature` above 0 and no seed are left alone: you asked for variety, so you get a fresh answer every time.

You can change this in two ways:

* **Per request:** send `"cache": false` in the chat-completions body to bypass the cache for that call (both reading and writing), regardless of settings.
* **Per organisation:** turn on **Also cache non-deterministic responses** in the configuration to cache `temperature > 0` requests too. This returns an identical answer for identical inputs even where the model would normally vary, so only enable it if that is acceptable for your use case.

## Exact and semantic matching

**Exact matching** is always on when the service is enabled. The platform builds a key from the request as you sent it (the model, the messages, the sampling parameters, and any tools) and serves a stored response when an identical request arrives again.

**Semantic matching** is optional, toggled on the **Configuration** tab. On an exact miss, the platform embeds the prompt and looks for a *near-duplicate* earlier request within a similarity threshold you set. This catches paraphrases an exact key would miss, but it can return an answer written for a *similar* prompt, so raise the threshold if you need matches to be tight.

## How a hit is served

The cache keys on the request **as you sent it**, before any other service runs. A hit therefore short-circuits the whole pipeline: the stored response is returned immediately, skipping retrieval, memory, the PII firewall, and the model call. That is what makes a hit both cheap and fast.

The trade-off is freshness. If you also use Recall, a cached answer reflects your documents and stored facts **as they were when the answer was first produced**, until the entry expires. The time-to-live bounds how stale a cached answer can get.

## Time-to-live

Set how long entries live on the **Configuration** tab under **Cache lifetime**: 1 hour, 24 hours, 7 days, or 30 days. Expired entries are never served and are pruned automatically. Lower the TTL when freshness matters more than hit rate; raise it for stable, repetitive workloads.

## Clearing the cache

The **Clear cache** action empties your organisation's cache immediately, after a confirmation prompt. Use it when you have changed something upstream (a prompt template, a Recall collection, a model) and want stored answers regenerated rather than waiting for them to expire.

## Metering and records

Cache hits are metered and appear in your usage. A hit records a small `cache.hit` charge **and skips the base inference charge**; that skipped charge is your saving. Misses run as normal and are billed for the inference they perform. Configuration changes and cache clears are written to the audit log.

## Privacy and isolation

Your cache is strictly scoped to your organisation: one tenant's stored responses are never visible to another. Cached responses are stored at rest and may contain personal data from the original answer, so they are isolated per organisation and bounded by the TTL; clearing the cache or letting entries expire removes them.

## Good to know

**Cache needs the service enabled.** While it is off, requests run the full pipeline every time.

**Only reproducible requests are cached by default.** `temperature: 0` or a `seed`; otherwise the request bypasses the cache unless you opt into caching volatile responses.

**A hit skips everything, including guard and the firewall.** The stored answer already passed those checks when it was first produced; new policy changes apply to fresh (uncached) requests or after the cache is cleared.

**Semantic matching trades precision for hit rate.** Exact matching never returns a wrong answer; semantic matching can return one written for a similar prompt. Tune the threshold, or leave semantic off if you need exactness.