> ## Documentation Index > Fetch the complete documentation index at: https://docs.akumi.eu/llms.txt > Use this file to discover all available pages before exploring further. # Model router > Route requests across providers with aliases and fallback. The Model router decides which model serves each request. By default everything runs on platform-hosted, EU-resident models. The router lets you choose which models your organisation can use, give them friendly names with automatic fallback, bring your own provider keys, and, once you have accepted the risk, route to external providers outside the EU. You will find it under **Services > Model router**. ## EU by default Out of the box the router only allows **EU-sovereign** models: requests stay on infrastructure inside the EU. External providers (OpenAI, Anthropic, Google, Mistral) sit outside the EU and are **blocked** until your organisation explicitly accepts non-compliant routing. That acceptance is recorded once, with the identity and timestamp of the admin who made it, and can be revoked at any time. Even after acceptance, an external call only proceeds when the request is protected: either the PII firewall is on for that request, or the request carries a valid compliance acknowledgement. The egress guard enforces this in code and **fails closed**: if neither condition holds, the call is refused rather than sent. ## Models The **Models** tab lists the model catalogue with the context window, capabilities and credit cost per 1,000 input and output tokens for each. Use it to enable or disable individual models for your organisation. A disabled model is rejected for your tenant even though it exists in the catalogue. You can sort the table by model, provider, or input/output price to compare options. The same catalogue is available over the API, so you can build a model picker or validate a prompt length without hardcoding anything: ```bash theme={null} curl https://api.akumi.eu/v1/models \ -H "Authorization: Bearer $AKUMI_API_KEY" ``` ```json theme={null} { "id": "mistral/mistral-large-2512", "object": "model", "owned_by": "mistral", "kind": "chat", "context_length": 262144, "max_output_tokens": null, "capabilities": ["tools", "vision", "streaming"], "eu_resident": true, "pricing": { "credits_per_1k_input": 1, "credits_per_1k_output": 2 } } ``` | Field | Meaning | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | `context_length` | Total window in tokens, prompt plus completion | | `max_output_tokens` | The provider's cap on completion length. `null` means the provider publishes no separate cap, so output is bounded by the context window | | `capabilities` | Any of `tools`, `vision`, `json_schema`, `reasoning`, `streaming` | | `eu_resident` | Whether the serving provider is EU-resident. This is the field to filter on if residency is your constraint | | `pricing` | Credits per 1,000 tokens, the same unit the meter debits | A `null` is a fact, not a gap: it means the provider publishes no value, and we would rather say so than invent one. The list contains only the models your workspace may actually call, so anything disabled for you is absent rather than present-but-forbidden. ## Routing aliases A routing alias is a name you invent that maps to one or more real models. You send the alias as the `model` in your request; the router resolves it to the first target, and if that target is unavailable it falls back to the next one in order. Aliases let you keep a stable name in your code, for example `fast` or `default`, and change what it points to here without redeploying. They also give you resilience: a primary model with an external fallback (or vice versa) keeps requests flowing if one provider has an outage. ### Aliases or a models chain? You can also send a chain inline, with `models` instead of `model`: ```json theme={null} { "models": ["mistral/mistral-large-2512", "fast", "openai/gpt-5.4"], "messages": [{ "role": "user", "content": "Summarize this contract." }] } ``` Both produce the same thing: an ordered list of models to try until one succeeds. They differ in who controls them and how quickly they change. | | Routing alias | `models` in the request | | -------------- | ----------------------------------- | -------------------------------------------- | | Who sets it | A workspace admin, in the dashboard | Anyone holding an API key for that workspace | | Where it lives | Stored on the workspace | The request body | | Changing it | Takes effect on the next request | Needs a client deploy | | Indirection | Yes, a stable name | No, an explicit list | **An alias is policy.** Its value is late binding: when a provider has an incident at two in the morning, an admin repoints the alias and every request moves, with no deploy from anyone. **A chain is a preference.** Its value is that there is nothing to set up: paste a list into the call you already have. They compose. An alias inside `models` expands in place, so the chain above tries the large Mistral model, then whatever `fast` currently resolves to, then GPT. That lets you lean on an admin's curated chain and still append your own last resort. Whichever you use, a fallback is never silent, but where the target that served is named depends on how you asked. On an ordinary response it is the `model` field. On a stream it is the additive `served` object on the finishing chunk, because every chunk of one completion carries one envelope and what will serve is not known when the first of them goes out. Do not read `model` on a stream to detect a fallback; read `served`, described under [Streaming](#streaming). ## Provider preferences A request can express a routing preference with a `provider` object. **Read this first, because the name is borrowed and one meaning differs.** Other gateways use `provider.order` to choose between several companies hosting the *same* model. Our catalogue maps one model to exactly one provider, so there is no host to choose. Here `order` sorts **the entries of your chain** by who serves them. The practical consequence: on a single-model request, `provider.order` does nothing at all. It only has an effect once there is more than one target. ```json theme={null} { "models": ["openai/gpt-5.4", "mistral/mistral-large-2512"], "provider": { "order": ["mistral"], "ignore": ["anthropic"], "allow_fallbacks": true, "require_parameters": false, "residency": "eu" }, "messages": [{ "role": "user", "content": "Summarize this contract." }] } ``` | Field | Effect | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `order` | Provider slugs to try first. Promotes, never excludes: providers you do not name keep their relative position behind the ones you do | | `only` | Restrict the chain to these providers | | `ignore` | Drop these providers from the chain | | `allow_fallbacks` | `false` pins the request to the first viable target instead of continuing down the chain. A pinned request is also served from cache only when the stored response came from the head of the chain, so pinning can cost you a cache hit | | `require_parameters` | `true` refuses a target that cannot honour every parameter you sent, rather than dropping the unsupported ones. Overrides your organisation's default for this request | | `residency` | `eu` drops every non-EU target, even ones your acceptance would otherwise permit. `any` defers to policy | ### Preferences only narrow Every field above can remove targets. None can add one. Nothing you send in a request can enable a model your workspace has disabled, reach a provider you have no credentials for, or route data outside the EU without a recorded acknowledgement. An API key belongs to one workspace, so a key holder reorders and restricts *within* what an admin permitted, and never past it. `residency: "eu"` is the useful case: it lets a single sensitive request insist on EU-only routing without changing any configuration, even in an organisation that has accepted external routing generally. If your preferences remove every target, the request is refused rather than quietly widened, and the reason is recorded on the audit event. ## Provider keys (bring your own key) By default, calls to external providers use the platform's relationship with that provider. On the **Provider keys** tab you can instead store your own API key for a provider. When a key is present, requests to that provider are billed to your account with them rather than brokered through the platform. Keys are encrypted at rest and never shown again after you save them: only the last few characters are displayed so you can tell which key is stored. ## Compatibility The **Compatibility** tab tunes how the router handles requests that do not map cleanly onto every provider: * **Unsupported parameters.** When a request uses an OpenAI parameter a target provider does not support, choose **lenient** (drop the unsupported parameter and proceed) or **strict** (refuse rather than silently ignore it). In strict mode a chain moves on to the next target instead of failing outright, so a model that cannot do tool calls simply steps aside for one that can. The request fails only when no target can honour it. * **Firewall tool content.** When on, the PII firewall also scans the content of tool and function-call messages, not just ordinary message text, before anything leaves the platform. ### What is additive over the OpenAI API Everything below is extra. An unmodified OpenAI client works without any of it. * `models` in place of `model`, for an inline fallback chain. Sending both is rejected, so there is no guessing about which one you meant. * The `provider` object described above. * `usage.credits`, a `{base, services, total}` breakdown on every response, so you can reconcile spend without opening the dashboard. `base` is inference, `services` is everything else charged against that request, and `total` matches what the ledger debited. Note that `services` is an amount of credits, not a list of which services ran. * Streaming follows OpenAI's convention: set `stream_options: {"include_usage": true}` and a final chunk arrives carrying `usage` with an empty `choices` array. We also keep `usage` on the finishing chunk whether or not you ask, so nobody loses sight of what a request cost by not knowing the flag exists. ## Streaming Set `stream: true` and the answer is written as the serving provider generates it. Chunks are the provider's own tokens, so they are variable in size and there is no fixed length to build on: read `delta.content` and append it. A cached answer and a provider that does not stream both arrive in even pieces instead, which is why the size is not something to depend on either way. Tool calls stream too, in OpenAI's incremental shape. Each chunk carries one `delta.tool_calls[]` entry with an `index`, and a call's `id` and `function.name` arrive once at its start while `function.arguments` arrives as a series of fragments that are not valid JSON on their own. Group by `index`, concatenate the argument fragments in order, and parse when the stream finishes. The `choices` of three consecutive chunks, with the envelope each of them also carries (`id`, `object`, `created`, `model`) left out: ```json theme={null} [{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_1","type":"function","function":{"name":"lookup_invoice"}}]},"finish_reason":null}] [{"index":0,"delta":{"tool_calls":[{"index":0,"id":null,"type":"function","function":{"arguments":"{\"id\":"}}]},"finish_reason":null}] [{"index":0,"delta":{"tool_calls":[{"index":0,"id":null,"type":"function","function":{"arguments":"\"INV-9\"}"}}]},"finish_reason":null}] ``` **What served is named on the finishing chunk.** Every chunk of one completion carries the same `id` and `model`, because a client accumulating them keys on that and a second identity appearing halfway through is what such a client cannot survive. So `model` on a stream is the name the completion was opened under, not the target that answered: the router has not called anyone when the first chunk goes out, and an alias or a `models` chain does not resolve until it does. The finishing chunk carries an additive `served` object instead, and that is what to read when you need to know a fallback happened: The finishing chunk, with the envelope and the `usage` object it also carries left out: ```json theme={null} {"served":{"model":"mistral/mistral-large-2512","provider":"mistral","completion_id":"cmpl-abc123"},"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} ``` | Field | Meaning | | --------------- | ----------------------------------------------------------------------------------------------------- | | `model` | The catalogue slug of the target that produced the answer, which is the alias or chain entry that won | | `provider` | The provider that served it | | `completion_id` | That provider's own id for the completion, for a support conversation with them | A cached answer is the exception, and it is the one case where it does not matter: it reached no provider, so it carries no `served` object and its `model` is the target that produced the stored answer, exactly as on an ordinary response. **Fragments are withheld where they cannot be re-identified.** A pseudonym can be split across two fragments of argument JSON, and rewriting half of one would corrupt the call your machine then executes. So on a request whose prompt was pseudonymized, the fragments are not sent at all and the finishing chunk carries the whole `tool_calls` array with the real values restored, entries indexed exactly as the fragments would have been. On a request with nothing masked, which is the default EU route, you get the fragments and the finishing chunk does not repeat the calls. One accumulator keyed on `index` handles both, so reading them the way above covers each case without knowing which one you are in. **A Guard block ends the stream rather than raising.** The 200 and the headers go out with the opening chunk, so once the body has begun there is no status code left to send. When an output rule blocks the answer, the last chunk's choice carries `finish_reason: "content_filter"` and, beside it, an additive `content_filter.categories` naming what fired; `[DONE]` follows. Treat `content_filter` as a refusal, not as a completed answer. A non-streaming request is unaffected and still receives the `guard_violation` 422. **A Guard redaction on a streamed answer arrives too late to apply.** An output rule set to redact rewrites the answer, and on a stream that rewrite is produced after the text it removes has already been sent to you. It cannot be sent afterwards either, because you would append it to what you hold rather than replace it. So the finishing chunk keeps its ordinary `finish_reason` and carries an additive `content_filter: {"redacted": true}`, which says a rule rewrote this answer and the copy you assembled is not the rewritten one. Everything the platform stores afterwards, the cached entry and the conversation transcript, holds the redacted form. If a redaction has to reach your readers, use buffered below or do not stream. Whether the answer streams while Guard's output rules are still to run is a workspace setting, under [Guard](/guides/guard). **Async**, the default, streams and reports afterwards: a block ends the stream, a redaction is only marked. **Buffered** delivers nothing an output rule has not passed: where the workspace enforces and a rule scans the output, nothing streams and the whole answer arrives at once, already redacted where a rule redacted it, and a block instead ends the stream with the same `content_filter` chunk before a byte of the answer has been written. Buffered holds nothing back where there is nothing to wait for, so a request still streams under it when no rule scans the output, when the enforcement mode is shadow, which is where a workspace sits until it explicitly enforces, and when the Guard service is off. **Any other mid-stream failure is an error frame inside the 200**, in the same `{"error": {"message", "type", "code"}}` shape a failed non-streaming request returns. A provider that dies partway through an answer is the case this exists for. **Fallback stops at the first byte.** The chain in [Routing aliases](#routing-aliases) tries the next target when one fails, but only while nothing has been delivered. Once any fragment of an answer has reached you, a later failure is reported rather than retried elsewhere: a second target would restart the answer from its first token, and you would render the fragments you already hold followed by the whole of a different answer, with nothing in the stream marking the seam. ## Good to know **External routing is never on by default.** It requires a recorded acceptance, and each external call still needs the firewall or a valid acknowledgement. The opt-out is deliberate and auditable. **Pseudonymised data is still personal data.** Routing a firewalled request to a non-EU provider reduces exposure but does not remove your GDPR obligations. Review the warning shown when you accept non-compliant routing. **Routing decisions are audited.** The model chosen, the region it ran in, and whether egress was allowed or blocked are written to the audit log for every request.