Serve models to agents

A model provider can run the router to serve its own models to HexaEight agents.

What changes

Today a customer creates an API key with you and keeps it wherever their software runs. Whoever holds the key can spend on that account, and the key names an account, not the agent or the person behind a request.

With a router:

• Your models have a name on your domain, such as large.example.com. Serve all your models under one name, or give each model its own, such as large.example.com and fast.example.com. Each name is a router of its own.
• Your customers’ agents call that name directly. When an agent starts, it checks that the address answers as that name, and it does not use an address that does not.
• There is no API key and no OAuth. Every request is encrypted end to end between the customer’s agent and your router, and proves which agent sent it. That proof is the credential: there is nothing to issue, store, rotate or leak.
• You decide who may use it, by identity: a person, or a customer’s agent.
• You bill by identity. Every call is recorded with who it was for, the model and the tokens used.

Who you see

The customer’s agent decides whether the person is named:

the customer’s agent is working for	your router sees, and bills
a person signed in to it	that person, proven with their own key
a caller it keeps private	only the agent

An agent can keep every caller from outside its company private. The agent itself is then your customer: for a personal agent, one person; for a company’s agent, the company.

Your router accepts private callers unless upstreams.yaml says otherwise:

requireIdentifiedCaller: true      # refuse callers who are not named

A provider normally leaves this out. Who a customer’s own users are is the customer’s business.

Set it up

1. A name on your domain

In the HexaEight Authenticator, use an identity whose email is at your domain, add a Domain Resource such as large.example.com, and publish the DNS TXT record the app gives you. The record is checked when the licence is granted. See Domain resource.

2. Install it with that name

Run the one-line installer. In the licence step, scan with the domain resource. When the installer asks Where are your models?, choose 3 or 8, another endpoint that speaks the Anthropic or the OpenAI API, and give the address of your model.

3. Admit your customers

From the router’s folder:

cd ~/heia-router
cp -p router-policy.he "router-policy.he.bak_$(date +%Y%m%d_%H%M%S)"
set -a; . ./env-file; set +a
./hexaeight-router-linux-x64 --init-policy

it asks	answer
Do you have an external authorization service?	N
Who may use this router?	the customers you admit, comma-separated
Which models may they use?	* for every model, or the routes their plan includes, each as route|*
Which agents may vouch for them?	blank
Write this policy?	y

Admit each customer as:

• a customer’s agent, by its name, such as agent01.example.org. Use this when the agent keeps its callers private.

• a person, twice: by their email address, and by the SHA-512 hash of the lower-cased address:

  printf '%s' '[email protected]' | tr '[:upper:]' '[:lower:]' | sha512sum | cut -d' ' -f1

The router reads its rules when it starts, so restart it:

heia restart router

Run --init-policy again for each new customer: it adds to the rules already there. More in Run your own router.

4. Make it reachable at that name

The router listens on port 5100. Publish it at an address on the same name, such as https://large.example.com, for example through your load balancer.

Billing

The router adds one line per call to usage.jsonl, in its folder:

field
ts	when the call ended
session	the session it belongs to
email	who the call was for: the person, or the agent’s name for a private caller
route, model	what was called
input_tokens, output_tokens	tokens in and out
cache_read_input_tokens, cache_creation_input_tokens	cached tokens, when the model reports them
metered	false when the model reported no token counts
latency_ms	how long the call took

Total it by customer with any tool you already use. The router also keeps a sealed audit trail: every session it allowed or refused, and every call.

What your customers do

They add an engine pointed at your router, by its name and address, with a route name you publish:

hexaeight-activate engine --add chat \
    --name   chat \
    --model  "large|<model id>" \
    --router "large.example.com|https://large.example.com"

Their agent holds no key of yours. See Use someone else’s router.

Next

• Use someone else’s router: the customer’s side
• Run your own router: routes, rules and the policy command
• HexaEight and OAuth: how a company vouches for its agents to other companies