LlamaIndex

Verified 2026-08-19 — PASS, 7s. Session persistence: Context serialize / restore.

The workflow Context is the session — serialise it to JSON on the way out, rebuild it on the way in.

---

The runner

import sys, os, json, pathlib, asyncio
sys.path.insert(0, os.path.expanduser("~/agenttests"))
import azcfg, hetool
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.workflow import Context, JsonSerializer
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai_like import OpenAILike

STATE = pathlib.Path(os.path.expanduser("~/agenttests/llamaindex/state"))
STATE.mkdir(parents=True, exist_ok=True)

def get_working_folder() -> str:
    """Return the agent's current working folder (absolute path)."""
    return hetool.get_working_folder()

def run_shell(command: str) -> str:
    """Run a shell command in the working folder and return its output."""
    return hetool.run_shell(command)

TOOLS = [FunctionTool.from_defaults(fn=f) for f in (get_working_folder, run_shell)]

async def invoke(sid, message):
    c = azcfg.load()
    llm = OpenAILike(model=c["model"], api_base=c["base_url"], api_key=c["api_key"],
                     context_window=131072, is_chat_model=True, is_function_calling_model=True)
    agent = FunctionAgent(tools=TOOLS, llm=llm,
                          system_prompt="Be concise. Use tools for operational questions.")
    f = STATE / f"{sid}.json"
    ctx = (Context.from_dict(agent, json.loads(f.read_text()), serializer=JsonSerializer())
           if f.exists() else Context(agent))
    resp = await agent.run(message, ctx=ctx)
    f.write_text(json.dumps(ctx.to_dict(serializer=JsonSerializer())))
    return str(resp).strip()

---

Engine definition

{
  "type": "process",
  "file": "/home/you/llamaindex/.venv/bin/python3",
  "nativeSession": true,
  "argsNew":    ["/home/you/he_adapter.py", "/home/you/llamaindex_session.py", "",            "{message}"],
  "argsResume": ["/home/you/he_adapter.py", "/home/you/llamaindex_session.py", "{sessionId}", "{message}"],
  "replyField": "result",
  "sessionIdField": "session_id",
  "approve": false,
  "workdir": "/home/you/work",
  "proxy": { "shape": "openai", "baseUrlEnv": "OPENAI_BASE_URL", "keyEnv": "OPENAI_API_KEY",
             "modelEnv": "OPENAI_MODEL", "model": "<route>|<provider model>" }
}

---

Adoption

1. Keep your agent and tools as they are.
2. Serialise the Context keyed on sid; rebuild with Context.from_dict when the file exists.
3. Use OpenAILike — not OpenAI — built from azcfg.load().
4. Seal the engine, point external.engine at it, restart.

---

Gotchas

Use OpenAILike, not OpenAI. The OpenAI class validates the model id against OpenAI’s known models and rejects anything else. OpenAILike is the class for OpenAI-compatible endpoints, which is what the HBIA route is.

is_function_calling_model=True or tools are silently ignored. LlamaIndex cannot infer this for an unknown model, and the default assumes no tool support — the agent then answers without ever calling a tool, which reads like a model problem rather than a config one.

is_chat_model=True likewise, or the request goes to the completions shape.

Set context_window explicitly. There is no way to infer it for an unknown model, and the default is small enough to truncate long sessions.

api_base, not base_url. LlamaIndex’s keyword differs from most other SDKs.