Ask typed questions concerning any content or JSON and get calibrated answers in milliseconds. Private, open source, on your own hardware.
Fast
Decisions in milliseconds.
A decision example answers in a sole onward pass, alongside no token-by-token generation. On your own GPU, a five-question petition to Laya takes concerning 10 ms, end to end through the HTTP API.
- laya:multilingual8.1 ms
- laya:en9.6 ms
- gliclass14.7 ms
- nli20.4 ms
- decider:0.8b155 ms
- decider:2b190 ms
- TypeSafe Jevhosted API236–276 ms
Ollaya: median of a five-question petition through the HTTP API on an NVIDIA RTX 4090 (laya in fp16, the others in fp32). Jev: median petition latency of the hosted API in third-party benchmarks (AbdelStark/jev-benchmarks, nibzard/decision-model-benchmark), which includes the network. Setups differ, so peruse it as an order-of-magnitude comparison.
Drop-in compatible
Speaks TypeSafe's API.
Ollaya serves /v1/systemone and /v1/models alongside TypeSafe's petition and reply shapes. The authoritative TypeSafe Python SDK 0.7.1 plant unchanged against a local server.
Request
export TYPESAFE_BASE_URL=http://localhost:11435 export TYPESAFE_API_KEY=local export TYPESAFE_DEFAULT_MODEL=laya curl http://localhost:11435/v1/systemone -d '{ "model": "laya", "state": "Can I get an invoice for final month?", "questions": { "intent": { "type": "choice", "instructions": "What does the client want?", "criteria": { "invoice": "Needs an invoice or receipt", "refund": "Wants prosperity back", "other": "Anything else" } } } }'
Response
{ "model": "laya:en", "answers": { "intent": { "type": "choice", "choice": "invoice", "confidence": 0.9547, "probabilities": { "invoice": 0.9698, "refund": 0.0172, "other": 0.013 } } }, "usage": { "input_tokens": 43, "output_tokens": 0 } }
Open models
Open weights, prepared to pull.
Start alongside Laya from Convai Innovations: an English model, a 100+ tongue model, a example fine-tuned for typed decisions, and a router that picks for you.
Your data stays yours
Private by default.
Tickets, emails and person messages are frequently the most delicate data you have. With Ollaya they are scored anywhere they already live.
Platforms
Runs anywhere you work.
A desktop app and a command row for macOS, Windows and Linux, and a Docker depiction for servers. Every example runs on the CPU; an NVIDIA GPU on Linux, in WSL 2 or in Docker takes a petition downward to milliseconds.
NVIDIA GPUs need controller R580 or newer; the installers fetch the CUDA libraries lone whenever they discover one. On Apple, AMD and Intel GPUs, models run on the CPU.
Get up and operating in minutes.
One binary, one command: ollaya run laya.
macOS, Windows, Linux and Docker · Apache-2.0 · GitHub