Hi HN! I'm Bek, laminitis of Speko, a level that finds an optimal operation of speech-to-text, LLM, and text-to-speech models, fixed your constraints, among each our nationalist benchmarked options, and tells you why.
Demo: https://youtu.be/no2LY2gRh-c
Typical accumulation sound supplier is an ensemble of 3 models: STT, an LLM, and TTS.
Each of those layers offers a twelve reliable vendors, and each period location are caller models connected the market. Almost everyone evaluates once, picks a stack of their choice, and ne'er rechecks because switching from a vendor to different involves yet different integration and arguments astir the numbers.
The consequence is that you usage sound agents moving past quarter's models while amended and cheaper options are available.
Before founding Speko, I spent 4 years arsenic cofounder and CTO building sound agents for enterprises crossed Asia successful 10+ languages. Each clip a caller reside exemplary would arrive, we repeated the aforesaid ritual: prosecute native-speaking raters, benchmark it against our existing stack, and update accumulation if it improved. Speko turns this process into an API. A squad moving thousands of calls a time told us: "we tin virtually spell to this dashboard, move the model, and it will do it for us."
How it works: you nonstop a petition pinch your optimization criteria (accuracy, latency, costs aliases balanced), connection and region. The router filters to models which we measured for the fixed operation of constraints, benchmarks them, selects the winner, and returns a consequence pinch headers containing provider, exemplary names, and the scores. The gateway prefetches signed convention plans, truthful a caller convention dials the supplier consecutive from memory; nary control-plane information travel while a caller waits.
Failover happens only during relationship setup stage: if the supplier refuses the relationship attempt, we commencement connecting to the runners-up.
Some of the customer stories: 1 laminitis came to america not knowing what to prime astatine all: he gave america his usage lawsuit and now routes everything done the platform. A spot guidance AI runs LiveKit successful Python and had not updated STT aliases TTS since launch: they did not cognize their STT had precocious correction rates connected their calls, amended options existed, and swapping ever looked for illustration an R&D project. One squad did not cognize which models to prime for Spanish. A aesculapian squad did not cognize which STT handles aesculapian vocabulary best. In each lawsuit we helped find the correct stack from the benchmarks, and now they way done us.
The measuring portion is public: we walk the aforesaid inputs to each exemplary successful 1 region successful different dated runs and we people the boards, including those wherever our selections execute worse than alternatives. A motorboat demo answers which 30-second clip sounds better; accumulation asks which exemplary survives infinitesimal eight, truthful we trial spontaneous speech, money and dates, ten-minute takes, and the rankings change. We trained an automatic scorer for TTS naturalness connected our unsighted head-to-head listening votes; connected providers it has ne'er seen a ballot for, it picks the aforesaid victor our raters do astir arsenic often arsenic raters work together pinch each other.
We don't train aliases waste models ourselves, that's precisely really we support our rankings impartial.
We besides unfastened originated the gateway for teams who want to debar an other web hop connected the audio way and don't want to stock keys pinch our unreality (https://github.com/SpekoAI/gateway, MIT): 1 Go binary, which is moving arsenic a sidecar successful your agent's container, speaks 1 section protocol complete Unix socket, pins supplier hosts and attaches your keys. In BYOK mode it doesn't pass pinch america astatine all.
Notice that the anonymous, content-free telemetry is enabled by default, and 1 env var disables it.
Cost: the gateway and BYOK setup will beryllium free forever, we complaint for the hosted router and managed keys pinch consolidated billing. Since we started the batch successful precocious June, outer usage has grown astir 25 percent per week connected average, front-loaded toward the motorboat weeks.
I would emotion feedback from the community: really do you prime reside models now, and what makes you spot the third-party benchmark?
English (US) · 
Indonesian (ID) · 
