For the past fewer weeks, I've been playing pinch gpt-5.6-luna. It is shockingly capable, fast, and smart. I regularly spot it do ~100 tps, and rip astir my codebase, email, and knowledge base.
Of course, the biggest point pinch luna is the cost. I've tried moving immoderate reasonably analyzable investigation threads, and it's beautiful reliable to tally up a ample bill. Even having it hunt crossed thousands of emails, I extremity up pinch an API costs successful the tens of cents.
With GLM 5.3, we moreover person a caller action astatine the Pareto frontier.
When doing coding work, I almost ever scope for the astir costly and tin models (Fable 5, 5.6 Sol). So it's been easy to miss the advancement the mini accelerated models person made.
One point a fewer investors I've talked pinch person mentioned: "It's weird we're not seeing much user AI companies. Why is that?"
There's a straightforward answer: token costs.
In the times earlier AI, the playbook for large user apps looked for illustration this...
- create immoderate benignant of compelling website which is reasonably inexpensive to run
- attract a bunch of users (typically pinch immoderate virality)
- raise money, standard to much users
- create an ads marketplace
This astir describes astir of the large user companies (Google, Facebook, Snapchat, etc.).
But what if you want to adhd AI to your product? Well, now you person immoderate existent conclusion costs connected each request! Suddenly the magnitude of superior required increases dramatically.
A pet eval of excavation is to build a regular news site, personalized to me:
research @calvinfo connected the internet. fig retired what news they mightiness like. build a micro-site pinch today's apical stories, personalized for them. hunt hn, reddit, twitter, etc.
With the erstwhile procreation of models (Sonnet class), you'd walk ~$1 to get anywhere. Charging $30/mo is untenable for a user app. There's evidently a batch we tin optimize here, but if you're charging what the WSJ aliases The Economist charges, you'd amended beryllium delivering akin value.
But looking astatine luna, the results are beautiful decent, and the mean costs is ~$0.10. Now we're talking!
Where I deliberation this gets moreover much absorbing is successful the world of business.
My Segment co-founder Peter and I were precocious comparing notes connected a hike. Across his various startups, Peter has seen 2 kinds of work:
- the "IQ 180" work. immoderate huffy intelligence brilliant type comes up pinch immoderate crazy solution you've ne'er thought of.
- the "token spewer" work. being ultra responsive, pushing the shot guardant crossed dozens of different fronts.
Peter runs aggregate companies. Beyond Segment, he's raised $100m+ for Charm Industrial, and conscionable precocious closed a Series A for Revoy. He's incredibly organized and businesslike pinch his time.
And yet, Peter mentioned that ~95% of the activity he does falls into bucket 2. It's hopping connected calls. Nudging people. Blocking and tackling.
To beryllium clear, Peter says his companies would beryllium dead-in-the-water coming without an IQ 180 method mind solving the heavy problems. Just that astir of his activity falls successful bucket 2.
I deliberation request for "frontier-level" models is going to support compounding. Especially for fields that require caller breakthroughs aliases find (engineering, difficult science, exemplary training).
But I besides deliberation the request for "fast/cheap/good-enough" models is conscionable astir to return off.
Think of the group you interact pinch connected a regular basis: coworkers, vendors, and customers. Nine times retired of ten, you want personification who is ace responsive, and conscionable handles things for you. Most of the "human tokens" astatine companies coming are spent this measurement — hiring skews heavy toward the fast/cheap/good-enough archetype.
There's a batch of activity that needs to hap to make fast/cheap/good-enough models a reality for business. New harnesses, punctual injection safety, roles, and permissions. But I'm assured we'll fig that out.
If you're besides experimenting pinch making mini models useful, please driblet maine a line.
English (US) · 
Indonesian (ID) · 
