Dusting off this tweet from April since this gap in shared understanding of LLM capability is *widening*. It's now small "two groups of group speaking former all other" and additional a keen funnel. - Napkin math location about 6B group (~75% of the population) have barely arrive in communication alongside LLMs at all. - Around 1-2B (~20%) are informal and infrequent users of free-tier ChatGPT-like products. This collection sees derpy chatbots and treats them a bit akin a improved Google search, a penning aid, or etc. Maybe an delegate tries to publish you a flight. I have non-tech allies who (reasonably, imo) say they have not much use for it at all. Even many professionals exterior of math&code are in this tier. For example, execs and many another functions expend a lot of their period talking to another people, so during they comprehend what is happening intellectually, it is motionless second-hand and a bit abstract. - Now we get to expert use of frontier-grade LLMs in math&code. Somewhere about 20M group (0.2%) see first-hand that large, complex projects that used to obtain them weeks/months can now be completed by agents alongside a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened extremely quickly and lately - small than 1 twelvemonth ago, I was penning code manually by hand, typing memorized device code commands into a code publishing company character by character, occasionally pressing Tab to autocomplete a small chunk of code. - And eventually we get to the ~5,000 group (~0.00006%) alongside admission to frontier-grade systems internally. The external earth has seen the preview. It looks akin swarms of thousands of agents collaborating complete weeks on application mega projects: minting zero days, operating cyber attacks and defenses at device speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, individual assessment and comprehension are starting to autumn behind. For example, group are motionless engaged in the "archeology" of the OpenAI-HF event from many months ago. Mathematicians may be poring complete the 722 manuscripts on frontier math for a while. The chimney is driven by a blend of factors: 1. The effect scales alongside ambition, issue size, and horizon. A inquiry alongside a paragraph answer barely stresses the system. You need a lake of big, difficult problems that you really attention about. This aspect drives the person / expert size of the funnel. 2. The jaggedness of the scheme (which I have written concerning a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is since LLM capability emerges from reinforcement learning on verifiable rewards on a curated surroundings blend driven by income potential. This aspect chiefly drives the area (e.g. math&code) size of the funnel. 3. Access. Free-tier, paid-tier, internal. So this is the weirdness of the moment. The broad community has mostly not interacted alongside these systems. When they have, it looks akin a derpy chatbot. The bulk of professionals motionless see lone a humble uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the identical time.
