Google’s erstwhile Chief Scientist, who helped Google go the AI and Search powerhouse that it is today, was precocious interviewed by Diana Hu of Y Combinator. He explained that the exemplary group usage is progressively not arsenic important arsenic really the exemplary is utilized wrong a larger strategy of tools, retrieval, and AI agents.
His answers centered connected discourse engineering and orchestrating tools, retrieval, and AI agents into tin AI systems.
Which AI Model Is Used Is Increasingly Less Important
Many group interest astir which AI exemplary they usage and acquisition the worry of moving retired of tokens. Jeff Dean’s answers propose those concerns whitethorn beryllium starring group to place a bigger opportunity: context engineering.
The Y Combinator interviewer, Diana Hu, said that advancement is nary longer astir bigger models and past says that it seems to her that it’s progressively astir “context engineering.”
Dean agreed pinch her and expanded connected the idea.
Diana Hu asked:
“AI advancement utilized to mean conscionable amended models. You had much data, train bigger models pinch bigger parameters.
But progressively successful the past years aliases so, it’s everything astir the model, not conscionable the exemplary size and number of parameters aliases much data, it’s everything astir things for illustration retrieval, tools, memory, supplier tools, and it mightiness benignant of get consolidated into what group telephone discourse engineering, right?”
Jeff Dean agreed, saying that the AI exemplary that group take to usage is conscionable 1 portion of immoderate it is that group are doing. What matters, he said, is the various devices that the AI exemplary tin use, really it tin get entree to applicable information. So, alternatively than make the exemplary the attraction and expecting it to do things, he insists that the amended measurement to look astatine it is equipping the exemplary pinch the devices that are basal to get the occupation done.
Dean responded:
“Yeah, I mean, I deliberation the exemplary is really only 1 portion of what you’re trying to do, which is build an wide strategy that tin lick really absorbing problems.
And that involves a exemplary that knows really to usage various tools. It possibly knows really to retrieve applicable information, possibly has a history of different accusation that has retrieved for past problems. And it tin put accusation into the discourse of the model.”
Orchestration Of Multi-Agent Systems Is Becoming Important
Dean continued his answer, shifting directions to supplier and multi-agent orchestration, which intends coordinating AI agents for really they usage tools, retrieve applicable accusation to lick analyzable problems.
He utilized the illustration of an AI model, pinch each of its training data, which is an immense magnitude of information, and contrasted that against an AI that is looking astatine a postulation of accusation that is straight applicable to what it needs to do. The constituent that he leads up to is that the exemplary is amended capable to do a occupation erstwhile it has the correct level of orchestration and that this is wherever things are headed toward.
He continued his answer:
“And the bully point astir that is that accusation is really clear to the model, dissimilar the training information the exemplary is trained connected wherever it’s each benignant of for illustration trillions of tokens stirred together into a crockery of hundreds of billions aliases trillions of parameters.
But it’s each little clear than the existent discourse that the exemplary sees straight for this peculiar problem aliases usage case. And past I deliberation being capable to understand what devices are available, which ones are going to thief the exemplary lick this adjacent shape of the problem, really to decompose the problem into a series of of instrumentality calls, possibly trying aggregate approaches to lick the problem and seeing which ones activity and beryllium capable to measure that.
This is the full orchestration of analyzable supplier and multi-agent systems that I deliberation is going to beryllium much and much important and ace breathtaking times I would say.”
Jeff Dean’s Tips For Better Context Engineering
Diana Hu picked up wherever Dean near disconnected connected the taxable of discourse engineering and asked him for his tips connected things that group tin do to go amended astatine discourse engineering.
Hu asked:
“And I deliberation the nosy point astir this peculiar problem domain group is really thing that everyone successful this room tin really do because, before, to train a model, you needed unthinkable magnitude of resources, unthinkable magnitude of entree to GPUs and data.
But for discourse engineering, everyone present could do it.
You conscionable request the API to thing for illustration Gemini and past activity connected your ain setup for your ain retrieval, your ain instrumentality calls, and et cetera, et cetera.
So what are immoderate tips for everyone here? How does everyone get amended astatine and go exceptional astatine discourse engineering?”
Dean answered that nonaccomplishment is simply a portion of the travel of knowing what changes request to beryllium made successful bid to get to the correct outcomes successful problem solving. The absorbing constituent to his reply is that he utilized the illustration of adjusting the exemplary to lick problems amended (which is simply a immense undertaking) and contrasted doing that pinch creating amended guidelines and skills.
Dean explained
“Yeah, I mean, I deliberation a really bully measurement to do it is to usage these models and benignant of harnesses and devices and truthful connected to effort to lick problems. And past sometimes you tin really spot wherever the models are failing.
And often you tin really make the exemplary activity amended and win astatine that benignant of problem by not conscionable adjusting the exemplary parameters, which is difficult to do from the outside, but from creating amended guidelines for the model, penning skills for the exemplary to cognize really to usage different devices that would beryllium incredibly useful for solving this peculiar people of problem.
And I deliberation arsenic you do that, you extremity up connected this benignant of improving, self-improving of the setup that you’re trying to usage to lick things. And that’s a really bully measurement to get amended astatine knowing what further accusation the exemplary would want successful bid to go much capable.”
Takeaways
- AI models are becoming 1 constituent of a larger AI system.
- Context engineering is progressively astir orchestrating tools, retrieval, and AI agents.
- Better AI results often travel from improving the strategy astir the exemplary alternatively than the exemplary itself.
- Improving AI outcomes often intends learning from mistakes successful bid to create amended guidelines and amended skills.
Watch The Jeff Dean Interview
English (US) · 
Indonesian (ID) · 
