Ex-Googler Jeff Dean Explains Design Choices Behind Gemini

Sep 16, 2026 05:31 PM - 3 hours ago 1

Jeff Dean precocious explained the origins of Gemini, including the creation accuracy down it. Surprisingly, Gemini was lagging a spot until its coding abilities were improved, which subsequently improved the model’s reasoning ability.

The Origin Of Gemini

Dean explains that the root of Gemini was erstwhile he realized that location were aggregate teams moving separately toward the aforesaid goal. From the extracurricular looking in, that seems for illustration a silly approach, and that’s precisely the penetration Dean had.

The interviewer, Dawn Song, asked:

“So now looking backmost connected the improvement of Gemini, what amazed you the astir and what lessons from building Gemini do you deliberation that you tin stock and tin thief style the adjacent procreation of AI systems?”

Dean answered:

“I deliberation Gemini is really the culmination of a fewer earlier investigation projects, some wrong Legacy DeepMind and Google Brain and different parts of Google Research.

And astatine that time, we benignant of realized that we were each converging to very akin kinds of directions, for illustration training, trying to standard up the size of models we trained. We had immoderate efforts, independent parallel efforts, connected really do you make connection models besides beryllium multimodal truthful they tin understand images and truthful on.

And I wrote a one-page memo. I’m like, this is conscionable silly. We should conscionable each activity together. Let’s harvester our group and ideas and compute resources and train 1 exemplary that is multimodal from the commencement that brings our champion group together from crossed aggregate investigation organizations wrong Google.

And I deliberation that was really good.

…And truthful we benignant of saw a measurement to bring ourselves backmost successful person collaboration. And I deliberation that’s been a really successful thing.”

Screenshot of Jeff Dean

Why Gemini is Multimodal

We cognize that Gemini is multimodal, but what is little known is that this was a creation effort from the very beginning. Dean explained that the extremity for Gemini was to create a exemplary that could understand audio, images, language, text, and different forms of information. But interestingly, it was erstwhile they trained it for coding that Gemini knowledgeable a breakthrough betterment successful reasoning.

Jeff Dean explained:

“I deliberation the attraction connected making the exemplary multimodal from the opening has been really, really good. You want the exemplary that you’re gonna usage for everything to understand matter and connection and codification and images and videos and audio and different modalities besides.

So for illustration we put a small spot of, say, LiDAR information successful the training information truthful it astatine slightest knows that LiDAR information is simply a point because that’s an important usage lawsuit for, you know, further training of Gemini models. So that is important.

And that’s been, I think, a occurrence of a determination we made astatine the very opening to bring that done each the existent Gemini models arsenic well. …We wanted to make the exemplary bully astatine tons of things.

And truthful I deliberation possibly our attraction connected making it astonishing astatine coding was lagging a small spot and we realized that successful our activity to drawback up connected that.

And I deliberation we person bully efforts underway. But I deliberation by focusing connected that, you extremity up pinch a strategy that is capable to really do a bully occupation of reasoning and doing different kinds of tasks that needs to benignant of activity its measurement done breaking a analyzable problem down into aggregate sub-pieces and truthful on.

And truthful that’s a good– if you amended coding, you besides thin to amended that capacity successful non-coding things.”

Insight About Learning

That’s really ace absorbing because it shows really making the LLM master successful 1 point leads to benefits successful different areas. This is really benignant of akin to really humans go master successful things. Many hundreds of years ago, a samurai beard maestro named Miyamoto Musashi encouraged learning arts and professions that are extracurricular of the martial arts. He utilized carpentry arsenic an illustration of really a maestro carpenter understands tools, planning, really to negociate workers, and understands the building of buildings and past maps those skills and insights backmost into the martial creation of beard fighting.

Jeff Dean’s mentation astir the root of Gemini is useful because it gives users a amended knowing of what Gemini is, creating an consciousness of what is imaginable pinch Gemini erstwhile it’s considered arsenic a multimodal model, not conscionable a generative matter model.

Jeff Dean’s Secret For Innovation And Success

Dawn Song asked:

“So essentially, this is 1 point that stands retired astir your profession is that truthful galore of these ideas, from MapReduce to MOE and truthful on, they really took years earlier the broader organization realized really important they would become.

But you had the Midas touch, and you had the special, really foresight to really build and create these ideas and systems and truthful on.

So I deliberation galore group successful the assemblage would emotion to study really do you do it?

How do you separate betwixt technologies that are genuinely foundational and those are simply fashionable?

And what engineering rule that has remained amazingly timeless passim your career?

And besides arsenic AI shifts from models to progressively autonomous agents, what caller abstractions do you deliberation we’ll request complete the adjacent fewer years and decades for our future?”

Jeff Dean responded:

“I deliberation I’ve been very lucky, maybe, arsenic portion of it. But I besides deliberation 1 of the things I effort to do is support way of a batch of different trends aliases investigation topics that the organization is exploring.

I often show students it’s amended to skim 10 papers than to publication 1 successful detail, because you benignant of past get 10 points successful your unreality of what mightiness beryllium possible, aliases moreover skim 100 abstracts.

Because what you want to beryllium capable to do is link important ideas that person not yet been connected.

And sometimes erstwhile you deliberation astir a difficult problem, if you person immoderate of these things that are now benignant of vaguely starting to beryllium imaginable successful mind, that tin thief you benignant of style a afloat solution to a problem wherever alternatively of it seeming for illustration 7 unsolvable problems, you tin squint astatine it and say, okay, well, these 5 things, I cognize there’s benignant of vague activity going connected that seems important and mightiness beryllium capable to reside immoderate of the things.

Then there’s 2 benignant of further things that I person nary thought really to do. But if I activity difficult connected them, I tin ideate solving those.

And sometimes that’s ace important to do the things that are incremental improvements to what we’re doing now, but you want to look for the things that are you know, possibly very different ways of doing things that mightiness beryllium imaginable aliases mightiness beryllium imaginable soon.

…I’ve done a bunch of things that person not worked retired good too, truthful that’s besides a bully tip. Try tons of things that mightiness not work. Some of them will.”