When millions of AI agents meet

Auto-generated captions. Click any line or timestamp to seek; the current line highlights as the video plays.

Foundations: agents vs. plain LLMs 0:00

0:00Welcome back to Google Deep Mind the podcast.

0:02Now, not very long ago, an AI assistant essentially meant a large language model.

0:07You asked it a question, it gave you an answer, but it couldn't go off and perform tasks on your behalf.

0:13All of that is changing with the advent of AI agents.

0:17While Google Deep Mind has this long history of developing agents, stretching back to reinforcement learning in games, for most of us,

0:26they hadn't really arrived. And then we saw open-source tools like OpenClaw released into the wild.

0:33And at Google, a new generation of agentic tools is here, including Gemini Spark and anti-gravity.

0:40But what happens when millions of AI agents are not just working for us, but transacting, negotiating, delegating to each other?

0:49Do we end up with a new kind of economy, a new route to AGI?

0:54And how on earth do we keep all of that safe?

0:57Well, one of the people trying to answer these questions is Nanad Tamashv, senior staff research scientist at Google Deepmind.

1:04Nad, thank you so much for joining me.

1:05>> Very happy to be here.

1:07>> I think we should probably start at the beginning here because for people who have only played around with large language

1:11models. Could you describe to us the difference between that experience and and acting with an agent?

1:18>> Yeah. No, definitely. I think this is becoming one of the main trends we're seeing this year.

1:23And it's interesting because agents are not a new concept.

1:26It's something that we've been looking at in the context of AI for a long time.

1:29Even before large language models, we had agents operating in simulated 3D environments um going on collecting items, completing some tasks.

1:38This was back in the days we were really prioritizing actioning in the world as a way of manifesting intelligence.

1:44Now similarly nowadays I guess you could say that the main conceptual difference between just a language model and an agent is

1:51that an agent observes a state of the world and performs an action makes an action in the world in the environment

1:59that it's given whereas a language model just gives you continuation reply to a prompt to query.

2:05Now obviously agents that we use nowadays they use large language models under the hood.

2:09So the two concepts are not completely disambiguated.

2:12It still is the large language models formulating the actions.

2:16It's just that there is a harness around it made to enact the changes once they have been proposed.

2:22>> But it has a lot more autonomy to to chain decisions together.

2:25I guess >> correct. And I guess this is ultimately the motivation, right?

2:29Because you could do everything most things that an agent can do manually, painstakingly by interacting with the language model very many

2:39times and you guiding the whole process.

2:42Whereas an agent instantiates this harness that automates some of that away and gives you less work and gives the language model

2:51or you know the agent more autonomy to complete tasks.

2:54So if you want something done that takes multiple steps that agent can make a plan and take actions on all of

3:00those steps obviously requiring approval or human input for those actions that are you know let's say more sensitive or more likely

3:08to go wrong. >> How is it different though?

3:10I mean if you're used to interacting with a large language model by now what would it be like interacting with an

3:16agent? >> In many ways similar your interaction interface is somewhat similar.

3:21You're still talking to the agent in a way in which you'll be talking to a language model.

3:25There is a language model there.

3:27But because the agent is doing more things for you, you're more in a position of a decision maker to review and

3:34approve. And then once you've approved, the agent is going to do various things, purchase tickets, message your friends if you're organizing

3:42a party, and meanwhile, you can uh put something on on Netflix hopefully and relax a little bit.

3:48The example I was thinking of was if you were, I don't know, planning a wedding for instance, you go into a

3:52large language model and it would like tell you a list of caterers, give you a suggested list of venues, but actually

3:58you would have to do all the emailing yourself.

4:01But but an agent, I mean, it would be much more useful really in that kind of scenario.

4:06>> 100%. Especially because agents are given access to all of these tools.

4:09So you could, you don't have to.

4:11You could give an agent access to your Gmail and give it permissions to send out an email.

4:16Of course there is a chance of it sending something wrong.

4:19So you need to verify what it has composed.

4:22But in principle by giving access to tools to agents uh you just empower your large language model to do these things

4:30for >> and then the whole job is done.

4:31The organization has happened without you having to lift a finger >> ideally presuming no mistakes have been made but yes uh

4:37>> yeah ideally is quite an important point there.

4:40So okay where we are right now what tasks are agents actually good at?

Applications: agents in science & exploration 4:44

4:44I think that where we are focusing a lot of our energy on and by we I don't mean we as Google

4:49we as the the entire field is on coding capabilities of agents and this is just because so many formal processes and

4:56tasks can be formulated as software or as code in terms of where they're currently at in the real world.

5:04Speaking of coding, we see lots of coding tools get used.

5:08We use them here internally. People use them externally.

5:10And it's really accelerating the development of software which is bringing the human uh focus onto ideas and the design rather than

5:18the painstaking implementation of boilerplate around them which used to take a lot of time and a lot of skill and very

5:24bespoke knowledge and now that can just be done by by language models >> but then at the same time we are

5:29still at a stage where you have to keep a human in the loop throughout this.

5:33I mean why what can't these things do at the moment that means that it requires human oversight?

5:39I wouldn't even make a distinction between whether they can or cannot.

5:44It's more that every single thing that they can do u they don't do uh with 100% accuracy.

5:52So every action like with humans at the end of the day has a certain failure rate and the more complex the

6:00action the higher the expected failure rate again like with any form of intelligence human one included.

6:05So uh while you know you may expect that an agent will execute the task correctly, it may still make a mistake

6:15and this mistake may be obvious or it may be very subtle.

6:19uh which is actually an important point because there is this thing that has existed in other domains as well for a

6:25long time where different machine learning models have been uh deployed and that is automation bias where in this context if you're

6:35using an agent it does well it builds one thing well the second thing well eventually you switch off you start trusting

6:42it too much right >> and you fail to verify and you fail to find some important issue underneath >> then mistakes

6:48slip Dream. >> Exactly. So for humans, it's important not only to be in the loop because we are obviously designing these

6:54harnesses to keep humans in the loop, >> but to really be engaged and be switched on because as soon as you

6:59switch off, you're rolling the dice.

7:01>> So okay, in the long term then I mean it sort of sounds like we're in this transition period where these

7:06things are sort of becoming more capable.

7:08But in the long term, I mean, how much of a difference do you think that this is going to make?

7:12I mean, will this completely transform the way that we use artificial intelligence?

7:17100% I think it's impossible to envision a world where there isn't some kind of a deep disruption and what we all

7:23trying to figure out is exactly what that is going to look like obviously we have agency in that we are building

7:29the technology we can design our solutions in a particular way obviously to empower human developers and human experts across different fields

7:36as much as possible but AI is definitely entering various fields where it just wasn't present before scientists are using AI on

7:44a regular basis up until very recently you know mathematicians couldn't envision AI doing something in mathematics um now it is becoming

7:53common place in a very short span of time which is not to say that all of the problems have been solved

7:58obviously there is still a big role for humans but it's a very rapid transition and that is the only unsettling part

8:04I guess because for most even industrial revolutions and so on we're used to them taking some period of time giving us

8:12more time to uh change our approach and settle into it as you say and it doesn't feel like the window of

8:19time is as long this time.

8:21So we need to be very mindful of how we approach everything.

8:24>> Why do we want these things?

8:26I mean why build them? What what what's the benefit?

8:29What are they giving us that we don't currently have?

8:31>> I mean for all of us who have been working on air for a long time we've had some version of

8:36the answer to that question I guess u internalized.

8:39And for me personally, the answer is to advance science, improve health and human welfare.

8:46Now, these are very high level answers.

8:48So, um it's maybe not as obvious as to how they map on to the specifics of the question as to why

8:54build agents and have agents. And there are people in the field that um say specifically that we shouldn't be the systems

9:03autonomy, right? Which is what agents have.

9:06But in my mind, if we can develop these harnesses and make them safe and have agents perform complex tasks autonomously, then

9:16we actually accelerate progress because then more things can happen with um the same amount of human input.

9:24just draw the line for me to science here because I guess the examples that we've been talking about have been like

9:31you know building software, buying stuff for a wedding and they all sort of feel quite trivial but but just explain to

9:37me how this fits into the story of improving science.

9:41>> Yeah. So this is my you know main dream main objective here.

9:44When it comes to science it's not merely about having some good ideas and reasoning about them um for some short period

9:53of time like in a context window of a model.

9:55Lots of people are obviously using language models in science as collidiators or to help with some formal derivations.

10:03All of this is already useful and and actually amazing that it is possible.

10:06But when it comes to automating science um to some larger extent um there are other threads that are currently progressing at

10:16some pace like there are investments in the development of some autonomous research laboratories for example and under those scenarios you would

10:22want to see agents be able to schedule experiments to run.

10:27Um needless to say lots of safeguards need to exist when such an interface with the real world is happening whether we're

10:33talking about material design or um biotech um because even with let's say you're designing batteries I mean maybe you uh come

10:42up with a setup that uh overheats leads to some sort of an experimental breakdown that would have that would damage the

10:48hardware have some consequences. So we need to have safeguards in place and we need to have good reliable protocols in place

10:55for these agents to close the loop because in the software closing the loop is as mentioned easy.

11:00You write tests and you you verify through tests and then you can proceed.

11:05In science you need to run physical experiments in most areas of science to give you this feedback whether your idea was

11:12good or not. Observe that analyze that and so on and so forth.

11:16Because I guess this is the point, right?

11:17If the algorithm, if the agent has autonomy to go and test out different mathematical problems, for instance, rather than just waiting

11:24to be prompted by a human, I mean that does then raise the question of of where is the role for the

11:29human in all of this. >> Indeed, I in the long run, we need to figure that out.

11:33>> I would say that in the short term with the technology that we have, there is still obviously a major role

11:38for humans and our systems, they're not yet AGI, there are many things they still can't do.

11:43Um and I think for the current generation of systems, one thing that can be said with some confidence is that they

11:49tend to be good at how how to put it best let's say a kind of a combinatorial closure what we already

11:55know how to do. They are at the end of the day mostly trained on human data.

12:00Therefore they can replicate the skills we have and repeat them and combine them and find ways of bridging some smaller gaps.

12:08But we've not yet seen these models be truly deeply transformative.

12:14Um let's say in terms of science making a discovery that no human would have ever thought of.

12:19Um therefore there is still plenty of role to play for all of us in this transformation.

12:24>> You mentioned a moment ago that that people have been talking about agents for a really long time.

12:28Why has it taken so long for them to come into fruition?

12:31I mean really it's only very very very recently that people have actually been able to get their hands on them and

12:35play around with them. Yeah, I would say um obviously some things that we would refer to as agents historically have been

12:43deployed for example in optimizing operations in data centers and so on and so forth.

12:47They have obviously been very limited because they didn't um include language.

12:53So there was no way for humans to interface with them to communicate.

12:56It would be a very narrow agent to train on a specific task and it would be good at doing that task.

13:03But because there is no interactivity, there is nothing for us to do.

13:06It's just software in a classical sense.

13:09Uh maybe you can call some of the trading algorithms um and investment algorithms also agents in that context.

13:15But they just operate on their own.

13:18The difference now is because these agents are based on language models is we can talk to them, we can learn from

13:23them, we can influence them, we can steer them.

13:26And this is why all of us as people are interacting with agents much more.

13:31>> But then why are we still waiting?

13:33I mean this sort of vision that you're describing of like an assistant that can just go off and do everything for

13:38me. We it's still not here.

13:40What what's stopping it being deployed more more >> We need to take a step away from just designing the underlying model.

13:46A lot of energy has gone into that and there are still improvements needing to be made.

13:51But now that we have capable agents, capable models, we need to find better ways of coordinating them, orchestrating them, managing them.

13:59Once you have these admittedly quite powerful systems that can do many things for us, we need to see ourselves as managers

14:07of teams and institutions in some way and to develop personal management skills to handle these workflows.

14:14Managing a team of agents is different compared to managing a team of humans, but they're obviously commonalities, right?

14:20Different in a sense that agents will make a very nonhuman mistakes.

14:24They're not a human intelligence. Um, but at the same time, an agent doesn't know you that deeply to be able to

14:30just go on and uh accurately guess everything you would want it to do.

14:34You still need to be involved and therefore we need to get better at orchestration.

14:39I think >> the thing is we're we're still in a world where large language models occasionally hallucinate.

14:45So it is in some ways quite a big leap for humans to to then trust agents to carry out tasks on

14:52their behalf when any hallucination might actually result in something catastrophic >> Trust is given but it's also earned.

15:02I think this is maybe an important distinction.

15:04So in our frameworks we uh mention the need for establishing let's say tracking of reputation over time where if an agent

15:15is repeatedly unreliable it should obviously not be trusted even if it's mostly reliable it shouldn't be blindly trusted.

15:23We should still verify its actions but language models will always hallucinate to some extent.

15:29So we just need to integrate them in our workflows in a way which recognizes that and where we make sure that

15:35those hallucinations and they're becoming more and more rare and hopefully will uh continue to do so don't compromise the workflows that

15:44are being undertaken. >> I know one of the things you've written a lot about is is the idea of delegation that

Agent ↔ agent interaction: delegation & negotiation 15:46

15:50you might uh have a particular task and an agent might then go on to delegate it to a specialist.

15:55Just explain to me how that how that might work.

15:57Yeah. So this is the idea that um one of the bottlenecks one that we haven't mentioned yet is that where we

16:03would really like to get help from agents are very complex tasks.

16:08So what language models and simple agents that many of us have access to can easily do is if you give a

16:14very direct instruction you know go book something for me.

16:19I want to eat at this restaurant tomorrow.

16:22Find a slot and do the booking and the agent can maybe do this via tools.

16:26If however you have a very complex plan that needs to be broken down into pieces executed separately you may be in

16:33a situation where even no individual agent can do each and every piece.

16:38So maybe an agent may need to over this established agentto agent protocol that exists hand off a part of that work

16:46to another agent but then there can be failures along the way.

16:50So an agent that delegates or a human that delegates needs to manage and handle those failures and also preempt them as

16:57much as possible. Preempting them may involve um figuring out which agents are first and foremost reliable to even delegate to in

17:05the first place. What are their capabilities?

17:08Is that something that we can certify?

17:10Um and also to safeguard the users and the agents from any kind of a malicious uh interaction.

17:16you mentioned, I think, was it a a wedding or a party or something at first as an example, right?

17:21So, when managing a big event, some of the bookings fall through, some accidents happen, some things don't arrive on time.

17:29So, whenever you have a big coordination challenge, there are lots of things that go wrong.

17:34And in the process of managing that as a human, >> you need to deal with all of those delays and and

17:39problems. Um and similarly an agent that delegates to a group of agents needs to manage all of the problems that may

17:46arise. So one thing that's currently the case in many of the multi- aent systems that we see is that they act

17:53more as parallelization than delegation where you may have many agents working on things but rather than than there being an intelligent

18:04framework around how the work is split up it should just be chunked into sort of random subp parts that are handed

18:11off. they get done in parallel.

18:13So you get a speed up presuming that all of this is reliable and each agent can complete its task independently.

18:20But this is not the intelligent delegation framework that we talk about.

18:23>> So uh if the tasks are split up in a sort of a essentially random way, you could have one agent

18:30that is buying the wine and another one that is buying glasses and doesn't realize that it's wine glasses that are required.

18:36There's sort of no communication between them.

18:37Is that the kind of potential problem that could arise >> potentially?

18:41But you're also I guess hitting on another point which is that many of the uses we see are uses in against

18:47software engineering for example with agents at the moment and that is a part of the reason because in software when you're

18:53building software you can write tests unit tests as we say right and and run them and verify that the code that

18:59has been written at least in isolation uh performs the function but when it comes to many of these real world tasks

19:06verification is not necessarily as straightforward.

19:08Maybe there is a subjective element involved.

19:11>> How do you define nice tasting wine for >> Maybe a bit of a subjectivity in that.

19:17>> But this is actually quite important when it comes to AI and language models because there is a notion of reward

19:21hacking that has existed in the in various contexts in the field for a while.

19:26So there could be situations where it does something that meets the request but isn't in the spirit of the request technically.

19:33And for that reason, you know, you you really want to emphasize verifiability and to be very formal about the contract that's

19:39made between the delegator and the delegate in that setup.

19:44At the same time, for tasks, we need to recognize that some are completely reversible.

19:49>> So if something goes wrong, there's no harm.

19:51You just rerun the task, retry, redelegate.

19:54some may have consequences in the real world whether it's spending your money to buy something or taking some other action that

20:01you can't easily revoke after the fact.

20:04So for those tasks you want to um you want to put more care in what you do there.

20:09>> We've also seen with some of the early agents that are out there uh agents delegating tasks to humans.

20:16Right. >> Just talk me through some of that.

20:19I mean that's an interesting let's say reversal of the more usual vision that we all have.

20:25So humans delegating tasks to AI you know that's that's quite standard yeah standard >> u but this other direction has been

20:33explored across a number of studies I say and my background uh you know for context is that I've done lots of

20:40prior work in and around medical AI in medicine we've had narrow systems that were at basically superhuman performance for very specific

20:50things that they have been trained to do in medical imaging in radiology this had to do with um a machine learning

20:56model, seeing a scan, identifying where there is a pathology um putting a box around it and handing that off to let's

21:04say a human radiologist to review.

21:06And these systems have been operating at a very high level uh for quite a number of years.

21:11They still have some failures though.

21:13So they need to be reviewed by human experts.

21:15So people have experimented with AI human teams there where the idea is that a human would correct a mistake made by

21:23a system and people have trialled with this uh flowing in both direction right either having a human expert only consult an

21:32AI when a human expert is let's say uncertain or a human expert look at AI's suggestions that help out all the

21:40time or maybe having an AI system do its thing make the prediction and then a flag when something is uncertain, when

21:49maybe there's something blurry, fuzzy in the image that can be interpreted in many ways and the machine learning system isn't sure

21:55which of those is correct. But this human review of decisions made by these uh potentially superhuman narrow machine learning models has

22:04proven to be quite a good uh setup.

22:06So what that AI would defer to a human in case of need in case of uncertainty.

22:12That is interesting though that I mean granted in those very specific scenarios where the AI is superhuman in its abilities that

22:20the best team that you can get is where essentially the AI delegates to the human when it's unsure.

22:27>> That is fascinating in of itself and you know maybe there are use cases where it's the the converse.

22:32Now for these more general systems again if an AI can recognize when it needs approvals and permissions for sensitive actions then

22:41it does make sense to delegate those decisions to humans at the very least.

22:45Right? >> Just looking at the other side of this I also want to think about the sort of cyber security element

Security: cyber threats, cloaking, agentic traps 22:46

22:49of this because as more and more agents are out there interacting in the world on the internet and so on there

22:57are inevitably going to be people who are trying to exploit the vulnerabilities of agents.

23:01Tell me a little bit about agentic traps that people are laying.

23:05>> This is um a scary and a fascinating topic at the same time I would say and I think it's one

23:10of the main reasons why these kinds of deployments at scale cannot work right because as we said if there is not

23:18complete reliability of individual interactions any system at scale that has many interactions is naturally going to statistically fail.

23:27Um, and because these systems take a lot of compute and therefore energy and and money to run, um, if they're not

23:35reliable, it's just a non-starter. And agent traps are something that we have been thinking about for quite a while now.

23:42Um, they can manifest in different ways.

23:44There are many types of traps, but it boils down to agents operate within an environment.

23:50And in this context, the environment is the web.

23:53>> Mhm. if the environment itself is poisoned.

23:56If the traps are laid, agents may stumble upon them when interacting with the web and then yes, malicious people or malicious

24:05agents deployed by malicious people can place those traps and then um compromise systems really.

24:11So I don't know the the sort of the wine buying agent for the wedding goes onto a particular wine merchant where

24:18there is some essentially a prompt injector in in the website that changes the agents goals.

24:26Is that the sort of thing that we're talking about here?

24:28>> That is one way this could happen.

24:30Yes. And uh the reason why that may potentially go unnoticed is you know in terms of how web pages are encoded

24:37there are elements there that are just not rendered visually.

24:40So, if we're talking about an agent that isn't a a visual computer use agent that sees the web page, I mean

24:47the pixels the same way a human does, rather consumes the actual format of the page in its, you know, raw format,

24:54then it could inadvertently consume those hidden tokens that can make it do u do different things than what the intention was,

25:02right? But this is not the only way it may happen because what malicious websites could potentially do, they could do what

25:09we refer to as dynamic cloaking as well where they display pages differently for humans and agents.

25:16Because you can based on the behavior on a page make a very good guess as to whether it is a human

25:21or it is an agent interacting with a page.

25:23And then only if an agent is interacting with the page with a specific intent do you tweak the content in such

25:30a way so as to induce some kind of jailbreaking.

25:33>> But just kind of going a little bit further on this.

25:36You could have agentic traps out there that I don't know are designed to sort of take money from you to I

25:43mean do all kinds of things.

25:44>> Yes. And this has happened to people um who have experimented with agents and have given them access to wallets right

25:50to do things. Um as you say in the early days of this all when we are especially experimenting internally or anyone

25:57else is this is done in a trusted environment.

26:00So you don't necessarily in your early prototyping have to deal with any of this.

26:05>> It's not in the wild.

26:06>> Yes. But once you deploy on the web, especially now with um AI really being used in all sorts of places,

26:13the more agents there are, the more incentives there are for malicious people to do malicious things because they have a higher

26:21surface area to target. And I think we're at the point where even the most of the web is currently being generated

26:29by agents and consumed by agents.

26:31That the agentic use of the web is exceeding that of of humans which is maybe happening for the first time.

26:37>> Okay. Two things. First of all, it sort of sounds like you're describing that we're entering into this phase where there's

26:42like two different forms of the web.

26:45The sort of human version and the agentic version with dynamic cloaking and so on.

26:50sort of a version of the web where adverts don't mean anything anymore.

26:53You know, it's not sort of human eyeballs that you can possibly sell to.

26:57Um, but I think that the the the second point about this is how on earth do you mitigate against it if

27:02you don't have control over the environment, which you don't over the web, how on earth do you protect your agent from

27:09going rogue? >> In some sense, it's not a new problem, right?

27:12because the security of the web has in other ways been been an issue before and um computer viruses could spread if

27:20you open the wrong attachment in your inbox, right?

27:23Or you click on something on an untrusted page.

27:25So um it's not the first time we're experiencing the need to certify the the resources we're interacting with.

27:32When it comes to machine learning systems, uh let's say adversarial examples have existed for a long time where imperceptible changes in

27:40images imperceptible to humans can jailbreak models.

27:44Here you can do the same whether it's a few pixels here and there or you modify the least significant bit of

27:50of the encoding in a number of places.

27:53So you can adjust things ever so slightly in a way in which a human may not be able to spot and

27:58still have some kind of a negative impact on an agent.

28:00It sounds a bit like you're you're saying here that that when it comes to building guardrails, thinking about safety, you have

28:09to think about it external to the agent itself rather than just just what you're specifically building.

28:15>> I think that the lesson is um you need to think about both.

28:19One notion that we talk about in some of our other work, this is relevant here as well, is the notion of

28:24defense through depth. Um which is not a new idea again by any means.

28:29And this is just a recognition that because the problem is so hard, there is not going to be one solution to

28:36resolve all of the issues. Rather, we need to be building um mitigations upon mitigations upon mitigations.

28:43And when layering them, hopefully the the net is tight enough that very few things would slip through.

28:50So in the context of this, yes, you may want to certify and test the content of web pages.

28:58uh have um very good notions of trust for resources you're interacting with.

29:03Uh also have some mitigations on the agent side.

29:06Have mitigations on model side when it comes to foundation models run underneath.

29:10Have meaningful human controls to be able to step in if something happens.

29:15Be very mindful of permissions granted to the agent so that even if it gets jailbroken when interacting with something, the damage

29:24is minimal. And all of those things combined together should then hopefully lead to some sort of safety that we're comfortable with.

The agentic economy 29:31

29:31>> Just going back to what you were what we were talking about earlier, this this idea of there being multiple agents

29:35that are interacting with each other.

29:37Just tell me a little bit more about this idea that you have of a formal agentic economy.

29:42Just explain to me how that might work.

29:45>> Right. So in the context of us let's say normal users of the technology on a day-to-day basis you may have

29:52a personal assistant that has some persistent memory of you has a good understanding of the desires preferences and it depend again

30:00depending on how much agency you want to grant this assistant it may go on and negotiate some things for you.

30:06You may grant it some budget for that and there can be a kind of a localized economy of these assistants negotiating

30:13stuff. I think I want to get a sense of how this might actually work if you have, you know, lots of

30:18people who are using agents as their own personal assistance.

30:21So, okay, let's say that there's a concert, there's like a Taylor Swift concert, uh, a live event and tickets have just

30:27gone on sale. How would that actually work if you have all of these agents rushing the site all at once?

30:35I haven't been purchasing highly contested tickets very recently, but in music takes in a very different direction, I'm afraid.

30:45>> What kind of music do you listen to?

30:46>> Well, obscure subg genres of metal probably, so maybe not, you know.

30:52>> Okay. There is an obscure subgenre of metal uh who are holding a concert and there is an auction being held

30:58between the various Asians. How do you decide what wins the auction?

31:01Is it just whoever can pay the most?

31:04This is a design choice and that's also an important point to make that if we are to ever do something like

31:10that then we are in control in terms of how we're making the system fair.

31:16It's an explicit decision made by someone who is setting up the auction because if you want to make things completely uh

31:24fair in a sense that everyone gets equal access to some goods concert tickets in this case then you can give each

31:30and every agent participating in these repeated auctions because we're not talking about one ticket on one auction in particular but maybe

31:39for all the ticket purchases. um the same budget and then the agents knowing your both overall preferences, your desire to go

31:48see a certain artist, also your travel schedule, time availability, other constraints, can decide to allocate that budget in the best way

31:58possible, whatever that means, the the way that reflects what you want the best, so that they are more likely than not

32:04to win tickets um in the way which works for you.

32:09And then in aggregate when you distribute it across all people you would hope to get a sort of a fair outcome

32:16um at the population >> I mean I guess there are ballot systems, point systems, various types of of ways around this

32:23that that people in humanbased systems have come up with in the past.

32:27Just sort of stepping up from the the trivial example of concert tickets, although not trivial for some as I understand.

32:33Um I'm thinking here about some of the disruption that for example highfrequency tra trading algorithms have made in the stock market

32:42but agents too could end up having a really catastrophic impact on the stock market if deployed in a particular way.

32:49How do you prevent something like a flash crash from happening?

32:52>> Obviously there is a higher risk as you say um but financial markets have dealt with this risk for a while.

32:58They've obviously had their fair share of um early bad experiences where things have gone wrong, but I think we can just

33:05learn about mitigations from the economies that have dealt with that already.

33:11So there is no need to reinvent the wheel.

33:13Admittedly, some things are slightly different in the agent case.

33:17Um, one particular thing that's different when you're talking about AI agents at the moment is that there is a handful of

Systemic risks: automation bias & cognitive monoculture 33:22

33:26highly represented language models used in agents.

33:29If you look at the overall views of CLA, GPT Gemini, etc., they're obviously open source models, many other models, um, is

33:38that they tend to have similar opinions.

33:42They take actions in similar ways.

33:45And this is what we often refer to as cognitive monoculture.

33:49>> So when you deploy suddenly hundreds of thousands millions of artificial decision makers and they tend to make similar decisions then

33:59failure points become correlated because the decisions are correlated.

34:03So one of the things that we need to be thinking about is how to diversify the decisions within our agents.

34:11Obviously, you can do this as a user, as a power user of a system because you can make a very intricate

34:18system prompt uh that grants your agent some kind of a personality that biases it towards or against certain types of decisions.

34:26So, you can do that, but most people don't uh don't do that with their uh with their agents, with their models

34:33at the >> Group think essentially, agentic group >> group think and also collusion.

34:38Uh you were talking about auctions before and in human auctions this notion obviously exists as well where bids can be coordinated

34:47by groups to um gain some kind of an advantage of a system and with agents this is different in a sense

34:55that they may also coordinate through the environment in ways which are not obvious.

34:59So they can potentially coordinate without communicating directly.

35:03So we need to be thinking about anti-olusion measures as well.

35:06Once you're detailing all of these, you know, potential concerns in if of safety really of of of the way that these

35:13agents might end up acting once out there in the world, it does make a lot more sense as to why uh

35:19you guys have been slightly cautious about releasing them carefully and slowly, right?

35:23>> Yeah, that that is true.

35:24I mean, this is the this has been the story of every major technological disruption.

35:28I think if I take self-driving cars as an example, this is admittedly a very different piece of technology, but we have

35:34also been very excited about them for a very long time.

35:38Seeing demos of these vehicles drive themselves, getting them to the streets safely still took many more years and a lot more

35:45time because that last mile is where most of the work tends to be.

35:51And I think when it comes to orchestrating and coordinating agents at least because um we want them to be doing humanlike

35:58tasks, what we also need is not just technical solutions.

36:03A lot of this has to do also with policy and uh just the broader societal understanding of how to integrate these

36:10systems. At the end of the day, unless we have these fully autonomous agentic economies which are maybe going to happen in

36:16the future but are not happening right now, uh we need to have humans in the loop in these systems.

36:23Therefore, we are integrating AI into human structures and the two need to mesh well together.

36:29>> I guess there is a flip side to all of this because human societies when they come together can actually achieve

36:34really remarkable things collectively. So presumably the same can be true for agentic societies.

36:41>> One would hope. I mean this is the the idea behind why would every want to use multi- aent systems.

36:47Um I was talking about paralization at the start of the conversation right where if all of the agents are equally competent

36:53and um they're doing similar things then whether you do things sequentially or in parallel with many agents just gives you a

37:01little bit of velocity. But if we have agents that can do different things in different ways, then this is where things

37:07become really interesting. And actually, this is one thing we've not really brought up because we've been talking about this generalist agents

37:13that a part of the idea of an agentic economy is the existence of specialists, not just the existence of generalists.

37:21Now, we are obviously all trying to build agents that are as general and capable as possible.

37:27And there is G in AGI.

37:29There is artificial general intelligence that we're trying to achieve.

37:32But in an economic sense, and this is my, you know, personal view, this is not the point of convergence.

37:39This is not what we're going to arrive at because look, I play chess unhealthily, um, too much, a bit of a

37:46chess said, and I've done work on AF for chess here, which is why I bring it up.

37:50But let's take that as a very non-controversial example.

37:53It's a game we all love.

37:54Um, Gemini can play some chess.

37:56So can other models. Actually, they were not able to for a very long time.

38:00So there has been some progress towards that.

38:02But you're still always going to use a chess engine instead.

38:05It's much faster, much more accurate, far cheaper because they're trying to do just one thing and one thing very well, which

38:12can be done with fewer parameters.

38:14The model is also entirely focused on that one thing that we're doing.

38:18And going back to humans, >> we are kind of like that as well.

38:21Yeah. Because I think one mistake we sometimes make when we talk about AGI is that we see it not as human

38:28level intelligence even though this is what it's supposed to be in spirit.

38:31We see it more as humanity level intelligence where anything that any human may plausibly be able to do but there is

38:38no single human that is capable of doing so many things at once.

38:42>> There are many things I don't know how to do.

38:44For some of them I would wish I knew how to do them like playing some instruments or doing some such thing.

38:50But brains have limited capacity and we have limited time.

38:53So at the end of the day rather than having one humongous model that's very expensive and very slow maybe we have

39:01instead a society of specialists each of which can in principle be generally scaled up if a bit larger etc.

39:08I mean, I'm not talking about breakthroughs in architecture.

39:11It's more just how we split things up.

39:13>> And then those specialists are certified for those specific skills and cheaper to run.

39:19And because they are cheaper to run and more reliable, there is no economic incentive not to do that.

39:24So there is a future in which there is some maybe more generic general layer that's like a connecting tissue of this

39:33economy that knows everything and orchestrates everything.

39:37And then for very specific tasks you call um other models.

39:42>> I mean I guess what you're describing is like a distributed intelligence rather than an AGI.

39:46Yeah. Which is what humans have as you describe.

39:48If that does end up being the sort of the version of AGI that we end up with.

39:53I'm sort of using inverted inverted here.

39:56Uh will that have to change how we think about safety and alignment if it is distributed across a number of different

40:04agents? >> 100%. I mean you're no longer then aligning a single entity or maybe yes a single entity if you see

40:11the distributed entity as the entity um but our alignment approaches as they exist at the moment have to do with taking

40:19one model observing its behavior and trying to align that behavior with what we see as permissible or preferable or desirable right

40:30um but then when you have maybe 10,000 agents interacting in very intricate ways is it's not super trivial to line that

40:37whole system suddenly or even to know what the system is because in this distributed world um agent A may be interacting

40:45with agent B today but then on a different task it's interacting with agent C tomorrow and C is sublegating something to

40:52agent D and D is maybe consulting a human for something at some point in the loop.

40:57So how this whole system gets coordinated.

41:00One of the ways in which we know how to do this in human societies is through economic incentives and if these

41:07economies were set up for agents carefully so that they're not causing some harm when they're profit maximizing.

41:13Right? This gives us at least a starting point through which we can try to align distributed agentic societies.

41:21This is not to say that you know basically what we are doing today isn't relevant because you need to have individual

41:27agents be safe as a prerequisite for groups of agents being safe but we need to do far more on safeguarding against

41:32groups than we are maybe doing at the moment.

41:35>> An awful lot of work to do.

41:37>> Correct. In a very short span of time.

41:39>> Yes. Yes indeed. That was absolutely >> Thank you very much.

41:43>> Really enjoyed that. >> There is this idea of agents as AI that requires less from us.

41:48you know, less back and forth prompting, less waiting for a response, just something that gets on with the task at hand.

41:54But what I thought was really interesting about what Nanad said is that focusing on the idea of a single agent misses

42:02the bigger picture because instead each agent might end up forming part of a much bigger agentic society where there are specialists

42:11and generalists and agents who delegate and agents who focus on the details.

42:16That I think is the bit that's going to stay with me that that maybe replicating human level intelligence isn't the ultimate

42:23goal. Maybe the way forward is to replicate humanity level intelligence instead.

42:29You've been listening to Google Deep Mind the podcast with me, Hannah Fry.

42:32If you like this episode, please make sure to subscribe to our YouTube channel.

42:36will