Transcript
Transcript: Bias, Barriers and Ethical AI: What Public Servants Need to Know
[00:00:12 Text appears on screen: Bias, Barriers, and Ethical AI: What Public Servants Need to Know.]
[00:00:19 Seyi Okuribido-Malcolm appears on screen.]
Seyi Okuribido-Malcolm (Senior Director, Communications Security Establishment): Hello and welcome to today's discussion on Bias, Barriers, and Ethical AI: What Public Servants Need to Know. My name is Seyi Okuribido-Malcolm. I am a senior director within Communications Security Establishment Canada. Similar to other organizations, AI is top of mind for us as well. But before we begin, I do want to acknowledge that we are here in Ottawa on the traditional unceded territory of the Algonquin Anishinaabe people. Wherever you are, I encourage you to think about the presence of Indigenous people past and present who call this land home because we cannot reconcile what we do not acknowledge.
[00:01:05 Abdi Aidid is shown sitting across from Seyi Okuribido-Malcolm.]
I'm here with Abdi Aidid, Assistant Professor at the University of Toronto Faculty of Law. He's also the Canada Research Chair in Artificial Intelligence and Access to Justice. And lucky for us, he is here as a visiting scholar with the Canada School of Public Service.
Welcome, Abdi, happy to have you here today.
Abdi Aidid (Assistant Professor of Law, University of Toronto): Thank you for having me.
Seyi Okuribido-Malcolm: So, the focus, I mentioned, is really on bias, barriers, and ethical AI, and what we as public servants should really know about this, and some have said the next 10 years of AI is going to be the most disruptive compared to the last 50 years of computing. So, as governments continue to adopt AI at a fast, accelerated pace, I'm curious to hear from you. Why do we need to, at this moment in time, take ethical AI seriously?
Abdi Aidid: Yeah, it's a good and very big question. So, I'll start by saying the most fundamental thing to know about AI is that we have the greatest capacity for good that we've ever had but also probably the greatest capacity for harm that we've ever had, in part because of the scale. So, the main reason that public servants have to think hard about AI and using it ethically is because you can do more damage ultimately, at the same time that you can do more good. And so, part of what I would like for people to think through is, well, what's distinctive about my role as a public servant that makes me different than, say, people in the private sector or other kinds of organizations, and that fundamentally is that there's public accountability. The public depends on you for things related to life and liberty, for example. And so, the room to experiment is a little bit tighter and the margin for error is slimmer. And so, you have to sort of know exactly what you're doing, and part of why I think ethics is a top-of-mind concern for public servants is because there's significant downside risk. It could be asymmetric risk. An AI tool can be insufficiently responsive or it can be biased and inadvertently discriminatory, and that could mean that somebody doesn't get an important public provision that they're entitled to. And so, there's an acute risk of harm and it can happen at scale.
Seyi Okuribido-Malcolm: Abdi, there's a lot of discussion around algorithmic bias. So, from your perspective, what does this look like within the public service but also for individuals and communities that rely on the government for programing and services?
Abdi Aidid: Yeah. So, algorithmic bias, to be clear, is one kind of bias that AI can bring about. Algorithmic bias in particular typically refers to the risk of projecting things in historical data forward in ways that maybe are undesirable. So, if you imagine how a predictive algorithm works, think about your credit card company. What they'll do is to decide what your credit limit should be. They'll look at your spending history, your income, but also they'll reconcile that against their data about what people with similar spending histories and similar incomes have done and whether they're at high risk of default, and they'll use that historical data to make a prediction about what you're going to do in the future. Now, imagine doing that in an area that's socially fraught, right? I think about criminal justice a lot, if we were to try to predict who's likely to be arrested in the future, right? Well, what's lurking in that history that we would use to train the model? It's, for example, a bunch of data evidencing racial inequities in arrests, right? Or in convictions. In Broward County, Florida, they use the tool to try to predict one's likelihood of recidivating or committing another crime for things like post-release conditions, and the technology was identifying Black first-time offenders at being at greater risk of re-offending than white offenders with longer criminal histories, right? And so, you look at that and you say, that's a racially inequitable outcome. But really, it's just projecting the historical data of arrests and convictions in Broward County, Florida forward. It's holding the predictive mirror up. That's one kind of bias, is the algorithmic bias. But often, in government, you're using generative AI tools.
Seyi Okuribido-Malcolm: Yes.
Abdi Aidid: Things like large language models which have, actually, I would say, those same risks of bias but also new ones because you're doing things like trying to approximate human language through technology. And so, think about all the ways, the clunky ways, the insensitive ways that we can use language. I mean, I think people know what LMs are by now, but technically, they're very complicated. Imagine the word "cat" for you and I is C-A-T, but to a large language model, "cat" is not just a C-A-T, it's a long number sequence, a unique number sequence, and then every different type of use of the word "cat" has a different unique number sequence. So, you can tell by geographic coordinates what cities are close together.
Seyi Okuribido-Malcolm: Yes.
Abdi Aidid: If you were to map them. If New York and Toronto are 0.11 and 0.22 but Paris is 0.55, you know that Paris is further away from New York and Toronto than they are to each other, right? So, imagine every word in the English language or any language has a unique identifier and you were to map it into a theoretical word space like geographic coordinates, right? You would start to see words with similar meanings being close together. Well, if large language models are learning from the way we use language, it might put "doctor" closer to "man" than it would put it to "woman". It might put "nurse" closer to "woman" than it would to "doctor", right? So, because it's learning from us, it's actually, a through technical process, mapping and understanding of how we use language, and we don't always use language in ways that are fair, reasonable, becoming of our social understanding. And so, there's that risk that's inherent in it. At the same time, technologists don't like that. They don't want that to continue. And so, there's a lot of effort, in part because it can be embarrassing when the technology is doing things like projecting stereotypes forward. So, there's a lot of effort to tamp down on some of those problems. And so, our job as public servants is to help guide those efforts and insist on them, for example.
Seyi Okuribido-Malcolm: Understanding what you just shared, it sounds like it's really important to keep the human in the loop, but the human also comes with biases too. How do we reconcile keeping the human in the loop, trying to correct for historical bias, but understanding that there is still current bias?
Abdi Aidid: Okay, so I'm going to say something that might be a bit unpopular, which is a human being in the loop isn't automatically the best idea every time. I think sometimes we use that as a bit of a crutch to tell us that this is how we intervene in the automated future, we make sure that human beings can make some decisions. That's true if the human beings actually have discretion. But if what you're doing is investing the technology with all the discretion and ultimately all the human being can do is validate, right? Then, what you've done is you've moved the discretion upstream, delegated it to the technology, and then you have human beings who really don't have the authority to meaningfully intervene. That, to me, is not automatically better. They have to be human beings that have the proper skills, training, authority, the proper capacity to supervise the technology, to explain it, to know what its outputs are, then putting them in a position where they're empowered to, say, disagree with the technology. To me, that's a meaningful human in the loop. But if you're putting the human in the loop merely as a rubber stamper and they don't have actual authority to zag where the technology might zig, I think that's probably window dressing on a bigger problem.
Seyi Okuribido-Malcolm: So, we've talked a lot about preventing harm but AI can also be a tool for creating equity. Can you share an example where AI was used successfully to support equity, diversity, and inclusion goals?
Abdi Aidid: Yeah, I think that, again, it's all about how we use the technology and how we harness it, but one of the things that AI does is that it's able to ingest larger volumes of data than you and I maybe can. And so, we might have biases but we might also have informational blind spots that AI can help with. I think often about doctors diagnosing skin cancers in people of colour. It's actually quite hard and for a long time was an area of lower performance for doctors to be able to diagnose things like melanoma in, say, Black people, in part because the heuristic tools that people were using, discolouration, whether there was asymmetry, those were often more difficult to spot on melanated skin, right. So, there's that sort of observational challenge, but the other was that there weren't a lot of us in the case studies and there weren't a lot of us in the empirical work that was informing the clinical practices. And so, if you're a doctor and you actually just want to spot every possible thing which could pose a risk to your patient, then technology can help you quickly synthesize a much larger universe of information. Not just that, it can give you things like scripts for what kinds of questions to ask to detect issues that are outside of your sort of typical zone of questioning.
And so, there's actual ways in which we can use technology assistively. I should say, it depends on where you point the technology, because we could use that same thing as a bloodless, triaging tool, and maybe we wouldn't have the same benefits. But with a learned professional who wants to do well, then we're talking about the true promise of being able to supercharge your capability, but I can think of a lot of examples, education's actually a really good one right now, where you can have… I mean, think about this. ChatGPT is trained on the sum total of the internet, just to super simplify it, okay? But imagine you could say, I want to use a large language model and its fantastic language generation capacity but what I'd like to do is restrict what it can make factual assertions from to an 11th grade physics textbook. What do you have now? You have an around-the-clock tutor for a kid who, like me, wouldn't be able to afford any of those sort of additional supports, right? That, to me, is some of the equity promise of these tools and part of why it's not worth throwing it out because of concerns that we maybe have a fighting chance of constraining.
Seyi Okuribido-Malcolm: Just building on that, can we go to the extreme where we trust it too much where we're not going to take the time to do the fact-checking?
Abdi Aidid: Yeah, I think that's probably true of any safety technology. Economists call it moral hazard. If you have anti-lock brakes in your car, you might drive less safely. If you have insurance, good insurance, you might drive less safely. If you have a backup camera, you might not do shoulder checks. So, I think we're vulnerable in the same way we are to any safety technology.
Seyi Okuribido-Malcolm: Abdi, you spoke to us about algorithmic bias. I understand that, you talked to us about generative AI bias, but I'm also hearing about agentic AI and to me, it sounds a bit concerning. And so, I'm just wondering, from your perspective as public servants, should we be concerned legitimately or is this a domain or an area we should be leaning into?
Abdi Aidid: Let me start by saying I think there's remarkable upside in all of these tools, but it really depends on whether or not we understand and control the downside. So, I'm excited about the possibilities of algorithmic tools, generative AI tools, an agentic tools, except that I think that we need to constrain some of the risks. And so, I'll talk about the peculiar risk of agentic AI. So, I talked about algorithms before. That's like surfacing insights in existing data, trying to mimic human cognition in some ways, pattern recognition. So, think about algorithms as technology trying to think like humans, and then think about generative AI. It's learning from how we use language. It's trying to approximate the way that we interact. Even deepfake tools, for example, are trying to approximate the human aesthetic, for example. And so, if algorithms are the technology thinking like us, think about generative AI as looking and sounding like us, and agentic AI is behaving like us, right? So, it's about taking all of that ability in the advanced data science techniques and then investing them in a persona, an AI-enabled persona. So, what's going to happen is that the technology is going to be capable, it already is, of engaging in these multi-step tasks or transactions without supervision at any one constituent step. So, today, if you want to do research, you might go to your computer and you might open up your browser and run some searches and you open up a Word document and you put some stuff in the Word document, then maybe you open up a new one where you're kind of writing the draft. Imagine the technology can perform all those tasks but without you having to give discrete instruction to open the document or to copy the stuff into the document or to run the searches, right?
And I mean, the best way I can describe it is, do you remember when you used to call tech support some time ago and they'd be like, is it okay if I remote into your computer, then you start to see the cursor moving around your computer? Think about technology being able to do that but with all of the insights of big data, right? And on top of that, with instructions that are appropriate for a given persona. And so, if you're talking about an agentic tool that's a call centre agent, then it's going to approximate what we understand based on data and based on observation to be the mannerisms of a call centre agent, the ideas of what makes a call centre agent good, but it's also leveraging all of this historical data, right? And so, the possibilities of agentic tools are high responsiveness, the possibility of agentic tools are more end-to-end task completion. For the public service, that can also mean a front line with the public that is informed and highly responsive and adaptable and can route people and direct them appropriately. Now, the risks are many. One major risk, of course, is in thinking through the personas, that we do things like import stereotypes, because we have to make determinations about what those appropriate personas are. And even if it doesn't end up stereotyping, we could flatten or narrow what are complicated types of roles, for example, right? So, that's a risk. And then, it just imports a lot of the same risks because of the data being the core engine. And so, I think that we don't need to necessarily think about agentic AI as posing distinct problems but potentially compounding ones if we don't resolve them when we're using other types of AI.
Seyi Okuribido-Malcolm: So, Abdi, as humans, we're evolving. And if this is a tool that learns as we are evolving, should it too not also evolve in a positive and hopefully healthy way?
Abdi Aidid: Yeah, I mean, evolution is slow for humans. It's evolving faster than we are, the technology. Think about unconscious bias, right? If you've ever had a conversation with someone who you suspect is harbouring some biases, they have a very hard time accepting it. And often, in the conversation, you get to a place where you're like, maybe the only thing that can work here is if there was some possible consequence for you continuing to behave that way, because you can't tell them to go back to their childhood and totally re-imagine what it's like to live with people on a footing of kindness and equity. But the technology, we can actually do everything from a erase its memory and re-program it to create arbitrary rules and consequences to discipline it, right? And so, there's ways in which it can be more flexible. But also, it can also shine a light on some of our wrongdoing. So, I talked earlier about the predictive mirror that algorithms, for example, can hold up to us. Well, I'll give you an example from the legal community. Judges will very often say things like, I can't trust an algorithm, I don't know what it's thinking. Well, I don't actually know what a given judge is thinking either. I don't know what underlying biases they might have. I don't know how they might feel about me. I hope through legal reasoning and through them having to justify decisions via case law that I can move even the most biased person safely into a terrain that I can interact with, but it's not a guarantee at the same time. And so, there's a way in which we look at the technology and say that thing has a greater propensity for bias than we do, but that might not necessarily be the case. And even if it is the case, there are sort of technological ways to intervene and fix some of those things if we're willing.
Seyi Okuribido-Malcolm: Abdi, from what I'm hearing from you, agentic AI, similar to humans, can conduct tasks end-to-end without supervision. However, humans have consequential accountability. Should we be instilling and developing frameworks of consequential accountabilities for agentic AI?
Abdi Aidid: Yeah, I think that's exactly the right question, and the reason why is because sometimes we look at what something can do and we stop asking the question about what it should do, right? So, my daughter is seven years old. She can technically do all the tasks involved in going to a grocery store, picking up food that she wants, and maybe with some prep, she can figure out a way to pay for it at the front desk, right? But should she be doing that? There's a safety issue to her going to the grocery store alone. There's a question of what choices she's going to make. For example, is she going to buy healthy items that she should really be eating or she just going to get a bunch of gushers and juice-boxes and those kinds of things, right? And then, I think here's what's core to it, is there somebody that's better positioned to do that work that isn't her? Those are still the questions to ask even though the agentic tools can technically perform the tasks, right? And so, for me, there's a question of what's technically possible, then there's what's normatively desirable. So, that's a first threshold thing to think about. More to your point about accountability, this is a problem that we didn't quite figure out when we had a chance to in the world of sort of automated technologies, right? Where we didn't exactly know where to place liability. Is it on the manufacturer, the developer, is it on the end user? So, I think it's something that we need to prefigure. We need to invest legal, judicial resources, legislative resources in trying to figure out, because you know in your job, if you use an agentic tool and it does a bad job, you know practically, you're going to be accountable.
Seyi Okuribido-Malcolm: Yes.
Abdi Aidid: Practically. But the question is, could you have intervened? If you couldn't have intervened, right? If it went haywire, for example, then maybe you shouldn't be the accountable party, right? Maybe it should be the manufacturer or developer. That's still stuff that we have to work out. So, public service, I think, rightly recognized that the buck does stop with them, but we also need, as we move towards more certainty, a clearer sense of where accountability might rest.
Seyi Okuribido-Malcolm: So, understanding that, Abdi, the good and the harm it can bring, where do we as humans have the capacity and capability and opportunity to… you said we cannot slow it down but if we need to turn the switch, can we do that? Who can do that? Who decides when to do that?
Abdi Aidid: So, there's a big picture turning the switch, which is things like legislation, just being active civic participants who have something to say about what the AI future ought to look like. Getting the right rules and constraints at a legislative level, I think, in the order of operations, that's task number one. But beyond that, part of it is also recognizing the different dynamic that exists in tech development versus, say, other kinds of manufacturing. For instance, tech development is iterative, right? Technology companies do the whole, you might have heard this term, land and expand. So, if there's a major tech company that wants to sell to the Government of Canada, what they want to do, really, is often they'll start with a smaller procurement, they'll then roll it out across the organization, but in the meantime, they're trying to learn about appropriate use cases. They're trying to also tinker with the technology to improve its usability for the rest of the public service. Because it's not like having to take apart your car, it's really about sending effectively what's called an over-the-air update, the technology, you have actually real, meaningful opportunities as a user to intervene in how the tech works so long as you see yourself as somebody who can provide them valuable information. And so, for public servants who are concerned about the way that an AI tool is operating, talk to the company, right? Find structured ways of submitting information to them. Very often, they'll ask you, in the technology itself, do you agree or disagree with the result? What's your feedback? That information is important. When I was a technologist working, trying to figure out how to design tools for people's use, particularly lawyers, we actually didn't get enough feedback often. There were people who had issues with the whole enterprise but weren't taking their discrete opportunities to signal important information to us that we were sort of happy to learn from.
Seyi Okuribido-Malcolm: So, we need to empower ourselves to take more deliberate control in how we use and future use the technology.
Abdi Aidid: Yeah, think of these things as unfinished products and think of yourself as the test group.
Seyi Okuribido-Malcolm: Yes. And so, we have influence in shaping.
Abdi Aidid: You do.
Seyi Okuribido-Malcolm: And directing.
Abdi Aidid: More than you would think, especially if you organize. I mean, the public service has things like affinity groups. It has groups that are oriented around interest. Get together and consolidate your feedback as well.
Seyi Okuribido-Malcolm: That makes sense. And so, as we look to the future, what's the mindset and the new skillsets that we as humans need to have to, to your point, be able to better govern these tools and to provide the proper level of oversight that will actually make a difference?
Abdi Aidid: I will say that I think that the technology that underlies AI is probably going to be mostly invisible soon. What I mean by that is we are not walking around the world thinking that hard about technological processes that underlie most of the things that we use and do, right? In fact, I would say we're fundamentally incurious about most technology and how it works. I think the reason that we're amped up to a particular peak of interest with AI, because we don't trust it, we want to look under the hood because we don't trust it. But if we can develop an ecosystem around AI where there's constraints, where there's someone to call if you have a concern, where we have success stories of AI being used, we probably will be more in a trusting environment, and then we can look a little bit less under the hood and think more about the important second-order questions like, what's the AI doing? What is it achieving? Is it serving our best interests? I give the example often of your laptop. You don't exactly know most of the technical processes that make the cursor move a millimetre on the word processor but you know who to call if it goes wrong, you know where to take it to, and you also know how to determine whether it's working or not.
If you have those skills, which, by the way, are not technical, they're about issue-spotting, for example, then you're probably well-positioned to succeed. And so, what I will say is once you layer AI on top of various public service efforts, you'll see that it'll maybe expose some of the areas that we should have maybe been investing in for a long time. So, one example right now is personnel management, right? So, if now you have a new technology that purports to supercharge people's ability and it can purport to replace some of their tasks, then the question is, what should people be doing? How do you get the most out of the people that you have? That, to me, is kind of an old school analog consideration. So, if you want to be poised to succeed in the AI future, double and triple down on things like change management skills, personnel management skills, the sort of 1:1 relational coaching stuff, because we're going to have to co-exist with these things and we're going to have to maybe do more with less in the future, and that's a deeply human challenge.
Seyi Okuribido-Malcolm: Abdi, we spoke about a lot of things. As public servants, what should we be primarily focused on right now?
Abdi Aidid: It's a good question. Okay, I always hesitate. I have thoughts but I very rarely have direct advice, but this is going to be the one time that I'm going to say what I think people should do.
Seyi Okuribido-Malcolm: Yes.
Abdi Aidid: The first thing people should do is think proactively about where they can use AI now, today, even if you're somebody who has ethical concerns. Why? Because the thing we need to be are credible objectors to AI, right? If we have objections. So, there are plenty of people who have issues with AI's environmental impact, and rightly so, there are plenty of people who have issues with its capacity to reproduce things like bias and discrimination, rightly so, but that doesn't necessarily mean that when your boss is talking to you about maybe using AI for internal summaries or some other low-hanging fruit task management thing, that you shouldn't figure out a way to do it, right? Ask yourself, what are the things that I'm truly concerned about versus what are the things that I'm just averse to because I don't like the technology? Think about ways to proactively use AI for all of the low-stakes stuff that don't implicate the bigger ethical questions, right? That way, you're a credible objector when you do have an ethical concern, right? You don't seem like a Luddite or someone who is averse to the whole enterprise, and part of why I say think now is because the horse is out of the barn, so to speak. There's no world in which AI is going to be meaningfully slowed down, anymore than we had a chance of slowing down electricity. The economic case for it is just too compelling. And as a matter of policy, there isn't a country in the world that isn't thinking proactively about ways that it can use AI for things like economic growth. And so, there is neither the technological possibility of slowing it down, the economic rationale, or the public policy appetite to slow it down. So, it's going to be a part of your life. And so, start by making a list of things that you can use AI for and maybe which things might be AI first.
Seyi Okuribido-Malcolm: As we look ahead, what is the one mindset or practice public servants need to adopt to ensure AI strengthens rather than undermines trust in government?
Abdi Aidid: I think it's really reminding yourself about your unique position as a public servant. You're accountable to the public but you're also delivering vital services, developing important policies, administering really important programs that people depend on. And so, in the same way that that should remind you when it's time to sort of hit the brakes on technologization, it should also encourage you to think about ways that you could use it too to better discharge that obligation. I think sometimes about people who maybe have sort of misjudged what the public wants from them, right? There's a bit of a cautionary tale I talk about sometimes, which is the example of graphic designers. Graphic designers are very valuable, important people in our society, right? They help make everything in our hyper-commoditized world legible to us among other things, and they are artists. They're artists, right? And they use their knowledge of art history and technology and aesthetics to give us things like brilliant logos that best capture what a company is trying to communicate out in the world, right? And they see themselves that way, and it turns out that the public wants unpixelated logos that they can use ChatGPT to generate in two minutes, right? And there's a story there, which is, is your self-image and the idea of what you're here to do, does it effectively sync up with what the public is expecting from you? Maybe sometimes the public doesn't want you to spin your wheels and take the scenic route to an answer just because you've learned along the way. Maybe actually what they want is high responsiveness, right? Maybe sometimes they want you to push the boundaries of human knowledge and bring your whole creative self to a task. Maybe other times they want something serviceable, right? And so, part of what we need to do is do the difficult work of thinking through what exactly are the demands of us, and then choosing whether or not tools can help us along that goal.
Seyi Okuribido-Malcolm: Abdi, as we sit here in the big data analytics centre, it's clear we are leaning in, into this space. However, Canadians may not be at the same rate or pace leaning in with us. How do we help increase the trustworthiness of AI with Canadian citizens as we look to modernize our programs and services and create that digital mindset?
Abdi Aidid: Yeah, I think that trust is really a function of minimizing error and horror stories, frankly. I think that distrust for AI comes from people hearing about things that have gone wrong. And so, if you have the mindset of, how do we reduce error, how do we be extra careful in circumstances where the AI is operating in an area that's already a little socially fraught, I think focusing on error minimization is probably the first way that we can start engendering trust. So, for example, if it is the case that we're going to be using agentic AI solutions to respond to things like public inquiries or even calls that the public might make into the revenue agency or into Service Canada, for example, then maybe we don't just say the AI is going to respond to all inquiries. Maybe we say some of them are going to be AI first. For example, if it's routing you to other information, that's good for AI. But if it's about giving you direct advice that's sort of bespoke for your circumstances, then maybe that's where you get routed to a person. And so, us disambiguating and saying, what are the places that generate the most error and how can we ease our way into using AI in those contexts and what are the ones that AI is perfectly well-suited for, how do we use it now, I think, is one of the first ways to engender trust, is to do it really well.
But there's another thing about trust which I think is more psychosocial and cultural, which is we have to change our relationship to technology in general. So, AI is not like hardware. It's not even really like software, right? It's a little bit more like a junior colleague in some ways, right? So, think about large language models. What do you notice when you're talking to a tool like GPT? It has the earnest kind of tone that somebody has when they know what they're talking about. But if you think about the phenomenon of hallucinations, it also hesitates to tell you when it doesn't know what it's talking about or when it doesn't have sufficient information, which is like every overzealous junior colleague you've ever had. And so, what do you do in that context? You trust but verify. You double-check factual assertions. You make sure there's layers of review. You don't just pass off what your most junior colleague who just started day one of their internship, you don't just pass on what they've done to the client or to the Prime Minister or to the public. You build in structures to get the most out of that person, recognizing their limitations, which is a different relation, a different posture than we have to technology. We think of technology in these binary states of success or failure. Did it get it right or not? Which might work for when you're talking about hitting a spacebar on your laptop, it doesn't work, okay, it's broken, that same way we think about a doorknob, you turn it, does it disengage the latching mechanism? Okay, but if your junior colleague brings you a memo and it's fine but could use some additional work, do you say task failed or do you iterate with them, do you dialogue with them, you work with them? And so, if we have that relationship to AI, which is that we are a player in a back-and-forth dynamic that requires some iteration, it's dialogical, then I think we can get the most out of it, and that's, I think, a cultural and mindset shift that we have to develop.
Seyi Okuribido-Malcolm: Thank you, Abdi.
Abdi Aidid: Thanks for having me. It was a good discussion.
[00:34:46 The CSPS logo appears on screen.]
[00:34:51 The Government of Canada logo appears on screen.]
