Transcript
Transcript: Leading Projects in the Government of Canada, Episode 5: Establishing Project Outcomes as a Business Owner with Larry Murray
Andre Fillion Hello everyone, I'm Andre Fillion. Today, we'll have a discussion with our guest, Mr. Larry Murray, about some of the critical responsibilities assigned to business owners. We'll talk specifically about the establishment of desired business outcomes in the early stages of projects. A task that requires business owners to begin with the end in mind to focus on what the project has to achieve to be successful, which then allows for detailed discussions about means to achieve the end; the solutions, the capabilities, the systems, the activities. Sounds logical and simple, but we wouldn't be talking about this today if it was that easy. So most of us have heard of projects that started with a precise solution in mind and delivery of the solution or the systems, not necessarily the desired business outcomes becoming the focus and even the criteria for success that unfortunately can create a tunnel vision, can limit opportunities for innovation, can constrain procurement options, can delay approvals, and unfortunately, that can lead to some problems. So we'll talk about all of this with our guest, Mr. Larry Murray today. He was the chair of the Independent Review Panel for Defense Acquisition at DND. Larry is a former Deputy Minister at Fisheries and Oceans, former Vice Chief of Defence Staff, an extremely busy person as Honorary Grand President of the Royal Canadian Legion. Larry's a member of the Order of Canada. So we're very privileged to have him with us today. Larry, welcome. How are you?
Larry Murray I'm good. How are you? You look well.
Andre Fillion Thank you very much. Thanks. Thank you for being with us. We know you're super busy as you as you continue your work with the Independent Review Panel for defense acquisition. On that, can you tell us a little bit about it, why it was created and what's its mandate since 2015?
Larry Murray Sure. And I think most people would probably recollect the controversy that was in play in 2013, 2014 around the F-35 acquisition. And there were other less visible challenges in the system at that time, which led to the various departments coming together to formulate something called the Defense Procurement Strategy. And one aspect of that, one small aspect of that was actually the creation of a third party, independent review body to review and try to bring greater clarity and credibility to the operational requirements for national defense of major projects in particular. But of all of our projects, this led to the creation of the Independent Review Panel for Defense Acquisition (IRPDA), which in hindsight probably should have been named the Independent Review Panel for Defense Requirements because the panel's focus is entirely at the front end and primarily focused on trying to get requirements that are clear, credible and understandable by ministers, by officials of all involved departments.
Andre Fillion I think it's fair to say that since then, the IRPDA has imposed a disciplined approach to establishing requirements or outcomes for projects. They were concerned at first, though, that this would take longer to a process that already takes long. So far, what's the feedback that you're getting? Is it a worthwhile investment in time to go with the panel, have a review, in some cases make adjustments to the requirements, the outcomes that projects have to deliver?
Larry Murray I think it's been very positive. We do our utmost not to impact on the schedule. We rely on the documents that are already being produced or that exist in relation to the project or the threat or the whatever that comes into play with a particular requirement. The scheduling and so on is done by the authorities, internal in DND that organize that. And we actually see projects a minimum of twice: once after they're formally declared projects. And in that initial engagement, we really focus on call it the front end, the Policy Foundation, making sure the capability gap, in other words, the need is well understood. And the high level mandatory requirements, we have an initial look at the options analysis and, you know, procurement approaches and so on. But the fundamental focus the first time around is, as I say, on the basic strategic context, getting the capability gap right and the requirements. We then see the project again after the options analysis phase, normally 1 to 2 years later and after the project has been approved by the internal DND authorities to go to either the Minister or Treasury Board for approval to move into definition. In other words, just prior to the government starting to spend serious money on these projects, all of which in our case we're looking at $100 million or more in terms of projects. And in that second engagement, we're primarily focused on the options analysis, the selected option, something called the preliminary statement of operational requirements. And in that regard, we're trying to ensure traceability between the preliminary statement of operational requirements going forward in the selected option and the high level mandatory requirements. In other words, does what is going to ministers for decision bear some passing resemblance to the high level mandatory requirements that ended up identified at the front end? Each of those engagements take about 3 hours, but there's a fair amount of work on the part of the owners, the implementers. And actually, in terms of the panel itself, we all go through the documents in both cases individually. We have a small staff that puts together an analysis note for us. We come together and ensure that we've got a common view before the engagement. I would also say that the panel composition actually matters. In other words, it's five GIC (Governor in Council) appointments and the model that was initially created we've stuck with, and that is two individuals with significant public sector background, one of whom has some military background, two of whom are private sector background senior executives, and one of whom is an academic. Our questions are basic questions. And I think, maybe that's a really key message. I was a little bit from Missouri at the front end of this process when I was asked to chair, I wondered if it would make a difference. And it has made a difference, but I think a part of that is we're talking about five people, none of whom are really experts. We're asking very basic questions and we're asking questions that everybody in the room has been provided with. So it's really a question of these big, complex projects of actually ensuring that the right questions get asked. I mean, having been exposed to this now for seven years and 80 projects, I wish I had used something like this at various other stages of my life.
Andre Fillion It's so true we're often thrown into positions where we don't have a lot of background into the discipline that's at stake, but yet we are accountable. And it's important for us to feel confident because basic questions can go a long way in finding out whether the foundation is substantial underneath a project. So in your role then as chair of the IRPDA over several years. Can you share your thoughts on what you would consider the characteristics of well-defined, high level mandatory requirements (HLMR), or what we would refer to as business outcomes for projects?
Larry Murray Maybe it would be useful to provide the DND definition, and then I'll give you my take on it. We had mandatory requirements, but the capital H, capital L, capital M, capital R, "HLMRs", were established in 2015, not coincidentally with the establishment of the panel because it was determined or deemed that the panel core business was the requirements and there needed to be guidance on high level mandatory requirements. And so guidance was put together. In fact, if the panel was a waste of time, the establishment of high level mandatory requirements would have justified the thing. And in their guidance on the subject, HLMRs are foundational statements of specific capabilities required by the Canadian Armed Forces to meet government defence objectives, and they thereby set out the core objectives of a project. They also serve as a high level binding agreement between internal approval authorities. Typically, the Defence Capability Board, which is a body chaired by the Vice Chief of Defence Staff and the project team regarding the capabilities to be delivered. Now, in terms of characteristics, which was your kind of opening question, I think good HLMRs are clear. They capture the essential elements of the project. In other words, essential that without each of these elements involved, the project can't really successfully move forward. Good HLMRs are comprehensive, they're results oriented, and they also provide a sense of sufficiency. And quite honestly, maybe something you want to come back to, but in things like trucks, planes, ships, one of the biggest challenges, even with a good set of HLMRs, is figuring out how many of them, like how much is enough. So there needs to be some considerable thought on this. And the final one is measurable. In other words, measurable actually to a pass-fail level in the DND case, so that they actually have enough specificity that they can be used correctly in screening the options.
Andre Fillion So maybe expand on that a little bit and the specificity in particular. For example, if you just take vehicles, you know, if the outcome that you're after is to move material from point A to point B, when is it appropriate to go to the point where you say we want trucks versus we just want to move equipment and we'll see what the art of the possible and what kind of innovation can come with that. And it's not an exact science. There's a lot of judgment involved in this. Maybe your thoughts on what's the balance between specificity and remaining at very high level outcomes for a project.
Larry Murray This is probably the toughest part of sorting these things out, because if you are too specific, you're ultimately heading for sole source by stealth or you're limiting the options down the road when industry gets involved. On the other side of the coin, if there's no specificity, which actually is the greater challenge that we see, then they're really of limited to no value. In other words, you can't figure out whether you're buying a barn or an airfield or a ship. It's a bit of a problem. And there is a real worry generally on the part of the business owners to end up with being too specific. I mean, I think HLMRs will always be a work in progress. But that is really one of the more challenging parts of the equation. I would say as well that when ADMs (Assistant Deputy Ministers) are trying to figure out where to get involved in these projects, getting involved at the front end at this phase [is] where it's really critical to have discipline, a strategic view. HLMRs, as well, are not requirements that get captured or generally not requirements at the level of detail that go in to technical statements of requirements. They're really a strategic level package that defines what it is you're trying to acquire. And I think they're not at a level of detail that makes them not the business of the ADM for a $100 million project or a billion-dollar project. That's kind of the front end stuff. If you get that right, if the capability gaps been well-defined, you get to the HLMRs which give you the ability to do a good options analysis and end up with a good option. Definition and implementation are a whole lot easier. But if you don't get those things right, it can be really problematic.
Andre Fillion As I said in my opening comments, if you're not careful, you end up in a situation where you rule out all innovation, you even constrain the procurement options. You can end up almost in some cases pointing to a piece of equipment, but then on the other hand, if you stay high level enough, then you can encourage innovation and competition. I would say this might be a bit of a feedback loop associated with the options analysis too, you know, what's the art of the possible, what can the market offer, what are the options? And perhaps do you see a bit of a feedback loop going back to HLMRs, outcomes?
Larry Murray Absolutely. In fact, we've done a number of studies on HLMRs because of some of these difficulties and we're about to within DND put out some additional guidance for the panels. But part of it is that down-the-road feedback loop on "did you achieve the HLMRs" and "was the approach credible". The other thing that we find –and it's I think part of the answer to what we were talking about a few minutes ago– is that HLMRs aren't cast in concrete. In other words, they were put together by human beings. You go into options analysis, it's entirely reasonable that it's discovered, which it is not infrequently as you would know in these military projects, that the capability that folks thought existed in industry actually doesn't exist or is not going to exist for five years. It's totally developmental. So therefore you walk back and you put together a credible HLMR. And I would say with IT projects, when we start talking about agile procurement now and that sort of thing, I know within DND the ADM MAT is trying to come up with something called continuous capability sustainment through the life of a project. Well, the guardrails that he's hoping to have for that are HMLRs, in other words, so that you can have the flexibility you need to hopefully end up with something that remains current and effective. And somehow we've got to use these various processes to buy smart and to be able to maintain them at a credible operational level, which means that HLMRs, in my opinion, are going to have to be flexible and revisited through the life of project, probably.
Andre Fillion That's a very good point. In fact, Treasury Board Secretariat hired an independent third party, Goss Gilroy Incorporated, to conduct a study of the management of the Transformation Pay Administration Initiative, or Phoenix. You know, they kind of speak to that and they say that the outcomes were defined but not necessarily tracked and managed. And one of the lessons that they documented in this, and it's a public report, says that you need projects need to develop and fully implement explicit outcomes of management throughout the life of Transformation Initiative from inception to post-go live. And I think it's perfectly appropriate. I think it's a way of thinking that you don't just treat this as an event that takes place at first and you forget about it. It's something that needs to be managed. It links back to some of the lessons learned the hard way with some projects. Maybe just a quickly again from your experience, lessons learned, dealing with so many business owners, so many projects, some of them, especially in the early days before the panel. Any words of advice on how to deliver quality requirements and outcomes?
Larry Murray I think the focus needs to be kept on the capability gap, the capability requirements. And as it moves forward through options analysis, you really need to ensure and certainly you don't need to be a technical expert to have a good sense of what's realistic. Is this realistic? Is it a valid, sensible capability requirement we're trying to come up with and also not allow the options analysis to go off into procurement approach as opposed to capability requirement. In other words, when the panel was initially created, we found in the options analysis you'd have a whole bunch of good work going on. But when you actually got to the options analysis, it was buy, sell or lease. In other words, it was oriented towards the procurement approach. And that's fair enough. But when you're trying to figure out what is the capability that makes sense, that will meet the HLMRs that will fill the capability gap, the options need to be capability kind of based options and the costing, although it's challenging in options analysis, which is early in the project, but to the extent that folks can try to ensure that the costing is as accurate as it can be, and it does tend to get better as you head through options analysis. By the time you have actually a selected option, there should be reasonable confidence that the costing is credible. Otherwise, you go through all this stuff and you find that you've got an option, that you've got 50% of the money for or whatever. But again, if you use a capability based approach rather than a buy, sell or lease procurement approach, we have started within DND to develop something called capability ladders because from the panel's perspective, find it really difficult if you have two thirds of the funding. How do you figure out what is the most important part of the capability or what are the most critical capability requirements? And so the capability ladder has started to be a standard part of major projects there. They're found to be particularly useful by decision makers who have to make really significant decisions on hundreds of millions of bucks and buy, sell and lease doesn't give you the ability to do that. But if you do have a visible capability ladder that's got the capabilities on one axis, the funding on another axis, it is possible to actually be comfortable as a decision maker that you've got a good sense of what you're getting. Is it worthwhile to press on in that context? Or do you really need to go back and ensure you do get the required money because the whole capability is so important? Or does two-thirds of it give you, you know, a significant capability that is much better than what you have and will enable you to get the job done.
Andre Fillion And ultimately it's what, what helps the decision makers.
Larry Murray Exactly. Right.
Andre Fillion So there's a balance there as to how far do you go, how quickly do you go to a preferred option, and what do you give to decision makers so that they are really understanding and fully in control of what they decide.
Larry Murray In my view, a part of this is to try to ensure that the decision makers have the tools to make a good decision. As the business owner, you have a particular scenario in mind. You've got a much better chance of getting the best possible decision if you've armed the decision makers with the wherewithal to make a good decision.
Andre Fillion Any final thoughts? You've dealt with a lot of business owners, sometimes it's intimidating, but you still own it at the end of the day. So perhaps any words of wisdom here and advice.
Larry Murray I think for major projects if you're moving forward $100 million or $3 billion project or anything that's significantly important to the department or to your chunk of the department, you actually have to be involved in this. I would say be involved at the front end. You'll bring a strategic perspective to this, that the technical folks or the more junior folks won't have, they won't have the strategic context necessarily. And this stuff is not rocket science. This stuff is common sense. I don't know how much time I spent in my experience in government and other places where we kind of wrestled with some huge problem without taking the time to figure out what the question was. I don't know how many times we've been around and three months into it, the light comes on and you say "this is what the question is." In this stuff, you really got to have a few strategic minds in place who haven't been involved in all the trees, have a reasonable view of the woods, to figure out what is the actual requirement and recognizing that money's tight. What is the actual requirement that is affordable and still enables us to get the job done or to look after people or whatever we're trying to do with the particular piece of kit or the capability we're trying to acquire.
Andre Fillion Thanks for that Larry. And on this useful piece of advice, let me thank you for sharing your thoughts with us today. No doubt in my mind that establishing desired business outcomes and implementing outcome management practices is critical to the foundation of a project. It gives it good DNA. For those with responsibilities in this area just a reminder about the Treasure Board Secretariat's outcome management guide and tools. I think it's a very good document, can be very helpful in the early stages of projects. So Larry, thanks again for being with us today.
Larry Murray Thank you. It was a pleasure. Thanks for the invitation.
