Designing and Delivering Modern Data-Driven Services in Times of COVID (DDN2-V08)

Transcript: Designing and Delivering Modern Data-Driven Services in Times of COVID

[The animated white Canada School of Public Service logo appears on a purple background. Pages turn, opening a book. A maple leaf appears in the middle of the book that also resembles a flag, with curvy lines beneath.]

Welcome. Designing and Delivering Modern, Data-Driven Services in Times of COVID - Executive Panel

[A logo in the bottom left corners says "Canada School of Public Service/École de la fonction publique du Canada.]

[On a purple background, a logo in the top right reads "GC Data Community." The panel's participants are listed. Moderator: Kara Beckles, Director General, Data Integrity, Privy Council Office. Panelists: Ima Okonny, Chief Data Officer, Employment and Social Development Canada. Tom Dufour, Director General, Statistics Canada. Jean-Francois Ruel, Director General, Canada Revenue Agency.)

[A woman begins a Zoom meeting.]

Martha: I would now like to welcome our moderator for today's presentation and our executive panel. Kara Beckles is the Director General of Data Integrity at the Privy Council Office. Over to you Kara.

[Kara Beckles appears on the call.]

Kara: Thank you, Martha, Thank you very much.

So for those of you who don't know me, my name is Kara Beckles and I am the Director General of Data Integration with the Results and Delivery Unit at PCO. I'm very excited about this session coming up.

My unit has a lot of experience bringing together a wide range of data from COVID related programs being delivered right across the Government of Canada. We bring together the data to inform the Prime Minister and other senior decision makers and there is an apparent wide divergence in how different government entities are collecting, storing, and using the data collected during the delivery of not just COVID programs, but really all of our government programs.

But I think COVID has really brought this to the light.

In some cases data is really well organized and tells us a story about who was using the programs, the impact that the dollars are having on individuals, businesses, communities and allows us to make decisions and adjust as we go and other times we struggle to answer some of the simplest of questions.

Our data limitations can sometimes limit the options that we have available to us and how we deliver those programs. So also understanding the big picture can be a challenge with all of those differences taking place, the interoperability of the data can be quite different and we can't tell the big picture story across sectors, across communities about how these programs are being used and how effective they are for different people in all cases.

But we've also got many, many success stories where we've broken
through barriers and we've pulled data together in record time to make decisions. Sometimes on the back of a napkin and over the course of a weekend to spend billions of dollars. So it has been quite an accomplishment, so I can't wait to hear from our panellists and hear what they've learned from this natural experiment that we've all been undertaking, what their lessons learned are, and how we can apply that knowledge going forward.

So with that, I'm going to turn things over to Ima Okonny, from Employment and Social Development Canada.

[Ima pulls up a PowerPoint presentation.]

Mises à jour de la Dirigeante principale des donnés.

[A logo reads "Bureau de la dirigeante principale des données," beside an infinity sign. Below it are the words, "Il est temps de faire plus avec les données."]

So over to you, Ima.

Ima: Thank you Kara. Merci beaucoup.

I wanted to thank the School of Public Service. Thank you very much for your work in bringing us together.

Would also like to thank my fellow panellists: Statistics Canada, Canada Revenue Agency.

Over the last nine months, we've been working very hard together to integrate data, to really look at ways to help Canadians through this difficult time. So just expressing my thanks for the volume of work we've been able to achieve together and looking forward to the continued collaboration.

And thanks to Kara from a PCO perspective. You've helped to push us to produce more data to really build the evidence base in terms of helping answering some of those burning questions that PCO has put our way. So thank you, my colleagues.

So, one if the things I would really like to jump into from a CDO perspective in ESDC is just to introduce you to some of the work that has happened in the last few months to meet the challenge and the crisis that many Canadians across the country faced.

So, if we would jump on the next slide, please.

Developing the evidence base - The Benefit Knowledge Hub

[A flowchart labelled "Spectrum of Access to the Benefit Knowledge Hub" has three sections: on the left side, "Personally Identifiable Information;" in the middle, "De-identified Data;" and on the right, "Aggregate Data." These options flow into various sections]

ESDC Program Delivery
Public
Who uses the data?

[Arrows point down from these into a final single section.]

Evidence-Based Decision Making

One of the things that happened to us with COVID was there was an enormous exponential growth in the volume of questions we were receiving. People wanted to know how Canadians were doing. They wanted to know if we were able to reach people all across this country.

They wanted to know how women were doing. They wanted to know how youth were doing. People wanted to know how our seniors were doing and from a CDO perspective we acted quickly.

And what we did is we reached out to our colleagues in CRA and Statistics Canada in terms of looking at how we could really integrate the data we were producing. So we worked together to build an integrated database on beneficiaries within ESDC. And one of the things we learned from this work--

I mean, having the technical know how is important but having the ability to form those strategic relationships with key stakeholders was even more important.

For example, for us to really understand the operational context around beneficiary data was critical, so a lot of this work happened through, not just CDO effort, but we had to reach out, so we reached out to StatsCan, we reached out to CRA, reached out to operational colleagues to integrate this data.

And also to our privacy and legal colleagues to make sure that we were always keeping in mind the need to protect the confidentiality of Canadians.

As you guys all know, ESDC collects a lot of rich information on Canadians and we wanted to make sure that as we're putting together the evidence base to support the various needs that were coming our way, we took this into consideration.

So a lot of what we've done so far is to ensure that we applied some privacy by design principles in how we put together the database. Kara has already spoken to the issues around data integration, data standardization and one of the lessons that we really want to take out of this is the need for us to really look at how we standardize data across departments.

CERB, which is one of the big benefit programs that ESDC and CRA administered taught us that it was important to be able to integrate data. It's important to issue the cheques on time, but it was important for us to report on outcomes and one of the challenges that we faced was ensuring that the data was consistent, was of high quality, and was able to support policy design as things changed in a very quick manner.

Going forward in terms of the ESDC context, one of the things that we're looking at is how this can really feed into our transformation. As we look at transformation, as we look at digitization. As we look at shifting to digital transformation, the question, I think, for all of us who are going through transformation is how do we make sure that data is embedded and strongly positioned when we talk about digital, and interoperability. Next slide, please.

[The next slide.]

Élargir la base de preuves grâce à des initiatives de collaboration avec Statistique Canada.

Lacune : nécessité d'avoir l'accès à des données désagrégées détaillées sur les caractéristiques démographiques pour permettre une analyse multidimensionnelle portant sur les groupes vulnérables.

"Le BDPD a comblé cette lacune en : Œuvrant avec Statistique Canada pour améliorer stratégiquement les enquêtes afin de saisir l'impact de COVID-19 sur les Canadiens, par exemple en incorporant des indicateurs de statut de minorité visible dans l'Enquête sur la population active, les Rapid Stats out les groups d'experts sur the web"

"Collaborant avec le Lab d'innovation afin de s'appuyer stratégiquement sur le partenariat EDSC-StatCan dont un accent sur l'intégration horizontale des stratégies de données, l'alignement des plans de travail et le co-développement des processus de gouvernance"

"Œuvrant avec Statistique Canada pour permettre l'intégration des données du PCU aves les données de Statistique Canada. Il s'agissait notamment de permettre aux analystes du EDSC d'accéder aux données pour une analyse approfondie par l'intermédiaire du Centre fédéral de données de recherche (CDRF)"

"Mettant en place un Groupe de travail EDSC-Statistique Canada sur l'analyse des interventions en cas de pandémie qui a contribué à l'analyse des données sur la PCU, y compris sur les répercussions qu'elle a eues sur les groupes vulnérables et racialisés"

"Réouvrant le CFDR d'EDSC avec de nouvelles lignes directrices le 16 novembre 2020, et en travaillant à mettre en œuvre un nouveau modèle d'accès aux microdonnées, p. ex., accès tous les jours, 24 heures sur 24, au CFDR et ordinateurs portables à utiliser dans les immeubles d'EDSC pour accéder à des données moins sensibles"

"Mettant à l'essai un nouveau modèle de proposition pour permettre un accès rapide au microdonnées pour les demandes spéciales de politique."

Another critical issue that came our way was that people wanted to know what communities were facing.

So for example, we have different communities all across this country, we have racialized groups, we have vulnerable groups, and we were getting questions in terms of disaggregated data in terms of how people were doing and we could not immediately answer this question, as you know, because we do not collect data on race, we also do not collect a lot of disaggregated data on vulnerable groups, Indigenous Canadians, and other groups, so one of the challenges we had was to find a way to come up with a way of measuring this.

And this is where our Statistics Canada colleagues came in very handy. We've worked closely with them, we've worked with the CRA to look at sharing some of the beneficiary data with StatsCan and what this will enable us is to be able to really look at some of the impacts COVID-19 had on some populations and then look at how in future we can design policy that captures some of these needs and the impacts that we saw.

The challenge with this is that we're looking-- it's like a historical look to the past. So in terms of future, one of the things we would like to do is embed this thinking into our transformation. How can you make policy just in time using current data, using active data when you're in the middle of a crisis, and I know we will speak more to this as we continue the conversation.

But this is just a taste of some of what the Chief Data Office within ESDC has been working through. And looking forward to continuing the conversation in this. Passing it back to you, Kara.

[A new slide appears.]

Next Steps.

"Working on strategic partnerships and collaborating on data initiatives." Below it, a bullet point reads, "Identify opportunities, gaps, and priorities, and ensuring alignment and links between initiatives."

"Enhancing our enterprise data culture and maturity," followed by a bullet point reading, "Developing training with the College@ESDC."

"Refreshing ESDC's Data Strategy to reflect current, new, future and emerging needs."

"The pandemic exposes the need for timely and granular data and confirms that the department requires a centralized authority with a clear data strategy."

"The next version emphasizes our need to fill gaps and ensure we have the required analytics capacity."

Kara appears in the call.

Kara: Thank you very much, Ima. I think we probably will get into many more of those questions as this discussion continues today. So, next I will turn it over to Jean-Francois Ruel...

[A PowerPoint's title slide has a blue maple leaf on a blue background.]

Designing and Delivering Modern, Data-Driven Services in Times of COVID.

"Executive Panel. December 16, 2020 meeting. Canada Revenue Agency"

...Director General at the Canada Revenue Agency for his presentation.

[Jean-Francois Ruel appears on the call.]

JF: It is interesting to see to what extent operational data versus statistical data has had a significant impact on the work we have done lately with operational data.

[The next slide.]

"Répercussions et résultats sur les données de la COVID-19 – Agence."

"Demande accrue de données, de statistiques et d'analyses à l'interne (pour la reprise des activités) et à l'externe (pour appuyer la politique fiscale)."

"Surveillance et établissement de rapports (horaires, quotidiens, hebdomadaires et mensuels) = Paiements uniques/complètements pour le PCU, PCUE, SSUC, PCRE/PCMRE/PCREPA, SUCL, ACE, CIPH, et crédit pour la TPS/TVH."

"Tableaux de bord organisationnels (internes et externes)," with the sub-points "Prestations d'urgence administrées par l'Agence" and "Données sur les services internes (cas de la COVID-19 parmi les employés de l'Agence, capacité technique, répartition et disponibilité de l'effectif, etc.)"

"Relations interministérielles." The next point is, "Augmentation de la capacité de recherche (à court terme) et d'analyse, y compris l'utilisation de mesures de communication,"

"L'Agence a géré efficacement les volumes d'appels et de connexions à Mon dossier grâce à l'application de techniques d'introspection comportementale."

"Transparence accrue, publications statistiques et gouvernement ouvert."

We were asked to quickly put data on the different networks and we could not work on the quality of the data in the same way as Statistics Canada could, so we informed Canadians to be transparent, while being aware that there were certain limits to the teams. All this to say that I will start with the presentation.

I'll just go very quickly about COVID, the new environment, and its impact a little bit on the agency. We are a tax agency and our job is about collecting tax, but our mandate kind of shifted or was complemented, I would say, with COVID measures that we had to put in place The CERB, the queues, and all those other measures we had to kind of move very quickly to be able to put those in place. That was our reality over the summer.

And we had to also create some internal reporting. So we called the dashboards but the les tableaux d'abord so we were able to present very quickly some cuts of the data and say "oh decision makers, here's what it looks like" and we were able to move that way. Also, you know, to work in our... [stammering]

Play and use our networks with different other departments And Ima mentioned it a little bit maybe the most potent or the most relevant example is the CERB which was co-managed between ESDC and CRA so we each had our little portion, so it required a lot of coordination and ensuring that we knew what the other was doing so that there was no duplication.

But there was also an alignment of our, of our objectives. What we did also is we augmented our research capacity on that topic. There's an example of-- I don't know if you remember. It seems it's an eternity now, but when we launched the first program, we said "If your birth date is from January to February/March, then you go on Monday, and Tuesday, and Wednesday and Friday, just to kind of...

...being able to push a little bit the curve of when people would apply and there was some communication aspects that were developed to be able to do that. And we were able to measure the efficiency of it.

And you know, if people registered on the Tuesday did they continue to register on a Tuesday afterwards? So it was, it was great.

[The next slide.]

"Theme 1 - People and Remote Work."

[Under the header "Challenges"]

• Reduced capacity
• Time required to set up employee workspaces to support productivity
• Employee health and well being, work-like balance
• Communicating with employees - tools and processes required to replace traditional ways of working and communicating.

[Under the next header, "Successes,"]

• Availability and flexible hours of work
• Employees adaptability
• Collaboration to support benefit programs
• Staff for areas whose workload on hold helping those tasked with the increased workload.

[In an infographic beside it, a person works at home with a laptop on their knees and a cat by their side. The caption is "Working remotely before, during and after the pandemic."]

"Proportion of businesses that reported that 10% or more of their workforce was working remotely: Prior to February 1, 2020, 16.6%; On May 29, 2020: 32.6%." More text explains, "22.5 of businesses expect that 10% or more of their workforce will continue to work remotely once the pandemic is over."

So, I'll very quickly, because I wanted to leave room for the questions, but I'll just speak to people process technology, the kind of the big three teams that are always there when we move forward. Employees, like everyone else, I guess, in the different departments, we had a reduced capacity, it took us a little bit of time before we could allow people to have access to their computer tools, a lot of people, among others, people who worked on the data needed more powerful computers, so they didn't necessarily have laptops, so we had to work on that.

There was the work-life balance versus work and communication. In the beginning we had, for example, MS Teams, so it was a little more complicated because it was the reality we all had to live through, which helped us, however, we had employees who were flexible in terms of hours and work methods, and then there was also moving people from people who found themselves with a reduced workload because the priorities had been loaded and could be moved and then work with others who-- or their workloads had been increased so there was flexibility in the movement of people as well. Second team technology...

I'll just say on this one what helped us...

[The next slide.]

"Thème 2 - La technologie."

[The word "Défis" is followed by these bullet points.]

• Accès aux systèmes pour les données et l'analytique avancée
• Interopérabilité - Obstacles à l'échange et à l'acquisition de données
• Bande passante limitée et utilisation limitée du réseau
• Ressources de la TI occupées avec la préparation et l'établissement du calendrier de la période de production des déclarations de revenus a l'approche celle-ci
• Travailler à distance sur des renseignements de nature délicate, protégés ou classifiés/accès à distance protégé (ADP)
• Matériel non adapté à l'environnement de travail (principalement les ordinateurs de bureau).

Below is the word "Réussites," followed by the bullet points: "Accroître progressivement l'accès au réseau jusqu'à atteindre presque la pleine capacité," "Collaboration au sein des équipes/utilisation d'outils numériques pour collaborer," "Accroissement rapide du matériel portatif," and "Acquisition et accessibilité des données."]

...at first, these are challenges, but you know, interoperability, integration, data integration, and the fact that the type of questions that were asked, were not necessarily the type of questions we had data for, all of those were, kind of, in play. Even the throughput, like at first we had to-- we could only work in the morning and not in the afternoon and vice versa depending on which region we were in Canada because we're a pan Canadian organization.

But we-- what we had is we had made investment in creating a, kind of, a hub of data for BI and analytics which helped us-- it centralized our capacity to be able to do the analysis, and I can speak to that later on in terms of the Q&A's, but it created a data lake, but it's really a govern environment for access to data to do analytics, and it allowed us to move very quickly and being able to do analysis when the new datasets came in for CERB, for queues, for all of those things.

And the last piece is the process. Theme Three. I don't know if we can switch-- turn the page...

[Slide three]

Theme 3 – Processes

[Under the header "Challenges"]

• "Various levels of verification and approvals within accelerated timeframes
• Increased reporting requirements internally and externally (PCO, TBS, PMO, etc)
• Different mandates and accountabilities/multilateral engagement
• Processing and approval processes not adapted to remote environment
• Quality vs. Timeliness
• Reporting: Evolution of stakeholder needs."

[Under the header "Successes"]

• Business continuity plan implemented early with focus on critical services for CRA and external partners
• Chief Data Officer governance and oversight
• Working together/breaking solos both within the Agency and with other federal partners
• Access to decision makers
• Leaner/agile processes and more efficient communication channels

Yup. Uh, so... Our traditional way of getting approval was various stage and very, like, we would-- mailbox type of thing. And then we waited, so we had to re-engineer some of our process to be able to deliver more quickly, which was good. It kind of shook the box a little bit and say, "OK what is needed and how can we kind of minimize the steps."

We increased our reporting also to be able to answer questions proactively, so put data out there so that people see it. So, a lot of data has been published on open data and canada.ca, but also we were reporting in support of PCO, TBS and so forth. We had also a business continuity plan, so we've embedded that and maybe that's the thing that I want to mention here, the role of the Chief Data Officer for the governance and oversight was key.

 Just to give you a sense, data was deemed-- like, data provision, in terms of-- was deemed an essential activity for our agency. So when we went to COVID, we streamlined to only to the essential activities that the agency needed to do. And data provision was-- data provision for decision makers was deemed essential activities and that was the first time before, like, in the other cycles it was never identified as quickly as such.

So it just shows how important those things are. So I think that covers in terms of it. Let's just go to the next slide, please.

[The next slide]

Leçons apprises et prochaines étapes

"La communication des données est principalement centralisée = statistiques uniformes et fiables."

"La fonction de dirigeante principale des données fait partie intégrante de la surveillance," then "Les communautés de données fédérales fournissent d'importants renseignements."

"Utilisation descendante des données - éclairer la prise de décisions à mesure que les activités opérationnelles reprennent," and lastly, "La mise en œuvre de l'infrastructure de l'intelligence d'affaires renouvelée a grandement facilité la capacité de produire rapidement des analyses."

Coordination was key. The aspect of having access to a new environment for us like, kind of a data lake was great in terms of moving quickly on some of that reporting components. Like, those relationships between the different departments were key also in terms of ensuring alignment and being able to report to Canadians what was going on. So that's it for me.

[Kara Beckles returns on screen.]

Kara: Merci Jean-Francois And to our next presenter, Tom Dufour of Statistics Canada. Tom?

[Tom Dufour's video pops up on the Zoom call. A new PowerPoint appears.]

Conception et prestation de services modernes fondés sur les données en période de COVID.

Tom: Hello, and thank you, Kara. Thank you first of all to the School of Public Service for inviting me here today. And it is a real pleasure, especially with Ima, Jean-François, and Kara, who have also been very important colleagues for us over the past ten months.

I'm Tom Dufour, I'm the Director General of Strategic Data Management at Statistics Canada. Just to give a little bit of context, my branch includes the Centre for Statistical and Data Standards. It includes the Office on Privacy Management, the Statistical Geomatic Centre and the newly created Office of the CDO at StatsCan.

You can see this branch provides a lot of the key infrastructures and stewardship services across our agency. It doesn't need to be said again, but clearly the response to COVID has been data driven and it really has demonstrated the value and the use...

[A pyramid is titled "How to use the power of data to respond to pandemics & other emerging societal challenges." The first and lowest tier is "Find and gather." Tier two is "Protect and clean." Third is "Describe, analyse, and model," and the fourth and top tier is "Tell the story."]

...of data for Canadians. It shed a light on many statistical challenges, I would say, not only for Statistics Canada, but we've all faced over the last stretch. It's certainly highlighted the broader need for high quality statistical information and a strategy to address, sort of, nationwide health issues, economic issues, and socioeconomic inequities as well.

The pandemic made us change as an organization and we really had to adapt as a result, we really had to, sort of, go after more timely information, more disaggregated information. Granularity was something that we often heard of and always, always, always at the same time, protecting privacy and maintaining trust. The first slide that you see in front of you is a fairly generic framework and I won't spend a lot of time on it, but there were a couple of points that I wanted to maybe raise.

First of all, when you connect the information and then it really doesn't depend on whether it's ancillary data acquisition or whether it's doing surveys.

We at Statistics Canada have adopted a framework recognized as a necessity, an important principle that we have adopted and then really to be able to make sure that we have done this reflection before we acquire data.

And finally, Ima has made the point, the protection of information relevant to privacy must be there and is an important part of it right from the start.

[The next slide.]

Un système de données solide permettant de lutter contre la COVID-19.

[A chart's title reads, "Éthique, respect de la vie privée et transparence." Below it are the words, "Lignes directrices sur les métadonnées, les données normalisées ainsi que la protection et la qualité des données."

This is followed by, "Soutien à l'infrastructure - Regroupement et facilité - Données de base et analyse." Below this is a table with four sections side by side, each with a description below it. They range in colour, starting at pale blue and ending at dark blue. The titles of each section are "Recueillir," "Protéger," "Améliorer," and "Offrir."]

Achieving fair principle. So we talk about FAIR, and I'm sure a lot of you know about that, FAIR, making sure that data is Findable, Accessible, Interoperable, and Reusable, is that desired end state.

And not only for us at an agency level, but well beyond that, across the whole data ecosystem. The ability to link and integrate social, economic, environmental data is the key to deriving meaningful insights. And again, another, sort of, important thing that Ima pointed out and Jean Francois also made reference to, this is a really important piece of information when we're starting to build these systems.

And finally, you know, maintaining public trust. Making sure that statistics are produced in a neutral way. Objective methods, and that the output is successful.

Everyone is very important. From the beginning of the pandemic our approach was quite simple. We really wanted to be of service and then as a data organization we felt that we could really play an important role. Right from the beginning of the pandemic we scrambled a little bit to not only continue to produce what we call our mission mission critical programs.

And some of the official statistics like the Labour Force Survey, the CPI, Gross Domestic Product were continued in that we didn't have any sort of delays on that, but we also had to address the need for different data and new information due to the pandemic.

Again, we've evolved from not only being a data producer, but we've also moved more into becoming a data steward. This has been sort of part of our modernization journey as an agency over the last 2 ½ years, and it served us well in this context. I'm really proud to say that we are able to supply data to meet critical needs, but that we are also there for a lot of our partners and stakeholders to be able to support them.

What you see across on the screen right there is-- you'll recognize as a standard stewardship framework. The 4G framework and, you know, just to point out a few of the little pieces. These are sort of examples how Statistics Canada reacted to be able to go in to meet urgent needs using innovative collection methods to advance collaborative work spaces, and to put data in the hands of partners and researchers, and to be able to make sure that we use new products, new visualization tools again, we've had a lot of success working with partners like NRCan, Public Safety to be able to do a lot of visualization tools, mapping using Geo-enabled data.

One of our first big challenges was to be able to put Geo-enabled health data and other socioeconomic data in a way that could be used by epidemiologists at PHAC. So again, these products were-- were sort of moved forward very quickly to meet some of these urgent needs.

[The next slide.]

PPE Supply Chain Modeling - Services Provided by StatCan (procurement, supply, demand & distribution)

[The center of the slide has an image of a handshake, above the words "Partnership agreements between StatCan and: Health Canada; Public Heath Agency of Canada; Public Services and Procurement Canada; Innovation, Science, and Economic Development Canada. StatCan stewarding data in central repository on our infrastructure."

To the left are five lines, leading to various points.]

• Cloud services and Protected B certification
• Statement of Sensitivity (SOS) for Protected B data
• Processing and reconciling daily data inputs
• Managing and iterating epidemiological and demand models
• Performing quality assurance and quality checks of model outputs

[On the right side are five more lines, leading to the following points.]

• Reporting via dashboards (data visualization); developing mobile-friendly view
• Providing user-access and user-support assistance (on-call 24-7)
• Providing analytical support on on-call basis
• Conducting bi-monthly PPE survey
• Working with partners on PPE Data Strategy

The final slide is a little bit of an example if you will, something a little bit more concrete of a strategic partnership and the work that we did on the PPE front. So it really started with a need to track the supplies of personal protective equipment for the country.

So you can imagine from that you're trying to assemble and present to decision makers very quickly a comprehensive picture of PPE procurement, whether it be at the provincial, by the provinces, by the federal government who is coordinating a lot of it, all the data obtained through various sources, different formats, different classifications, and try to put that in a way very quickly that we were able to sort of meet those needs to be able to, you'll imagine you could you go back to the beginning of the pandemic when we were trying to reopen the economy, the discussion on PPEs dominated the news.

It was on everyone's mind and it was a critical piece of information that we are putting together. But it also involved there and it moved into trying to model the supply-- not only track the supply, but model the demand. So again you do that through models, bring it together, large number of existing data by industry occupation and others. We had to put a survey in place. We use different collection methods with data scientists and stuff so all that to say we were able to sort of meet that. So one good example I wanted to bring. Back to you, Kara thank you.

[Kara Beckles appears on the call.]

Kara: Thank you very much Tom. I can definitely see the importance of your example around PPE and as time continues I'm sure the next challenge will be around vaccine delivery and tracking data and information around that. We can see questions on that already coming our way.

So thank you to all of the presenters for your speedy presentations. I'm sure you could have all taken twice or three times the amount of time, but what we're going to do now we're going to launch into a few questions and a bit of a discussion, but I encourage all the participants to, if you look at the meeting chat, there is a link to Wooclap. So if you have questions, please place them there and will get to your questions very shortly.

So, my first question is, "What lessons can we learn from COVID for the design and implementation of modern programs?" Both in terms of launching programs during an emergency like COVID, but also as we move more into normal times and have a bit more time to think through implementation for this question. I'm going to start with JF, JF over to you.

[Jean-Francois Ruel's video turns on.]

JF: There's always a little bit of a lag between during the time that you want to talk in the time you really talk, so thank you so much. So what lessons can we learn from COVID? I would maybe modulate it around 3 three component or three drivers.

The first one is that there's a value in moving quickly. Putting the data there and having it seen and letting people put their hands on the information. Be transparent about what the data means.

Moving it quickly means sometimes that you have to do compromise. And all that aspect of communication so very, very briefly speak to all of them, and Ima and Tom also mentioned it is, all of this is done in-- while ensuring and this is kind of the first priority that privacy and confidentiality is preserved, so all of our system and our process are build into ensuring that this data is used in the proper way, so I'll just go to my notes.

What we realized very quickly is that, you know, and I mentioned it earlier and I think Ima also is that there's an appetite for this information and to get it in near real time basically and this pressure. So one of the lessons that we've learned is that this pressure needs to be balanced with communication.

So we need to know who should provide what data but be responsive because as soon as we started providing data if we were not responsive enough, people would knock on other doors and try to get the data. So it was kind of that coordination and having that central hub helped a lot. And what helped is getting that information public very quickly and-- and this is where I want to maybe do a little bit of a nuance. We're data people are all on this call, so it's maybe it's easier for us to have that discussion, but.

The operational data is different than the statistical data. So what we are as an agency we're putting out is this is the data as of today and it will change tomorrow and it will change the day after and that's what I meant about being transparent and completing the risk around the data. So if someone takes a snapshot of the data but reports a month after that, the data will be stale.

While you know, Statistics Canada takes more time to review the data and create a more statistical sense, our data won't always align as we move forward. This is one of the challenges that as we move forward, we want to make sure that if there is difference that we can explain them. But the appetite for data and then going out there. It's a risk that we wouldn't have taken before, but now we did. And I'm glad we did.

It means that the information is out there more quickly, with the proper caveat around them. The delivery also is kind of a key component. We did deploy in stages and I don't want to get technical, but there was waves of development on the IT side as we would producing the data.

So what we've learned is that agile aspect was instrumental in us being able to deliver some of those and they're not in my domain and they're not in the area that I work, but you've seen from a government perspective, being able to deploy solutions around the benefits for the entire Canadian population was quite a challenge and from a technology point of view also it was something that we've learned through and lessons that will build it to future release.

So, that's what I have to say.

[Kara Beckles appears on the screen.]

Kara: Merci JF, so Ima, Tom I want to give you also a chance to pipe in here. Let's start with Ima.

[Tom Dufour and Ima Okonny appear on the call.]

Ima: I just want to jump in on what Jean-Francois said about the operational context. One of the things we found with dealing with some of the questions we are getting around COVID is that context is key. Context is so important because there were decisions being made that impacted Canadians, like, as we spoke like even tweaking the benefit programs required them to look at the data and do some policy analysis around the data.

So looking at the differences between operational data versus the more integrated massaged data was key. The other thing I wanted to point out that we dealt with was the issue around legislation, privacy, regulation and how that became a barrier for us to be transparent to Canadians.

We will get requests for data and we had challenges sharing some of this data with the provinces and territories just because of the way our legislation is written. So the challenge for all of us in the data world is how do we really drive transparency through data if we're not fixing the issues with our legislation.

On the surface, we were all collaborating really well and CRA, I mean, Jean-Francois was like the excellent CDO partner anybody could-- could dream of, but the challenges that the ITA Act stopped us in doing some of the work we needed to do in terms of data sharing and being more open and transparent to the public.

 The other thing that I learned from this was that in terms of the CDO role, I think having a CDO in a lot of the line operational departments is extremely, extremely important.

I'll give you examples. Like, for example, there were cases where I got urgent calls from PCO, and Kara, this was before you even took the role, urgent calls and Jean Francois was giving him calls at 7 o'clock, 8 o'clock, clarifying the context around the data, so we're able to send some of this data to PCO. Having that person that you can reach out to who understands the context we were working in was extremely important.

The operational context is important too but having somebody who could understand the language around data. And could provide me with the required caveats to respond. I think it's very important and moving forward, if we could feed some of these lessons to other organizations and other departments that are still struggling in putting together the CDO role, I think there's a lot we can really bring to that conversation around some of the concrete lessons that we have learned with dealing with COVID.

The other thing I wanted to really emphasize on is the importance of partnerships. I've spoken about collaboration, but partnerships, co-creation, is extremely important and some of the work that we're doing with Statistics Canada in terms of leveraging data in a more strategic, co-creation type of relationship.

I would say it's a game changer because within ESDC we are a department of over 20,000 people and you have people who have the policy context, the operational context, the program context and the challenge is bringing everybody together so that we're able to leverage data and what some of what we did immediately was bringing in our policy program and operational experts to sit down and have conversations with StatCan and Chief Data Office and what we found is this kind of co-creation really brings more value in terms of what the some of the insight we can get out of data.

I could speak for hours about this, but I know we don't have much time, so I'll pass it back to you, Kara.

Kara: Thanks so much, Ima, and I have to say from a central agency point of view, I completely agree with your comment about the value of having a CDO and having that one person to go and contact about the data instead of, as Jean Francois mentioned, you know, trying different doors within a department to try to get to the information that that you're looking for.

It's just such a value to the organization.

Tom, I'll turn it over to you for any thoughts you have on this topic.

Tom: I'll be really quick, I agree with what Jean-Francois and Ima were saying and to underscore what Ima was saying, we often say that data is a team sport and those partnerships and being able to sort of work towards a whole of government, even sort of across governments, to be able to sort of approach things and work together collaboratively.

We're Better Together, and we're working Better Together than separately, so I think that's one of the real key points that we need to take away.

Kara: Great. Thanks, Tom.

So we'll move on to the next question that I have. What are the key features of a modern data-driven program?

I think we've already touched on several of these in the presentations and even in the last question, but Tom, maybe we'll start with you, coming from an organization that maybe doesn't deliver programs directly to Canadians, but is definitely vital in pulling the data together and telling those stories, so why don't we start with you?

Tom: Thanks, Kara. This is really interesting and I think when I hear that question the first thing I sort of go back to is that we really need if you're talking about a sort of a modern, data-driven program you need to start back on those discussions about making a commitment, making an organizational commitment towards using data as a strategic asset, and that you really sort of start from that basis and say well, how do we kind of go and improve that and to me, I always come back to those FAIR principles.

Findable, Accessible, Interoperable and Reusable, and I think, sort of you get to that state, right, where if you don't know what's out there, if you can't access it, you don't know what it means.

Using different classifications and other, you're not going to be able to get what you need out of that to be able to take full advantage, so I think, you know, that commitment to move towards that and again, an alignment with standards and open standards where possible to be able to sort of get that real robust metadata to really understand what you're talking about and what you're using, we're all moving towards digital, making sure we move maybe towards modern digital to be able to sort of, you know, right from the beginning and the importance of stuff like data cataloguing and other things like that where you can really be able to sort of get to that ability to sort of use that data more importantly.

I mentioned it earlier to linkages and data linkage platforms and collaborative work spaces. Secure Collaborative work spaces is really important as well. As one of the things we've seen, visualization and dissemination tools and really being able to use again, I think we've gone really forward with a lot of the sort of geospatial work that's been done. And when you present this to data users, you know, that a picture is worth 1,000 words, kind of pops up and the importance of being able to do that.

And now you know whether it be for looking at COVID cases, whether it be to look at infrastructure investments, whatever it is. But when you can overlay that into understanding a little bit more deeply through visualizations, I think that's important.

And finally, you know I have to say, you have to use the right frameworks. You have to make sure you have a data quality framework that guides your work. The legislative framework Ima referred to and Jean-Francois we all know well. We need to make sure we respect those, but we also need to find ways to work together. So that's what I would say is as an opening statement on that question.

Kara: Ima do you have anything that you'd like to add to Tom's remarks here?

Ima: I mean I, I really liked what Tom said about standards, we have to find a way to solve this standard problem within the Government of Canada we are all collecting, many of us collect the same data in so many different formats in so many different ways and one of the challenges is integrating this data.

So if we could have standards around address information, for example, to ensure that we capture the right data we need to answer some of these questions that coming our way. I think it will put us so much further ahead, so I really wanted to zero in on that. The other point I wanted to also raise is a way for us to work more integrated with the provinces and territories and even with some of the cities because what I found is the work we do and the people we serve, it's all interrelated and at the end of the day we're serving individuals.

So if we could look at a way of having a way to manage information around people as opposed to around just programs and then would be able to better see the cross section and intersectionality and impacts of some of our policy and programs on people and people we serve and then design better outcomes for Canadians.

So one of the key points for me is as I think of transformation, digitization, all these beautiful words that we're saying, it would be very powerful for us to look at how we can fix the-- the standardization problem, the data standard, the data quality, metadata management, master data management. Those might not be as fun to talk about like some of the other things, but I think it's important and it's work that we need to look at doing as the Government of Canada.

Thank you, Kara.

Kara: Thank you, last word to you, JF, on this topic.

JF: It's nice, when you go last there's always less to say which is great. I agree totally with Ima and Tom. It's nice I would say this group of about 150 on this call is probably a group that can understand what the value of standardization and this is hard to communicate.

I would say it's one of the challenges that I surely I have in my organization in terms of demonstrating the value, but I think a modern data environment that informs decision making needs to be able to do that because there's so much more value that you can do by putting side-by-side datasets.

The only one thing that I want to add, maybe that's a little bit different is we are looking at information more broadly, like the notion of semi-structured and unstructured information, and how we can bring it into the same framework as our data management in terms of metadata in terms of standardization, because more and more tools go and take advantage of that type of information also.

For us it's a challenge in terms of how can we make the best out of that information, that is not always coming nice columns and rows and that we need to leverage and using our operations. That's it.

[Kara's mouth moves silently.]

Kara: Thank you! Alright, so I have more questions but I don't want to run out of time for questions from those of you who are listening in.

JF: Maybe I just forgot to push one point is that standardization is something for all of us, so everyone that's on the call, you need to be at the table and to talk, because if in the end we end up with somewhere where it doesn't feed the needs, you know we need to have that discussion and you know [inaudible] and DG data lead is one of the forums where we meet and talk about those things so be engaged and participate. I would like for me it's I always get some new aspect that I need to think of when I go through to those meetings.

Kara: No, no, no, I don't think standardization can be stressed enough in terms of making the best use of the data that's collected right across the government. So, maybe we'll turn to-- I think that we're gonna pop up on the screen some of the questions that have come through the Wooclap.

[A list of participant questions appear on the screen.]

There we go. Excellent. OK. So we'll go through these and feel free, any of our panellists, to jump in, whoever wants to answer first. But let's start at the top.

"When designing what steps can be minimized or eliminated for the purpose of rapidly responding? So this was one of my questions as well. Is that we've delivered COVID programs really quickly. In some cases from thinking of what had to go out to actual implementation happening in mere days or weeks.

Who would like to respond to this question first?

JF: I can put an angle, but I think I've mentioned this already so I apologize if I repeat. One of the steps that we have-- I'll talk about the public release of information.

So the release, for example on canada.ca, we had a program for that. Every year we would publish data and it would be sanitized, analyzed, DQA'd, data quality assessments to make sure that the data that we would push out was, you know, "nickel" and that year over year you could do comparison because it was always the same cut-off date.

Now what we've been asked to do is extract the data. And put it out there and renew it very quickly. So it means that we-- that's the balance between quality and timeliness, so it-- it shortcutted some of the data quality assurance that we would do typically before we publish on the website. And then we balance that by putting caveats at the bottom and saying OK, this data is subject to change, you know, come back next week and you'll see how much it has moved.

But this is one of the aspects that communication helps us palliate for some aspects of some of that DQA that we would do this initially that that we are not necessarily able to do in a timely fashion to be able to respond to the demand for data.

Tom: I'll add something here, Kara.

Again, Jean-Francois also puts up a great point, and, you know, when you talk about, you know fit for use in what you're looking at. So, just to-- I don't know that I would actually, kind of-- I twist this a little bit and say, you know, sometimes it's a matter of using more modern methods and to be able to maybe change your collection method. So to give you an example, when Statistics Canada builds the GDP, the quarterly GDP official statistic.

Well, that usually comes out 60 days after the reference period when you need to know in the context of COVID what's going on today. 60 days is past the reference period is a little bit different, so then that's when you've got to sort of say, OK, there's got to be a different way for me to get something meaningful now, so whether it be we've talked about Flash estimates to be able to provide something so, you know, built with partial data sources, maybe some modelling in there to be able to get to something.

So I think it's not as much I would say minimized or eliminated, I would say just using more modern methods and maybe different collection methods.

Ima: And I agree with you, with Tom, because I think the challenges were very reactive.

It's more being more intentional about how we're doing data collection, how we're doing policy design, how I think it's more around the how then as opposed to eliminating things because, OK, look at privacy, for example, if we could formulate a better way of having privacy by design, that will eliminate the need for us to go through a lot of cumbersome steps like consulting with legal, consulting with privacy, consulting with five different people, with five different briefing notes before we can publish data.

So I think part of it is being more intentional at the beginning, like when we've been thinking about program design, when we're thinking about policy-making when we're thinking about digitization, transformation, we should be having these conversations now, as opposed to when there is a crisis, and then we're reacting because the challenge is that when you are reacting, you-- what happens is you try to cut corners or get stuff done really quickly and, as we know, we have the famous audits that come back.

And hindsight is always 2020 so I think part of it is us being more intentional in how we look at data collection. Breaking down some of the silos like the silos when we're not in a crisis situation. It would be good for us to have conversations with StatCan, other departments with PTs when we're not forced to.

So that this this all this makes us save time when there's a crisis, and I think this is one of the big lessons coming out of with COVID-19 in terms of forming those relationships, building the processes upfront as opposed to waiting for something to happen and then be forced into that situation.

Tom: Thank you.

Kara: Thank you, I think this next question sort of touches on what you were just talking about, Ima.

Is collaboration with other departments and agencies easier these days?

We know that with things having to happen quickly, sometimes some of those natural silos have been broken down, but, Ima, given your remarks just now, maybe we'll start with you.

Ima: I would talk to JF even if there was not a crisis but I would say that the crisis forced the regular conversation. It also forced the need to really look at how we could get PT the data they needed.

It forced the need for me to really scan what's going on with the city in terms of some of the data collection. But what we're trying to do within the Chief Data Office of ESDC is to have a more intentional road map with Statistics Canada in terms of how we collaborate and build these strategic partnerships going forward.

I'm very lucky to have a really good liaison in StatCan in terms of how we work, so we're building a format where we'll be doing a lot of co-creation and this will feed some of our collaboration in the future and then in terms of partnerships with researchers, right now we're looking at some of the work the CRDCN does in terms of how can we work with researchers in terms of being more proactive in terms of policy making.

So I think one of the big lessons coming out of this is we just need to be more intentional and proactive around how we do policy design, how we do data collection, so that we're ready for the next crisis. Thank you. JF, something to add?

JF: There's nothing like a good crisis. I guess that's basically it. It forced us to look at ways to prioritize demands also because by discussing with Tom or discussing with Ima and others around government, the appetite for data was so big that it kind of brokered that discussion and it forced us to have those discussions and create that network of kind of emergency need on data and being able to respond.

So naturally that has taken place and we were very quick to bring people in that we needed, in terms of the expertise, from a legal perspective, from a privacy perspective, from even expertise on the subject matter expert.

 For us, it's tax data. So tax experts. To make sure that the data is well protected and that we follow all the rules, but to the extent that we can we make it available in a format that allows for all partners to be able to use it and leverage it and make decisions out of it.

So, we had to be a bit, like, we had to be nimble in that way and do more of things that we were doing on an ad hoc basis became more of our core business to a certain extent in terms of that relationship with others.

Kara: Tom, anything else?

Tom: Maybe just really quickly. I think it is easier now. I think it is better and I think there's still room to improve there's still, sort of, silos and issues, but I think we're doing better.

Kara: Great. So I really like this next question.

Do you look for help from expert advice in public health to be able to interpret data?

And in my mind this isn't just public health, but I think it's expert advice on any kind of a data, and I know from personal experience in preparing stock takes for the Prime Minister where we're pulling in data from departments all over the GC, we always like to take our visualizations back to those departments and make sure, is the story we're telling actually accurate. Have we interpreted and understood the data properly?

So, Tom, maybe we'll go to you first, StatCan's business is collecting data.

[Tom silences his phone.]

Tom: Yeah, absolutely. And in fact, it all comes down to partnership and being able to support and have roles. As-- it's interesting, this one, because I was surprised when we started I didn't know that Statistics Canada had 10 epidemiologists on staff. I knew that we had a lot of experts in our health analysis division and stuff like that.

You need the experts and in fact one of the first things that we were challenged with was really putting as much data and again, particularly in the case of the geo-enabled data into the hands of epidemiologists for us to do that work, and give them that information and then to be able to sort of let them do their work on that, I think that's a huge and important key part of this whole thing.

Kara: Alright, really quickly, Ima?

Ima: I totally agree with Tom. Within ESDC, we're dealing with the complexity of the EI program, and without that expertise from the operational policy and program perspective, you're just producing information that might not really be making sense. I think one of the things even within CDO that we're pushing is the whole concept of data literacy across the organization.

One of the things we really emphasize is that yes, it's good to have access to data, however, given that we're serving Canadians, it's important that we properly contextualize the data to make sure that we're telling the right story.

We're also applying the data ethics lens, so I mean bringing in the professionals, the expertise that you need to be able to tell the right story at the right time.

I think is extremely important, and it's one of the lessons that came out for all of us as we looked at in newspapers and saw contradicting information around COVID and the numbers, big lessons learned there.

Kara: JF, last 20 seconds to you.

JF: I'll just mention that, for example, the expertise that ESDC had was key, we spent a bunch of the back and forth in terms of creating notes at the bottom of our tables, and for that, because the program was-- like the data was captured by us, but the program was ESDC's, we were creating that relationship to ensure that the data was interpreted in the right way, so we had that.

You know, this is the data. What do you think? This is a note and then back and forth to make sure that.

As we put it out in the public, the expert have weighed in terms of how it should be read, in terms of a table, so like it gives you a concrete example of how that plays out.

[Six more questions are listed on the screen.]

Kara: Great, thank you. I think we probably could have kept the session going for another hour with all the questions we see on the screen, but unfortunately, we are out of time so I'll turn it back to Martha to close this off.

[Martha turns her video on.]

Martha: Thank you so much and I just-- I'm so grateful to you all for being here today and someone just reminded me if we were in person right now there would be a round of applause.

So on behalf of myself, the GC Data Community and people online, just know that there's a virtual round of applause for all of you for joining us today.

I would also like to note that in terms of there's so much more that we can discuss and IRCC, CSPS are working so hard to put together this year's fifth annual data conference that's coming up in February, so I encourage everyone online to have a look at the CSPS events page.

Registration is available now, and it's going to bring these types of discussions to another level as well, so please check that out on our events page and thank you so much for joining us today and wishing you all a safe and happy holiday.

Thank you.

Tom: Thank you, Martha.

[The panelists wave.]

Kara: Thank you.

[The animated white Canada School of Public Service logo appears on a purple background. Pages turn, closing a book. A maple leaf appears in the middle of the book that also resembles a flag, with curvy lines beneath.]

canada.ca/school.

[The government of Canada logo appears: the word "Canada" with a small waving Canadian flag over the final "a."

The screen fades to black.]