Government of Canada Data Competency Framework (DDN3‑J03)

1.1.7 Maintains awareness of broad data trends and their potential impact on the Government of Canada.

1.1.8 Uses the services, resources and support available through Centres of expertise (Chief Data Office, Data Standards/Governance Division, etc.) and other trusted sources.

1.1.9 Understands complex data, statistical, and analytical concepts.

1.1.10 Knowledge of the data that exist within the organization and how they are being used to inform or support decision making.

1.1.11 Works in a multidisciplinary partnership with core partners such as Information Management, Digital, Information Technology, privacy and security, legal, subject matter experts and end user.

1.1.12 Enables colleagues to use key data terms when speaking about and working with data.

1.1.13 Leads by example and contributes to building and enabling a data literate workforce and a data-driven culture by encouraging the use of data within home organization and across government.

1.1.14 Identifies and removes roadblocks so others can work at their most efficient capacity around data (e.g., ensuring access to training, infrastructure and software).

1.1.15 Defines and implements strategies to advance the talent, systems and infrastructure needed to improve organizational data literacy.

1.2.3 Knows how to protect and share confidential data.

1.2.4 Understands the ethical implications of using AI.

1.2.5 Identifies indicators of bias and ensures policy, programs, or services do not reinforce unintended biases.

1.2.6 Ensure data are used in alignment with their intended purpose by consulting with data owners or stewards.

1.2.7 Applies processes and procedures to ensure ethical approaches to research and data throughout the data life cycle.

1.2.8 Identifies ethical issues, privacy/security implications and barriers to accessibility.

1.2.9 Understands advanced concepts such as necessity, proportionality, sensitivity, and explainability.

1.2.10 Assesses data for bias, representation, accuracy, and validity. Identifies and implements steps to resolve issues if needed.  

1.2.11 Tests ideas and potential solutions with a wide range of diverse data to challenge assumptions and ensure solutions that are inclusive by design.

1.2.12 Designs and implements policies, processes and procedures to ensure ethical approaches to research and data use.

1.3.4 Locates data to inform decision-making and supports assessment of their suitability. Documents when and where required data are not captured, collected, or are missing.

1.3.5 Uses data to understand users' needs and to design and develop products, programs and services that meet those needs (see Government of Canada Digital Standards: Playbook for more details).

1.3.6 Uses data and analytics to weigh the merit and impact of solutions or decisions prior to implementation.

1.3.7 Communicates the complexities and nuances of analytical findings to non-technical audiences to promote understanding and the business value added.

1.3.8 Elaborates and implements policies or initiatives to develop the talent, systems, or infrastructure that allows for the use of and timely access to data for decision-making.

1.3.9 Evaluates data for suitability to inform decision-making by integrating ethical, methodological, and subject-matter or context-specific perspectives.

2.1.1 Is familiar with data governance and data sovereignty, including First Nations Data Sovereignty, standards, directive, processes, including accessibility standards.

2.1.2 Can identify and proactively flags data issues or conflicts pertaining to data governance, stewardship.

2.1.3 Applies key policies, procedures and standards for the collection, access, and management of data.

2.1.4 Collaborates and negotiates to:

• ensure common understanding of data;
• manage access to data;
• identify data privacy, security, or accessibility implications and barriers.

2.1.5 Ensures sound management and oversight of data while limiting data duplication and respecting the data's intended purpose.

2.1.6 Selects methods, processes or tools that minimize the duplication of data, and utilizes data where it resides whenever possible.

2.1.7 Monitors data issues and changes to ensure data is actively managed, accessible and enhanced to maintain trusted, usable data in the enterprise.

2.1.8 Assesses enterprise and organizational priorities for the operational and strategic use of data.

2.1.9 Assesses data requirements for projects, programs, departmental reporting and policies to ensure that data are managed according to value and use.

2.1.10 Assesses requirements for the retention and disposition of data.

2.1.11 Adapts or implements data governance within the operational business context.

2.1.12 Develops rules, governance principles and guidelines prior to the collection, creation, use, or sharing of data.

2.1.13 Develops policies, governance or standards that balance and ensure security, interoperability and access to data.

2.2.4 Extensive knowledge of data collection methods and tools, and how to apply them to ensure quality, consistency, and timeliness.

2.2.5. Supports with mitigation or resolution of data collection issues (e.g., sampling problems, missing data, errors and inconsistencies, etc.).

2.2.6 Designs and implements data collection methods that directly align with a stated question(s).

2.2.7 Designs and implements automated data collection methods when appropriate.

2.3.3 Reviews data to ensure validity, accuracy, and completeness.

2.3.4 Aligns with data standards to ensure and improve data quality.

2.3.5 Engages directly with stakeholders and recognized authorities of data to build relationships that enhance trust within an organization, with stakeholders and with Canadians.

2.3.6 Identifies and analyzes outliers and anomalies within data and uses problem-solving approaches (e.g., Root Cause Analysis) to improve data quality.

2.3.7 Ensures and assesses data for quality and interoperability to maximize value and trust.

2.3.8 Uses or develops master data management (MDM) and reference data management (RDM) solutions to ensure consistency and integrity in data and their use.

2.3.9 Raises awareness about data holdings for integration and re-use.

2.3.10 Develops or coordinates additional data entry training when entered data quality or validity is a concern.

2.3.11 Maintains data lineage and audit trailing to ensure the appropriate fit-for-purpose use of data.

2.3.12 Builds multi-disciplinary partnerships across the organization to foster a communal responsibility towards data governance, standards and stewardship, and to enhance trust throughout the organization.

2.3.13 Evaluate fitness for use of a dataset (i.e., sometime no data is better than the wrong data). Documents and shares rationale transparently.

2.4.5 Supports the use of self-serve data and data access by design.

2.4.6 Awareness of data holdings, open data and accessible data that are available for (re) use and integration.

2.4.7 Identifies and prioritizes the release of high-value datasets to the Open Government Portal for re-use and interoperability.

2.4.8 Applies approved security-enhancing tools to minimize risk when working with data, such as de-identification, anonymization, masking, or other similar methods.

2.4.9 Publishes data in machine-readable formats, to the extent possible.

2.4.10 Understands a variety of conversion methods in support of interoperability.

2.4.11 Develops provenance and mappings to support the preservation of data.

2.4.12 Uses, creates or manages metadata, database indexing, or information analysis and synthesis to improve information access, retrieval and use.

2.4.13 Strives to identify and remove barriers to data sharing and release, and champions open data and transparency in government by sharing and working in the open by default.

3.1.4 Frames questions around business and user needs that can be answered with the support of data.

3.1.5 Identifies pre-existing data that can be used to support or inform questions or decision-making.

3.1.6 Explores data and uses relevant frameworks to identify issues, support research or generate questions relating to practical situations.

3.1.7 Works in multidisciplinary teams as an expert to inform a variety of areas using data-driven approaches.

3.1.8 Applies understanding of organizational and user needs, mandates, and directions to generate business requirements associated with data and to improve problem framing at the outset of new projects, policies, programs, service delivery and more.

3.2.5 Uses basic analytical methods and related tools, qualitative or quantitative, to generate insights.

3.2.6 Takes stated questions and develops an analysis plan including assessing relevant data and methods.

3.2.7 Understands various statistical methods, analyses, and related tools and when their use is appropriate.

3.2.8 Disseminates data and analytical findings openly to support other government endeavours as per the Digital Standards.

3.2.9 Recognizes and explores patterns, relationships and trends within data and across data sources to generate insights using a variety of methods.

3.2.10 Sets clear goals prior to analytical endeavours to ensure value added analyses and consults with all stakeholders to ensure data collection and analytical plans align with this value.

3.2.11 Conducts analysis using relevant data and methods for consumption across a wide range of audiences.

3.2.12 Evaluates results of analyses and compares with other findings.

3.2.13 Shares data and analytical findings openly to support other teams' work.

3.2.14 Can access and manipulates data from different sources, for example using flat files or Structured Query Language (SQL) queries.

3.2.15 Applies common analytical methods and tools to data, qualitative or quantitative, to generate insights.

3.2.16 Applies with the Directive on Automated Decision-Making, including the completion of an Algorithmic Impact Assessment, to identify, evaluate and mitigate risks associated with deploying an automated decision-making system.

3.2.17 Uses a wide range of advanced analytical methods and their related tools, qualitative or quantitative, to generate insights.

3.2.18 Links together data from multiple sources to increase their usefulness and value.

3.2.19 Conducts exploratory analysis to support problem framing, identify areas of risk and more.

3.2.20 Builds and validates statistical models from data.

3.2.21 Uses appropriate, accurate, valid and efficient modelling solutions across a variety of complex data types.

3.2.22 Utilizes advanced tools and techniques to perform data exploration such as data mining, web scraping or machine learning.

3.2.23 Assesses and selects tools based on their benefits, use cases, ease of access and inclusivity, and considers open source options when available.

3.2.24 Considers digital and platform design as well as organization-wide and government enterprise-wide considerations when selecting tools.

3.2.25 Assesses data for their fit for use, quality, accuracy, relevancy and potential bias.

3.2.26 Ability to apply automation for the collection of data.

For more details on data science competencies, please refer to the Competency Profiles developed by the Data Science Network for the Federal Public Service.

3.3.5 Displays holistic information, telling complete stories rather than presenting selective or incomplete evidence.

3.3.6 Ensures data presentations link directly to the original questions or line of thinking.

3.3.7 Assesses audience needs, familiarity with data and understanding of subject matter.

3.3.8 Evaluates storytelling and visualizations for accuracy and misrepresentation.

3.3.9 Communicate best data visualization practises and tools among data teams and others to avoid common mistakes and make data visualizations more effective.

3.3.10 Provides drill down capability in reports and summary information to allow further investigation.

3.3.11 Develops dashboards, infographics, and interactive visualizations using different software, including common business intelligence tools (e.g., PowerBI, Tableau) and/or specialized libraries (e.g., D3.js, seaborn, plotly).

3.4.3 Uses data for the purpose of measuring outcomes relating to policy, program implementation, legislation, regulation, decision-making and more.

3.4.4 Engages regularly with users, collecting feedback and data throughout the implementation of ideas or solutions to take into account how well they are working/performing, and utilizes the feedback to adjust course as needed (see Government of Canada Digital Standards: Playbook for more details).

3.4.5 Cross-compares results with other research and findings.

3.4.6 Retains and maintains original data and information utilized in decision-making to analyze outcomes compared to intent.

3.4.7 Collects follow-up data to assess the efficacy of decisions or solutions.

3.4.8 Identifies key takeaways from charts, tables and graphs to integrate with other information into future decision-making processes.

3.4.9 Analyses data to understand the impact that identified solutions may have on diverse groups of people, policies, programs, services, processes, and more.

3.4.10 Evaluates and measures outcomes of decision-making using data.

3.4.11 Uses end-user feedback to establish evolving requirements to adjust policies, programs, services, and more to meet users' needs.

3.4.12 Obtains stakeholder agreement on criteria for assessment prior to implementation.

3.4.13 Implements changes to policies, programs, service delivery and other government functions based on ongoing feedback and user research.

3.4.14 Monitors the outcomes of automated decisions to safeguard against unintentional outcomes.

4.1.4 Considers architectural needs across business, IT and data when developing enterprise solutions.

4.1.5 Considers full system life cycle in the development or deployment of organizational or enterprise architecture.

4.1.6 Assesses system requirements (such as storage, retention and schedule) to ensure that organizational or enterprise architecture is designed for usability, interoperability and to provide value to users.

4.1.7 Develops and maintains monitoring of AI Systems, which includes but is not limited to health, bias, and data drift.

4.1.8 Designs, implements, or monitors systems that allow for access at source, using appropriate RDM and MDM, reducing the duplication of information.

4.1.9 Anticipates and identifies problem points in systems and solution development.

4.1.10 Prototypes and demonstrates solutions for clients their environments to enable further development.

4.1.11 Builds automation software to operate systems needed for data storage, data management, data science codes, distributed training, model repository, feature repository, continuous delivery, model serving, and monitoring.

4.1.12 Operates production AI systems and makes sure they are available, scalable, and performing effectively.

4.1.13 Builds and shares the technical expertise necessary to analyze and recommend enterprise-grade solutions for operationalizing AI or advanced analytical models.

4.1.14 Sets the required architecture and deployment processes for AI, from data ingestion to production and maintenance.

4.1.15 Provides technical advice to management and other data scientists as it pertains to the operationalization of models.

4.2.2 Ensures that system designs aligns with organizational needs.

4.2.3 Ensures security of data and information through appropriate levels of permissions and access.

4.2.4 Develops, implements, tests and troubleshoots system workflows.

4.2.5 Develops end-to-end data pipelines based on an in-depth understanding of the available tools and data management ecosystem, data life cycle, and business problems to ensure analytics solutions are efficient, predictable and sustainable.

4.2.6 Is comfortable:

• using best coding practices to generate reproducible, verifiable work
• extracting, transforming and loading data with appropriate tools and practices
• using data analysis tools and techniques, including open source

4.2.7 Considers all organizational and enterprise-wide interdependencies that require processes, standards or the preservation and storage of data to maximize organizational value.

4.2.8 Extracts data from data sources utilizing appropriate tools and practices.

4.2.9 Transforms data using a variety of tools.

4.2.10 Knowledge of extract, transfer, load (ETL) functions and their appropriate use.

4.2.11 Assesses, builds or implements data structures such as data lakes, data marts, relational databases, or in-memory databases.

4.2.12 Designs or implements data systems and processes that utilize data at source rather than replicate.

4.2.13 Designs, implements, or assesses data collection and transformation tools, methods and processes.

4.2.14 Assesses the value of cloud storage vs on premise storage.

4.2.15 Uses Cloud and Software as a Service (SaaS), such as Enterprise Resource Planning (ERP), Service Management applications, or Distributed Architecture.

4.2.16 Maintains and supports data availability and longevity through backup systems and appropriate data recovery plans.

4.2.17 Develops and recommends performance indicators to measure the efficiency, efficacy and impact of Information Management solutions.

4.2.18 Determines problems with stakeholders, understands where data science can add value in supporting strategic and operational decision-making to create impact, and designing data science solutions and metrics to these problems.

Artificial Intelligence

Information technology that performs tasks that would ordinarily require biological brainpower to accomplish, such as making sense of spoken language, learning behaviours, or solving problems.

Algorithm

A set of instructions used to solve a problem or perform a computation.

Analytics

The discovery, interpretation and communication of meaningful patterns in data. Concerned with turning data into useful information for making better decisions. Analytics often relies on the application of statistics, algorithms, and software to research potential trends, analyze the effects of decisions, or evaluate the performance of a policy, program, or service.

Dashboard

An information management tool which is used to visually track and represent key performance indicators, metrics and key data points.

Data

Data are facts, figures, observations, or recordings that can take the form of image, sound, text or physical measurements (ex: distance, weight, wave lengths).

Database

A large and organized collection of data. It allows data to be easily accessed, manipulated and updated.

Dataset

A collection of data. It often corresponds to the content of a single database or data file.

Data cleaning

The process of preparing data for analysis by removing, correcting, and documenting data that are incomplete, incorrect, duplicated or improperly formatted.

Data drift

Variations in the production data from the data that was used to test and validate the model before deploying it in production. Data drift leads to model performance degradation.

Data lake

A storage system or repository that holds large amounts of raw data.

Data literacy

The ability to understand, create and communicate data as meaningful information.

Data mart

A subset of a data warehouse focused on a particular line of business, department, or subject area.

Data science

A multidisciplinary field that combines the scientific method, computer programing, statistics, and business to extract meaningful insights from data and inform decision-making. Although data science shares commonalities with statistics, these two terms should not be used interchangeably as they differ in their processes and applications.

Data warehouse

A system that aggregates data from multiple sources into a single, central, consistent data store to support data mining, artificial intelligence, and machine learning.

Enterprise data lake

An enterprise-wide data lake for information storage and sharing.

Extract, transfer, load

A data integration process that combines data from multiple data sources into a single, consistent data store that is loaded into a data warehouse or other target system.

Information

Data converted into a meaningful and useful context. Knowledge captured in any format, such as facts, events, things, processes, or ideas, that can be structured or unstructured, including concepts that within a certain context have particular meaning.

Interoperability

The ability of different systems, devices, applications or products to connect and communicate in a coordinated way, without effort from the end user.

Master data management

The technologies, tools, and processes to ensure master data is coordinated across the enterprise.

Model

A mathematical representation of a real-world process. A predictive model forecasts a future outcome based on past behaviours.

Open data

Structured data that is machine-readable, freely shared, used and built on without restrictions.

Personal information

Recorded information about an identifiable individual other than business contact information. It includes any information that can be linked back to or identify an individual through reference or association.

Qualitative data

Data describing the attributes or properties that an object possesses. They represent subjective variables that are sometimes not easily measured (e.g. taste, eye colour).

Quantitative data

Data expressing a certain quantity, amount or range. Usually there are measurement units associated with the data (e.g., meters for heights, kilograms for weight or Celsius for temperature).

Raw data

A collection of numbers or characters before it has been cleaned or processed.

Reference data management

An approach to maintain accurate, consistent, and standard references to data across the enterprise.

Root cause analysis

A range of approaches, techniques, and tools to systematically identify the causal mechanisms that underlie the potential roots of a problem.

Statistics

In the context of data analysis, statistics refer to a type of information obtained through mathematical operations on data. Statistics also refers to the science of developing and studying methods for collecting, analyzing, interpreting, and presenting data.

Structured data

Data that has been organized into fixed fields within a database or file so that it can be readily searchable and easily analyzed; for example, data an excel spreadsheet.

Structured language query

A programing language used to manage relational databases and conduct operations on their data.

Unstructured data

Data that do not have a pre-defined data model or that are not organized in a pre-defined manner. They are thus not stored in a database in fixed fields. Examples include the text included in a report.