Transcript
Transcript: Product Management in Action: A Job Bank Case Study, Part 5
[Title card: Product Management in Action: A Job Bank Case Study
Part 5 of 5: Delivery and Conclusion]
Job Bank manages its suite of products via a bi-monthly release cycle.
[A text appears on the screen: Mina Riad. Product Manager, Employment and Social Development Canada]
This frequency allows us to balance between meeting user needs in a relatively quick manner while respecting departmental processes related to the deployment of new code into a production environment. This means that we create a release plan for each sprint that lasts two months involving the following process:
Each product group:
- Reviews their roadmap and identifies the themes they want to focus on in the upcoming sprint.
- Breaks down the identified themes into smaller work items considering whether they can be completed within one sprint or require multiple sprints for development.
- Completes the business requirements related to the work items after consulting with subject matter experts and stakeholders
- Prioritizes those work items using the RICE scoring methodology. This scoring helps assess the potential impact and value of each item.
[A text appears on the screen:
- Reviews their roadmap and identifies the themes they want to focus on in the upcoming sprint.
- Breaks down the identified themes into smaller work items considering whether they can be completed within one sprint or require multiple sprints for development.
- Completes the business requirements related to the work items after consulting with subject matter experts and stakeholders
- Prioritizes those work items using the RICE scoring methodology.]
[A text appears on the screen:
Resume Upload in Resume Builder
Descriptions
Allow job seekers to upload and host external resumes, using Resume Builder platform, in
order to apply for jobs via Resume Sharing functionality. This feature will be implemented
with ESDC's DocUpload technology and new Sharepoint file storing solution.
Requirements
In phase I, the Resume Upload must meet the following reqirements:
- The Resume Upload feature is available for standard and plus account users, who have
access to Resume Builder application;
- Users can upload resumes of multiple file formats, including word doc (.doc, .docx), PDF,
text file (.txt, .rtf);
- Each resume should not exceed 5MB in file size;
- Each user account can store maximum of 5 uploaded resumes;
- The uploaded resumes can be stored as long as the user account is active. When an user
account is deactivated, the uploaded resumes will be disposed of after 90 days from the
account deactivation date.]
Then, we (by "we" I mean product managers and product group leads) conduct a review exercise to consolidate all work items from all product groups and ensure consistency in RICE scores. Through this review exercise, we identify if there are any missing dependencies.
[A screenshot of Azure DevOps work items appears on the screen.]
[A screenshot of Azure DevOps Dashboard appears on the screen.]
When we finalize the review exercise, we enter the finalized work items, along with their RICE score, into Azure DevOps. This enables the IT team to start implementing the items according to their priority level. The highest-ranked items are implemented first to ensure the most critical features are completed within the sprint.
[Title card: Prototype]
Job Bank utilizes different types of prototypes based on the complexity of the work item and its development stage. Let's take an example of the current feature development for searching and browsing training opportunities.
[A low fidelity mockup graphic appears on screen]
First, we created initial prototypes in the form of wireframes and low-fidelity mockups to gather feedback from product group members on design and flow. We use tools such as Paint, PowerPoint, and hand-written sketches on paper for that purpose. Initial prototypes are also shared with senior management to ensure the minimum viable product meets their initial expectations.
[A high fidelity mockup graphic appears on screen]
Once initial feedback is received, we develop high fidelity mockups to gather supplementary feedback from the product team members and also start reaching out to stakeholders. In this case, based on the feedback we received on the earlier prototype, we engaged training providers to ensure that their micro-credentials (such as project leadership, writing effectively for business) are presented effectively. For high-fidelity mockups, we use Adobe XD, a software created for that purpose.
[An Interactive Design of Training Marketplace graphic appears on screen]
Finally, with all feedback incorporated, we create a fully interactive design this time to gather feedback from real users. It usually looks like what you see on a webpage but with no code written behind. Interactive design is also developed in Adobe XD and is an optimal way in receiving some final feedback before making the IT investment in coding the actual solution.
[Title card: Testing]
We recognize the importance of testing products with users and in recent years, we have been actively adapting user testing methods before launching a new feature or product. Yet, I have to admit that we need more testing with users in the federal government because it helps us gather feedback and make informed decisions.
To give you an example, before launching a new search filter related to green jobs, we conducted a survey on our website to understand the importance of the environmental impact in the job search journey. We asked in the survey to what extent job seekers think that the environmental impact of their job is important to them when conducting a job search. We also asked them to rank the environmental factors in comparison to other factors such as salary, work hours, benefits, etc. Over 3000 job seekers responded and their feedback guided us in determining the most appropriate solution. The survey results confirmed that there is value in having a "green job" filter, as more than 50% of respondents considered it an important factor in their job search. However, in comparison to other factors, it ranked relatively lower. This information helped us decide on the appropriate level of prominence for the filter.
Currently, we are in the process of developing a Job Transition Tool that enables users to explore possible job transitions with less than 12 months of education/training based on their current job title.
[A text appears on the screen:
- User interviews
- Observations]
To gather feedback, we conduct user interviews and observations with potential users. We present them with an interactive design and closely observe and document their interactions, from entering the job title in the search field to browsing the different possible transitions on the results page. Throughout this process, we also interview users to gather more information on the criteria they prioritize when transitioning from one job to another.
[A text appears on the screen: Continuous improvement]
Setting performance metrics for our product and consistently monitoring its performance over time is crucial. For instance, at Job Bank, we have a feature that enables job seekers to create an account, granting them access to additional features such as job matching, resume building, and one-click job applications. Ensuring a smooth account creation process is essential to us. Therefore, our performance indicator in this case is the percentage of users who successfully complete the account creation process out of those who started it. This metric helps us assess the effectiveness of our solution and identify any pain points. In our case, the account creation process consists of four steps: 1) User information, 2) Confirmation code received via email, 3) Security questions 4) Authentication via SIN, date of birth and mother's maiden name. So it's important for us to pinpoint if there is a particular step with a higher drop-off rate compared to the others.
[A text appears on the screen: Account creation process:
- User information
- Confirmation code received via email
- Security questions
- Authentication via SIN, date of birth and mother maiden name]
[Title card: Conclusion]
To conclude this case study on how we apply product management at Job Bank, I want to say that adopting product management practices within the government context brings immense value to our organization.
[A text appears on the screen:
- User feedback
- Data-driven decision making
- Rigorous work routine]
Through the integration of user feedback, data-driven decision-making, and a more rigorous work routine, we can achieve remarkable outcomes. Product management serves as an efficient and cost-effective means to deliver maximum value to our users by prioritizing features based on data-driven insights. It empowers us to take an iterative approach, where user feedback plays a pivotal role in shaping the development process.
By neglecting product management, we increase the likelihood of encountering various challenges. Scope creep becomes a looming threat, as requirements expand beyond initial expectations, leading to project delays and compromised timelines. Furthermore, without proper product management, there is a risk of delivering solutions that fail to meet user and business needs, requiring extensive rebuilding and wasting valuable resources.
Incorporating product management principles enables us to mitigate these risks and foster a culture of innovation, adaptability, and user-centricity. It equips us with the tools and methodologies necessary to consistently deliver high-quality products that address user needs effectively. Ultimately, embracing product management practices within the government context empowers us to optimize resource allocation, minimize risks, and deliver solutions that drive meaningful impact for our users (for example, job seekers and employers) and stakeholders (for example, provincial and territorial governments and other federal departments such as IRCC).
[A text appears on the screen:
"Embracing product management practices within the government context empowers us to optimize resource allocation, minimize risks, and deliver solutions that drive meaningful impact for our users and stakeholders."
[Title card: End of part 5 of 5]
[The CSPS logo appears onscreen. A text appears on the screen: canada.ca/school. The Government of Canada logo appears onscreen.]
