Transcript
Transcript: Product Management in Action: A Job Bank Case Study, Part 4
[Product Management in Action: A Job Bank Case Study
Part 4 of 5: Planning]
Planning begins by aligning objectives with the product vision based on insights gained from user research.
[A text appears on the screen: Planning begins by aligning objectives with the product vision based on insights gained from user research.]
[A text appears on the screen: "Facilitate the integration into the Canadian workforce of individuals who need assistance in competing in the labour market by providing information on employment opportunities to help workers find suitable employment and help employers find suitable workers."]
Our product vision for Job Bank is to "facilitate the integration into the Canadian workforce of individuals who need assistance in competing in the labour market by providing information on employment opportunities to help workers find suitable employment and help employers find suitable workers".
[A text appears on the screen: Mina Riad. Product Manager, Employment and Social Development Canada]
Managing a product in the public sector context involves 4 pillars, the user, the business, stakeholders and technology.
[Title card: Aligning Product Vision with Objectives]
To align product vision with the objectives, we break up the product vision statements into five high-level objectives that take into account those four pillars. The first three objectives are:
1. Provide valuable tools to help job seekers find employment
2.
Provide valuable tools to help employers find workers
3. Provide valuable labour market information and career planning tools to help users' decision-making
These three objectives are tied to user needs, more specifically the need of job seekers, employers and career planners
The fourth objective is:
4. Achieve increased integrity, efficiency and interoperability
This one is more aligned with business goals and stakeholders' input as 1) the more internal efficiencies we can create, 2) the more interactions with other systems we can put in place and 3) the more integrity the platform can achieve, the more time, as a team, we can spend focusing on adding value to the product and responding to user needs.
The fifth and last objective is:
5. Maintain high-quality IT infrastructure and software delivery performance
This objective is more focused on ensuring the technology supporting our product and the way we deliver it remain intact and up to date. Technology is a key pillar of our product and without it, none of the other goals can be achieved.
[Title card: Product Strategy]
Our product groups at Job Bank are constantly engaged in discovery and learning through user research and usability testing. We analyze user interviews, user segmentation and behaviour data, card sorting activities, user feedback and online survey responses to gain insights and continuously improve our product. All these insights guide our planning and product strategy.
For example, user feedback and behaviour analysis, showed us that job seekers tend to unsubscribe from Job Alerts when they receive too many emails. For that reason, we decreased the frequency of alerts sent from twice a day to once a day and provided a weekly frequency as a new option.
What each product group learns from user research goes into a list of new features or enhancements to serve as a high-level roadmap for next iterations. That's what we call a backlog. Currently, these are documented as business requirements, but we plan to shift to user stories to keep user voices at the centre of our work and remind ourselves of the problems we are solving.
We consult subject matter experts and stakeholders within each product group to define the detailed specifications of the changes, including design, content, functionality, user experience, and data requirements. We will talk more about these later.
[Title card: Prioritization Methods]
To quote a Product Management expert, "The only way to combat the speed and chaos of building products is to develop a ruthless mindset... All high functioning teams must prioritise. Not once a month, not once a week — but rigorously, and ruthlessly"
There are many prioritization methods out there.
[A graphic appears on the screen: Reach x Impact x Confidence / Effort = RICE score]
At Job Bank, we use the RICE scoring method to prioritize new features, requirements, and bug fixes to ensure our IT team implements them in the most ideal order.
[A graphic appears on the screen: Calculation of the total score of one item depending on the score for reach, impact, confidence and effort.]
We start by assigning numbers, let's say from 1 to 5 for reach, impact, confidence, and level of effort of the working item or requirement. This entails team discussions and conversations until we reach a collective agreement on the assigned scores.
- Then, we multiply the agreed-on scores for reach, impact, and confidence and divide the obtained result by the effort score.
- We repeat the same process of obtaining scores for all requirements or work items.
- Then we order the scores and their corresponding work items into descending order. The work items or the requirements with the highest RICE scores are prioritized for implementation.
[Title card: Metrics]
We collect a lot of metrics from various sources to ensure that all that we do contributes to the overall health of the product.
We tie metrics that we collect to the objectives we are working on achieving to ensure that our decisions are based on evidence and feedback. For example, to measure the success of the second objective "Provide valuable tools to help Employers find workers," we continuously measure or monitor employers' experiences with Job Bank via a survey in which we ask them to rate their experience on a scale ranging from "excellent" to "very poor." We aim for a 70% satisfaction rate as our healthy indicator.
We measure the impact to ensure that every change we make on the platform (major or minor) has had its expected outcome. For example, we measure the percentage of the employers who attribute their success in filling a vacancy to Job Bank through a closeout survey that the employers fill after their job posting expires.
[A text appears on the screen: Lagging indicators: results from the closeout report, 40% of responses to attribute filling a vacancy]
We aim for at least 40% of responses to attribute filling a vacancy to our platform. These metrics are not actionable in themselves, they show a general trend. If it dips, it means that we need to look deeper into what is causing it.
We also set some leading indicators for work items or requirements to inform some hypothesis on what we expect the change to be.
[A text appears on the screen: Leading indicators: changes to the content of Job Alert emails, 10% increase in the number of job seekers clicking on jobs]
Let's say, we make a change to the Job Alerts email's content and aim that it results in a 10% increase in job seekers clicking on jobs they receive.
[Visual of an example of a release impact analysis at Job Bank appears on screen]
Then, we compare this target to the actual percentage by conducting a release impact analysis in which we collect user administrative and web traffic data to confirm whether our hypotheses were correct or not. And if the results are below the expected ones, we go back to the drawing board and start looking at what causes these drops and look for alternative solutions.
[Title card: Roadmap]
If you wonder how we keep track of everything, I would say through roadmaps. Roadmaps help us communicate our strategic plan for different audiences.
[Visual of a roadmap appears on screen]
For that reason, we create tailored roadmaps with various levels of detail ranging from high level to very detailed ones. For example, for our oversight committee and external stakeholders, we communicate our current state and the desired future state via a high-level roadmap to keep the committee informed and to align our objectives with our work items.
For internal stakeholders, that include 1) ESDC senior management, 2) the Skills & Employment branch we are part of, 3) Job Bank various teams (Operations, Partnerships), and 4) other branches in the department that use Job Bank to advance their respective priorities, we create a 12-month plan along with the different activities related to each of our main objectives.
[Visual of the Microsoft Teams Planner appears on screen]
For our product team, we use Microsoft Teams Planner to manage, assign and prioritise tasks (for example the ones on the top are the ones to complete first), and organise them into backlogged and monthly tasks.
All these roadmaps are revised and updated periodically.
[Title card: End of part 4 of 5]
[The CSPS logo appears onscreen. A text appears on the screen: canada.ca/school. The Government of Canada logo appears onscreen.]
