Hira Jamshed Portfolio
Indeed, the #1 job portal, strives to bring the right opportunities to job seekers around the world by maintaining a large and diverse job board. A significant portion of those job seekers in the U.S are fresh graduates who increasingly use other sites for job searching such as LinkedIn. To increase user satisfaction and maintain its standard, Indeed was curious for a ‘fresh graduate job seeker’ perspective on how well the platform is doing:
Questions from Indeed
8 weeks (October - November 2021)
Contextual Inquiry
Heuristic Evaluation
Usability Tests with Post-Task Interviews
Task-Based Benchmarking
Leadership | Strategy | Recruitment | User Interviews | Testing | Data Analysis (Qualitative and Quantitative)
Lead Mixed-Methods UX Researcher
Project Strategy
1 UX Project Manager: Thomas Thornton
2 (other) UX Researchers: Avni Kering, Taniya Bhosale
2 UX Designers: Raja V S, Dhrumik Sharma
Client: UX Director of Indeed - David Yeats
2 weeks
1.5 weeks
2.5 weeks
2 weeks
Strategise project objectives
Recruitment (10) + Scheduling
Contextual Inquiries (10)
Data Analysis: Decide userflows (3)
Form a schedule
Framework and severity levels
Evaluation of userflow #1
Evaluation of userflow #2
Evaluation of userflow #3
Debrief session
Recruitment (10) + Scheduling
Tasks (3)
Test script
Success metrics (4)
Usability Tests + Interviews (10)
SUS Survey
Thematic analysis (Affinity Maps)
SUS scores
Quantify success metrics
Debrief session
Prioritise major pain points (8)
Actionable recommendations (8)
Things I Contributed to are underlined
This case study sheds light on some critical usability issues and insights into what fresh graduates find lacking on Indeed. In fact, a significant portion of our participant pool did not find Indeed as useful. This further underscores the importance of our findings related to what users value in their job process.
To summarise:
Users value a straightforward process and find the process of applying on Indeed to be complicated
Users value customisable aids in their job search process but do not find such satisfactory options on Indeed
Users value up-to-date and relevant job boards but find Indeed to lack in this regard
“Why is there only one option in the filters drop-down? Is that normal?”
- Participant
“[...] Didn't ask to enter my projects, my resume is incomplete without my projects”
- Participant
“The search isn’t accurate - there are irrelevant areas and titles popping up. How can I trust Indeed to make my job search easier this way?”
- Participant
Amongst graduate students, there are different types of constraints that define their priorities during a job search. For example, an international student might focus on companies that offer sponsorships whereas student with disabilities might focus on companies with an inclusive culture. However, I needed to choose a group that balances ease of access, limited resources, and diversity.
Team brainstorming
To get natural feedback and truly understand a job seekers’ perspective, I conducted 3 out of 10 contextual inquiry interviews by going to the particpants’ work environments. Some research questions I created:
What platforms are commonly used for job searching?
What are the most valued features in a desirable job searching platform?
What are the issues faced in commonly used job searching platforms?
Interestingly, 9/10 participants mentioned LinkedIn to be their primary tool and Indeed as a supplementary tool as they do not find Indeed reliable. To further unpack this, I carried out an affinity mapping excericse to discover 4 major themes about: platforms most commonly used, what job seekers value, what they would like to see more, and feedback on Indeed.
I then led the debrief session to finalize the userflows that revolved around using job boards on Indeed as well as using the company finder. The userflows are listed as follows:
USERFLOW
#1
Search for a preferred job role, fill in personal information, upload their resume, and ultimately apply for the job. This also includes the use of advanced search filters.
USERFLOW
#2
Search for a preferred job role, fill in personal information, create their online resume, and ultimately apply for the job. This also includes advanced search filters as a part of the flow.
USERFLOW
#3
Search for a company to understand more about the company, the job openings, salaries, benefits, and reviews.
A heuristic evaluation had two benefits: critically familiarize ourselves with the flows and weed out major usability issues before move on to testing. The team utilized Jakob Neilson’s ”10 Usability Heuristics for User Interface Design” over a severity scale of 0-4.
After independently completing the evaluations, I also led the session for combining evaluations wherein a consensus was reached for each of the evaluations in each userflow. The analysis informed us about the state of usability problems on the platform.
Combining the results across all the userflows, it was found that the platform has a significant number of usability issues. Specifically:
USERFLOW
#2
Confusing navigation and lack of flexibility/personalisation during resume building.
USERFLOW
#3
Cumbersome searchability and discoverability of companies
The data was collected through qualitative (‘think-out’ loud protocol and post-task interviews) and quantitative (post-task surveys) means. I played the role of a notetaker in the rest of the interviews. Some highlights worth mentioning:
It was observed that Indeed provided relevant job boards in search results for tech job seekers but was not able to accurately judge the requirements of non-tech job seekers. Thus, I was able to uncover the disappointing sentiments related to irrelevant search results and out-of-date job boards.
I underestimated task times and created vague scenarios. This led to an all-over-the-place usability test, and so discarding the data of the first participant. We learnt our lesson and made the necessary changes for future tests (revised our script to clarify task scenarios, and kept the third task as optional).
The methods revealed that job seekers might experience confusion, dissatisfaction, irrelevant content, and lack of flexibility. Therefore, I decided to measure:
Task success: to measure learnability and confusion of the platform
Time on task: to judge the convenience and ease of the platform
Self-reported difficulty level and satisfaction level were used to provide context for feedback that might not be apparent from the metrics, such as ‘I was able to complete the task, but I find it to be an unnecessarily confusing experience’.
SUS Scores: benchmarking the percieved usability of each task
INSIGHT #1
When participants searched for a job as per their preferrence, they ended up getting varied results and were not confident with the search results. Some got great relevant ones, whereas others got extremely irrelevant to their search term.
“It’s really frustrating when the platform is not able to understand what roles I am looking for, especially since the process of job searching is too time-consuming already” - P2
INSIGHT #2
Participants find the experience of optimising their search using filters to be extremely limiting. This is because of the inability to select more than one filter within a particular category at a time. While they were able to complete the task, they experienced significant delays due to the lack of personalised results.
“I am uncomfortable in moving forward with my application if I can’t double check my application materials” - P6
INSIGHT #3
Once the resume was built through the online resume builder, participants were unable to view the finished document and were directed to the next step of the application immediately. In case of a ‘word document’ version of attached resume, participants were unable to view it as well.
“What if I am a developer experienced in more than one coding language - how can I make sure to get all relevant results in one search?” - P9
INSIGHT #4
There are multiple ways of reaching company pages on Indeed. Due to a lack of clarity in Indeed’s UI, a significant number of participants were confused. The more ways participants discovered, the more doubtful they became.
“This is a really nice feature! I wish they would make reaching to it more clear - I wouldn’t want to spend so much time simply understanding how to get to the company pages.” - P8
The severity level of a finding is measured by the following:
Risk: the likelihood that a user will be affected by the issue
Severity: how negatively or positively the issue would affect the user
Magnitude: how many users would be affected.
Throughout the methods above, we see a few themes pop up repeatedly, which point to the root of the usability problems within the Indeed website. Some of them include:
Throughout the methods above, we see a few themes pop up repeatedly, which point to the root of the usability problems within the Indeed website. Some of them include:
Description
When users searched for a job as per their preferrence, they seemed thrown off by the results. While some participants (software developers) got relevant results, a large proportion got extremely irrelevant results to their search queries.
Recommendation
The Indeed search algorithm needs to be made more precise. This would help decrease frustration and make every user feel recognized in terms of their professions.
Occurence - 80% of the participants seemed frustrated with this. For example:
Description
The resume builder on Indeed is a feature which populates a general resume template based on user input.Users expected to view their completed resume. However, Indeed does not provide it.
Recommendation
Give the preview of the resume right after the user completes all the required input.
Occurence - 80% of the participants seemed frustrated with this. For example:
Description
The search option on Indeed allows users to select multiple filters to personalize their search. However, each drop-down list limits user to only one selection. Users were frustrated having to do conduct different searches just for utilizing multiple selections from the drop-down list.
Recommendation
Have a checked box mechanism for selecting filters
Occurence - 40% of the users reported the issue of wanting to select multiple filter elements. For example:
This was my first-ever UXR project. While I learned valuable knowledge on the different UXR methodologies and tools out there during the course, the project itself taught me a few valuable lessons as well, such as:
Importance of planning: seeing things going wrong during my first usability test taught me how activities like pilot testing can inform our plans and save time and resources. Not only that, including the recruitment of diverse graduate students in our research plan helped us discover a crucial insight about the differential treatment by the Indeed platform.
Proactivity in managing time: activities like recruitment & scheduling taught me that being proactive can make a big difference in how much time can be saved. In addition, delays in work due to different schedules taught me how controlling my attitude and mindset can still ensure deadlines are met.
Organisation through color coding and tagging: analysing data through affinity maps and excel sheets taught me how the use of color and tags can not only make it easier to do the analysis but also enables efficient collaboration.
Following this study, I would like to:
Diversify our pool of participants: recruit participants from different backgrounds and people with disabilities. This would help us explore how their backgrounds and capabilities inform their job search practices.
Perform comparative analysis with other popular platforms: our study found multiple preferences for job application platforms. I am curious about the wants and needs of users navigating their job search process. To bring transparency, I would like to pick the popular ones and evaluate reasons behind users’ preferences.
Explore 'company reviews' and other functionalities provided by Indeed: I would like to increase participants for task 3 and determine the optimal path to utilise the functionality. Building onto that, we would also like to test other functionalities, such as blogs and employer activities.