Improving Navigation, Visual, & Content for Mission Electric through Moderated Usability Testing January - March 2021
Mission Electric, developed by the New York-based environmental non-profit Empire Clean Cities, is an electric vehicle campaign and a one-stop shop for all information regarding resources, events, and funding incentives to help users transition to electric vehicles. This project uses moderated user testing to find out usability problems on the homepage, EV page, and News page of missionelectric.org website and provides potential solutions for its navigation, visual, and content.
Client: Empire Clean Cities
Duration: 5 weeks
Location: Remote
Method: Moderated User Testing
Design Tools: Figma, Adobe Illustrator
Teammate: Aichen Guo, Amiya Dewan,
Chris Denney, Qilin Zeng
My role:
-
Analysis: participant profile, finding 4 on the News and Events page and its recommendations, usability problems in task 4 and 6, data calculation
-
Materials: moderator checklist & script, pre-test questionnaire, consent form, notes template, presentation deck
-
Team: moderator for 2 tests, notes taker for 2 tests, arrange and set agenda for the team meeting, files organization, present to client
Project Summary
ABOUT MISSION ELECTRIC: ENSURING CLEAN AIR FOR FUTURE GENERATIONS
Our client Empire Clean Cities is a New York-based environmental nonprofit and local US DOE Clean Cities Coalition whose mission is to ensure clean air for future generations. Mission Electric is an electric vehicle campaign hosted by Empire Clean Cities to help users transition to EVs.
According to the first client meeting, there are 4 goals that our client mainly wants to achieve through this user testing:
-
To find out the effectiveness of the Learn Pages and the News and Events Page
-
To find out whether users are able to learn about EV charging options
-
To find out whether users can find a list of EVs that would suit them
-
To find out if people feel like they can take action from visiting the Mission Electric website (long term goal)
Designing a Remote User Test
OUR PROCESS
.png)
Created tasks and prepare for the tests
Contact participants
Conduct remote user tests with 8 users
DEVELOPING REALISTIC TASKS
The team of expert usability researchers explored the Mission Electric website and developed a set of six tasks that we believed would map to the real user’s experience while using the tool. These tasks were built upon topics explored in the meeting with the client and reflected our team’s understanding of the usability and function of the website.
.png)
Task 1
On the homepage, tell me what do you think the site's purpose is.
- this task focuses on the navigation and clarity of the homepage
Task 4
Besides charging at home, you may need to charge your vehicle somewhere else. Find out whether there are public charging stations near you.
- this task focuses on the "charging station locator" feature and its discoverability
Task 2
Find out what financial benefits you would have from purchasing an electric vehicle rather than a gas one from this website.
- this task focuses on the "Learn incentives" feature and its clarity
.png)
Task 5
Based on the factors you mentioned earlier that are important to you, find out what type or model of EV would work best for you
- this task focuses on the "Learn types of EVs" feature and its clarity
Task 3
If you get an electric vehicle, you may need a charging station in your home. Find out how you would get one installed.
- this task focuses on the "Learn installing charger" feature and its clarity
Task 6
Electric vehicle technologies are rapidly improving and innovating. Find out the latest news, announcements, events, and updates about electric vehicles.
- this task focuses on the "News and Events" feature and its clarity
We also designed and conducted pre-test, post-task, and post-test questionnaires with the participants in order to capture a representative sample of the larger intended community of users as well as to understand the users’ thoughts and emotions after using the website.
Recruiting Individual Drivers & Non-Drivers
DEFINING USER PROFILES
The client provided us a list of user profiles that Mission Electric intends to target.
User 1: Individual Driver or Family
-
Probably based in the Lower Hudson Valley and lives in a single-family home. The family has 1-2 personal vehicles.
-
If based in NYC, more affluent or Tesla-fan. Not necessarily environmentally conscious, but aware of the benefits of electricity versus gasoline.
-
Interested in the health of their family, and savings opportunities of an electric vehicle versus an internal combustion engine vehicle.
-
If already considering an electric vehicle, then has questions about electric vehicle charging and how to access charging, or how to install charging in their homes
User 2: Individual, Does not drive
-
Typically young and environmentally conscious, either currently a student or recently graduated.
-
Interested in personal transportation choices and how to make an impact on their carbon footprint.
-
Typically do not own a vehicle, either because of their age and are currently in college or they live in NYC and do not see the practicality.
PARTICIPANT DEMOGRAPHICS
Based on the pre-test questionnaire result, all participants live or plan to live in New York, which meets Mission Electric’s target users. Over 50% of the participants have an annual income higher than $100,000, and half of the participants own a car, which splits the participants into user profile 1 individual driver and user profile 2 individual who does not drive as provided by the client.
While 5/8 participants claimed that they are likely to purchase an EV (with a likelihood rating higher than 4 out of 5), all participants who answered the question believe that climate change is important to them (with an importance rating higher than 4 out of 5).
Findings & Recommendations
After organizing data of the post-task questionnaire, I calculated the average rating for each task to identify where usability problems concentrate on. We could see that tasks of EV installation and finding an EV model receive the lowest scores, followed by Homepage and News and Events page.
By further analyzing the user test results from eight participants, we found 12 unique usability problems, which centralize in four areas on the website: Homepage, information about EV selection and affordability, installing EV charger at home, and the News and Events page. The 4 major findings and recommendations corresponding to the occurred problems are outlined below.
Finding 1: Participants were confused about the purpose of the website on the homepage
Figure - Current view of the homepage without and with scrolling down
Three problems were identified from the homepage:
-
Problem 1: The landing homepage lacks instructional information for new users, especially without scrolling
-
Problem 2: On the navigation bar, the “For-Hire vehicle driver” tab confuses users’ understanding of the purpose of the website
-
Problem 3: The information is a bit overwhelming on the homepage
Recommendation - make the homepage more appealing
-
Recommendation 1: Add an indication of scrolling down
-
Recommendation 2: Rearrange the navigation bar
-
Recommendation 3: Rescaling the size of icons and reducing text boxes
Finding 2: Participants could not find sufficient information on EV models, especially their costs
-
Problem 1: unable to find enough information on specific EVs, regarding factors that are important to them when choosing a vehicle.
When? When trying to find potential EVs that would suit them. (Task 5 - EV Page)
-
Problem 2: unable to compare the cost of owning an EV to a non-EV.
When? When trying to find financial benefits of owning EV over non-EVs. (Task 2 - EV Page, Incentives Page)
Recommendation - enrich the EV page
-
Recommendation 1: Categorize EVs into groups based on multiple factors, rather than only “type”
-
Recommendation 2: Link each model on the list of EVs to their manufacturer page.
-
Recommendation 3: Create an infographic with a few examples of EVs and non-EVs showing their features and cost breakdown.
Finding 3: “Installing Charging” section lacks clear instructions
-
Problem 1: Users don’t have clear directions to follow in order to install a charger in their home.
When? “Installing Charging” section of “Electric Vehicle Charging” page
-
Problem 2: The information on the “Electric Vehicle Charging” page isn’t organized to allow users to find the information they want efficiently or effectively.
When? The “Electric Vehicle Charging” page
Figure - Current view of the “Installing Charging" section
Recommendation - provide clearer steps for charger installation & add dropdown bars for the EVSE section
-
Recommendation 1: Provide a specific “How to install a charger” section that gives users concrete steps they can take to install a charger in their home, if they need to.
-
Recommendation 2: Cut down on information when possible; make the “Types of EVSE” section more concise and less intimidating
-
Recommendation 3: Add a couple of short questions to guide users to a charger that would work best for them.
Finding 4: Participants were frustrated towards the News and Events page’s layout and contents
-
Problem 1: The News section is not presented in an attractive way as people think it looks like citations. Participants expected the layout to include pictures and short descriptions.
-
Problem 2: The News section lacks updated and interesting news. Participants expected to read more about recent articles on EVs rather than legislation and discourses.
-
Problem 3: The “Event Details” button is hard to notice in the Events section. Events could potentially be difficult to navigate due to the vertical layout if there are more events listed.
Recommendation - update content, redesign layout, & incorporate more visuals
-
Recommendation 1: Both News and Events sections should include more recent articles and webinars on EV models.
-
Recommendation 2: Redesign the visuals of the News section with incorporations of more images.
-
Recommendation 3: Redesign the layout of the Event section and add a filter function.
Result: Simple but Effective Improvements
-
Our client expressed that they are very impressed with the amount of information and recommendations provided in the presentation and the report. The redesigns seem so simple yet solve the usability problems effectively.
-
They love the redesign of the News and Events page. It is very easy to change and the layout presents information in a very clear way.
-
Adding a filter function may be challenging for them at this moment since they need to hire an extra coder, but they still really appreciate the recommendation and would consider add the function in the future.
QUOTES FROM CLIENT
REFLECTIONS ON MODERATED USER TEST
Remote Moderated user testing still includes particular limitations to consider. Limitations relevant to this report are listed below.
-
The order of the tasks may impact the result
Since the test questions were asked based on our user testing script, the tasks' order might impact the ease of achieving each task. For example, when completing one task users may have come across some of the features of the website, like the EV charging station locator. This may have made it easier for them to find that feature for a later task and thus affects the test result.
-
Internet quality may impact the completeness of the usability testing
A few participants' unstable internet connection turned out to be an obstacle for our group to collect all the data regarding each task. Due to the unstable internet connection, a few participants were lagged out during the middle of the test and could not rejoin the Zoom meeting room or share the screen with the evaluators. Therefore, the data of some post-task question ratings was not sufficient.
-
Potential deviation between user tests and the actual users' experience
Potentially, there could be differences between the user experience of user tests’ participants and actual users. In the moderated user test, participants were asked to accomplish specific tasks and with guidance from moderators. Thus, there is a possibility that the participants gave high ratings in the ease of completing the task without paying too much attention to the content. However, when actual users browse through the website with a need of purchasing a new car, they don’t have tasks guiding them and so the navigation and clarity of information would matter a lot more. Such deviation would raise the standard and require extra efforts to put in redesigning the website.