Scenario 4: Metrics & Impact
You recently launched a new onboarding flow for first-time users of the plugin. After two weeks, you’re seeing mixed engagement results.
Question
What metrics would you review to assess the success of the new onboarding experience? How would you determine if further changes are needed?
Onboarding flow metrics
Overall onboarding flow completion rate
How many started the flow ➗ how many completedThis will give us an idea if people are getting to the end of the onboarding flow or not.
Breakdown by each flow step
Review each of the flow steps to look for drops in completion rate to help focus on where the problem is. From experience, having too many steps means people are less likely to complete every step, but it could also suggest that people are having trouble performing the step.
Using product analytics tools enables you to more easily see the progress visually.
While going through RankMath’s current onboarding flow there are steps to connect Search Console. If you identified a drop here, you might need to look into who has connected their Google account vs who has assigned a property.
Activity outside of the flow
While looking at the impact of a new onboarding flow, it’s important to consider the wider effects of this change too, and check some further data. Here are some I would check.
Returning users rate
It’s always good to check if the returning user rate for those who’ve experienced the onboarding flow update. If this has changed for the negative, this could be an indicator that they have been put off by it.
Alternatively looking at uninstall rates for those seeing the new flow would similarly be a strong measure.
Key actions
Using event tracking, I’d look to see if the tasks that users are performing after the onboarding flow have changed. Is the rate or frequency at which users are optimising their content (or other key activities you’ve determined create stickiness that contribute to their “aha” moment) also changing?
Identifying if changes are needed
By looking first at the onboarding flow steps, it’s easy to narrow down any pain points to see if you can replicate the issue. Replaying this with user session recording is also useful (though as I previously said, I’m not sure if this is allowed within WordPress plugins).
If you want to understand if there are too many asks for a flow, I would review the overall flow completion rate, compared to the previous iteration.
Ultimately, all changes to a flow also need the context of the wider landscape. If users are now coming back more regularly and engaging in with RankMath features, then this is sends strong signals that the onboarding flow is working. They’re not encountering blockers because they’ve been removed during onboarding.
Thank you
I appreciate you taking the time to read my submission! I really enjoyed answering the questions.
Hope to hear from you soon.