5 min read
How I led a field study for an AI Copilot that increased in-app conversions by 14.5%
I was the sole UX Researcher at Jetson responsible for user research, getting feedback and building prototypes.
Results
Increased D1 and D7 retention
Jetson is a mobile education platform that helps new entrepreneurs take daily action. It started as an education platform and evolved into an AI co-pilot for first-time founders.
The goal was to help entrepreneurs start a business from their phone but users weren't making progress.
An early version of the Jetson app featured a 30 day roadmap to start a business. Each day contained a mix of learning materials including definition-style quizzes, bite sized case studies and an action item. During onboarding the user would enter the business idea that they wanted to work on and would then start day 1. However, a significant portion of users were not making it to day 2 and of those that did, another portion weren’t making it to day 3*.
How might we increase user adoption and improve in-app conversion?
Increase conversion from day 1 to day 2
Increase conversion from day 2 to day 3
Jetson v1: Day 1 Action without AI
Reducing cognitive load would be key to reducing friction and improving metrics.
Data
Looking at our conversion funnel, I could see that the biggest drop was happening during the action item step. For day 1 this meant that a lot of users would complete both pieces of education content but then not input the name of the business they were starting*. The same was true for day 2 but replaced with not attaching the logo of their business.
User Feedback
I reached out to a portion of the churned users via SMS and the answer became clear. When confronted with the action to name their business or add their logo, most didn’t have the answers or files readily available to them so they closed the app and forgot to come back.
AI apps have pre-packaged prompts ready to go but they're not predictive.
Immediately the team knew this was a place where AI could have a significant impact. Since Dec 2022, job postings related to AI have risen by 42% and the job title “Prompt Engineer” has become a hot topic with a range in salary from 200K to upwards of 300K. Put simply, knowing how to interact with an AI system is incredibly valuable, yet also nuanced. The difficulty with a lot of Generative AI (GenAI) tools, e.g. ChatGPT & Midjourney, is that they force users to unknowingly become prompt engineers. That said, chat interfaces are fairly straightforward to start. Take a look at ChatGPT or Microsoft Copilot, they’ve even pre-canned prompts for me to use. However, getting high-quality outputs from these tools requires high-quality inputs.
Here are two examples
Microsoft Copilot and ChatGPT
Early testing provided insights and created a strong feedback loop.
At the time, I was running workshops at local high schools teaching students how to start their own business. It was the perfect way to get in front of an audience and test ideas in real time. We had built a web version of an AI idea generator and a competitor research tool. Once a student found an idea worth pursuing, they could copy that idea over to the competitor research tool to learn more about the market and industry.
Leading the workshop. Tap through to see AI generators.
Because the generators were independent of each other there was some confusion about copying ideas over to the new generator. Additionally, sometimes students would really like an idea but generate another idea for exploratory purposes. If the student hadn’t copied the idea they really liked at the start, they’d have to type it from memory.
Being predictive meant understanding a user's goals, motivations, and desires.
Following the SMS feedback from users and the insights from the student workshops, I designed a new type of AI tool, one that didn’t require any text input from the user but rather built on itself. In the case of the name generator tool, rather than suggesting any random name, we used the business idea that the user had already input within the generation prompt. Suggestions could be scaled and personalized to each user and the businesses they were starting.
Jetson day 1 action with AI suggestion
Not another chatbot.
By reducing cognitive load and anticipating user needs, we were able to guide first-time founders through crucial early steps in their entrepreneurial journey, improving retention and increasing conversions. As the app has evolved, and now includes open access to a variety of AI tools, I’ve found that users who use an AI tool have a 7% higher retention than those who don’t. Understanding this about our user base has been influential to our product roadmap and strategy.
Day 1 to day 2 conversion
14.5% increase
Day 2 to day 3 conversion
70% increase
Retention of users who used an AI tool
7% higher than those who did not
Unlocking the parking brake and removing barriers to entry.
I remember reading a book years ago called The Catalyst by Jonah Berger. The book is about change and opens with a simple metaphor: "Sometimes Change doesn't require more horsepower. Sometimes we just need to unlock the parking brake."
AI can significantly enhance user experiences when it simplifies decision-making and delivers personalized, actionable results. In this project, the introduction of AI tools helped remove barriers and unblock users who were stuck and improved our roadmap conversion significantly. AI is still in its early days but the lesson here is clear: the power of AI lies not just in automation, but in its ability to seamlessly integrate into a user’s workflow, reducing friction, and providing value at every step.
*Specific figures cannot be shared due to data privacy considerations.