Product Improvements for Duolingo

06 Sep 2020

I’ve spent one year learning Chinese on Duolingo. It’s a well-designed product that effectively encourages users to build a habit around language learning. As any reasonable PM would say, though, no product is perfect—there is always room for improvement!

Here are some improvements I’d look into making if I were a PM at Duolingo.

Replacing Leagues with XP Challenges

There is a lot of gamification in Duolingo. There are a few different motivators Duolingo uses to increase app usage:

1. XP: points earned for completing a lesson.
2. Crowns: earned by completing five lessons from a particular skill/topic/vocabulary group.
3. Streaks: earned by completing lessons . This used to be reaching a daily XP goal every day, but some time in the past year it changed.
4. Gems/lingots: earned by reaching a daily XP goal or watching ads. Gems can be used to buy certain items, namely streak freezes (which allow you to keep your streak even if you miss a day) and outfits for the Duolingo mascot.
5. Achievements: earned by doing certain tasks (e.g. uploading a profile photo, doing lessons on the weekend) or reaching a certain quantity of XP

Another motivator is leagues. Leagues are groups of 50 users who compete to get the highest XP totals over the course of a week. The top 10 users with the highest XP counts are promoted to the next league; the bottom 5 get demoted. There are 10 leagues in total (from lowest to highest: Bronze, Silver, Gold, Sapphire, Ruby, Emerald, Amethyst, Pearl, Obsidian, Diamond).

Problems and Resultant Anti-Patterns

Leagues have motivated me to binge the app before, but I suspect this sort of bingeing isn’t sustainable. There’s something incredibly frustrating about pouring your heart into a league for a week only to get bumped out of the top 10 on the last day by somebody else at the last second. Even worse is missing out on an achievement (i.e. the “Legendary” achievement, which requires you to get 1st place in the Diamond league) because somebody else just so happened to have more time on their hands.

When a friend started on Duolingo, they were curious about leagues. I told them they were a recipe for burnout—the higher your league, the more competitive your opponents will be, and the more you’ll have to do to keep up. There are strategies to get around the rat race, but they actually result in using the app less. That’s an anti-pattern.

Here’s an example: I wanted to get the Legendary achievement, because it’s conceivably the only remaining achievement that I won’t automatically get through continued use of the app (the remaining achievements I have to unlock are a function of XP and words learned, which I’ll passively get as I progress through the Chinese course). Given that the self-selecting group of people who make it to Diamond League are already pretty competitive, getting the top spot is uber-competitive, especially considering a rare achievement is on the line (don’t believe me? Read this post).

My workaround to avoid the rat race? Don’t complete any lessons between Monday and Saturday morning¹, and then binge for the entire weekend. Passing on lessons for the majority of the week puts you into the less-active side of the Diamond-league user spectrum, so overall XP counts will be much lower (and therefore much easier to conquer with concentrated effort). This strategy worked really well for me (if we’re defining “well” as minimal effort for maximal reward): the 2nd place user in the Diamond league that I won finished the week with about 1600 XP. Compare that to stories on Reddit where greater than 1000 XP in Diamond is needed to not get demoted.

Solutions

If one of a product’s KPIs is retention, and a feature rationally encourages less usage of the product, then chances are it’s a suboptimal feature. I’m guessing Duolingo has some data suggesting Leagues encourage retention, but my question is: “Over what time frame?”. I bet Leagues lead to short-term bingeing at the expense of long-term retention.

Are there any examples of features that encourage users to take a break in the short-term for the benefit of the long-term? Yes. Apple’s Screen Time is a perfect example:

Apple was not concerned with people using their devices less as a result of Screen Time, because it aims to provide the best usage experience, not the longest. “We don’t need to make you use it every minute of every day,” said Joswiak. “Our business model doesn’t depend on how much you use your devices.” (source)

Now, I think it is in Apple’s best interest for their users to use devices more, at least a little bit. Greater iPhone usage means more digital purchases for which Apple gets a 30% cut, plus an accelerated depreciation of the phone that leads to device repair or replacement. But what Apple is doing by introducing features like Screen Time is playing the long game—they think they can make more money from loyal customers by caring about their health and wellness compared to letting customers binge use products and grow disenchanted. In other words, they think sustained usage is better than binge-y usage.

For Duolingo, what would a good league replacement look like? Ideally, it would reward a user’s effort in a vacuum—that is, your reward would not be a function of how your effort compares to others around you. Additionally, it would encourage long-term retention instead of short-term binges in app usage.

I think something like “XP challenges” would fit this description: something where the user is encouraged to hit a certain amount of XP in a given week in order to advance to higher levels. You can still make hard-to-earn achievements with XP challenges—just set a wild XP target (5K, maybe?) for a given week. Users can go after the achievements when they want and not worry about gamesmanship from others.

Feature Parity Across Courses

When you read Duolingo forums or talk with friends who are learning a different language, one thing becomes clear: not all courses are created equal. Some features exist only in certain courses. Take Stories, for instance, which are a way to earn XP through answering questions associated with a (you guessed it) story.

As of this post’s writing, Stories are available in the Spanish, Portuguese, French, and German courses for English speakers, which reaches 54.4M speakers² but not for the other 32 courses (who have a collective 47.7M users).

I’m sure there is a ton of work that goes into launching new experiences like this, and I trust that the Duolingo team prioritize languages which with the new experience will have the greatest impact. But hearing a certain feature isn’t available for your course leaves a bad taste in your mouth. It disincentivizes retention—which is why feature parity across courses is so important.

Side note: Stories are generally regarded as easy XP. If the previous section didn’t persuade you that leagues are bad, this point should help. If certain users have access to easy XP and others don’t, then users aren’t on a level playing field when competing to get the highest XP totals in a given week, which is what leagues are all about!

Standardizing the Advertising UI

Just like many other tech companies, Duolingo monetizes non-subscription users through ads. Optional viewing ads (i.e. ads that you can exit out of within a few seconds) show up at the end of each lesson. Users can also choose to view ads to double gem rewards, though these ads generally require viewership for 15 or 30 seconds before you can claim your reward.

Each ad I see on Duolingo has a slightly different UI: there is no consistency with where the video content, call-to-action, and cancel button are placed. I have a hunch this lowers click-through rates for ads—and thereby lowers ad revenue for Duolingo—because users have to take on the cognitive load of parsing the page before being persuaded by the content.

Duolingo doesn’t run their own ad network–they instead serve as a property for Google and Facebook-served ads—so there may not be much they can do about this. But it would be an interesting scenario to A/B test, if possible.

Normalizing Sizes for Matching Exercises

This recommendation is a nit-pick, but I may as well list it here. At least for Chinese, there are five different exercises Duolingo uses to teach:

1. Foreign-native translation: translate a written Chinese sentence using an English word bank.
2. Native-foreign translation: translate a written English sentence using a Chinese character bank.
3. Character selection: choose the character for the presented pinyin.
4. Character-pinyin matching: match the Chinese character to its pinyin spelling.
5. Listening: write out what’s spoken in a Chinese audio clip out using a Chinese character bank.
6. Speaking: recite a Chinese sentence.

My problem is with (4): users have to click on very small tiles to do the matching. There’s a ton of space left on the screen that would make the experience easier on the fingers and eyes—just take a look at this screenshot. I’m optimistic this will get fixed, though, as I’ve seen a re-designed experience a few times now. I’m guessing they’re A/B testing the re-design before releasing it for good.

Unify the Web and Mobile Experience

Duolingo has both a mobile app and a website. Each platform has a slightly different experience and feature set, which I bet leads to a lot of unnecessary overhead for the engineering team to support³.

One example: users have a different currency on each platform (lingots on the web compared to gems on mobile), each with different accumulation rates: gems are won in random quantities after you reach your daily XP goal, whereas you get one lingot for every multiple of ten you extend your streak to (i.e. a 370 day streak gets you 37 lingots).

Certain features of the web app aren’t fully built out either. For instance the web app has keyboard shortcuts for some of the exercise types listed in the previous section, but not all of them—that’s an accessibility problem. I’d be a die-hard web user if there were keyboard shortcuts for each exercise type. It’d let me iterate through lessons so much faster!

---

¹ The league you’re placed in is determined by the time you finish your first lesson for the week. If you finish your first lesson for the week fewer than 24 hours prior to the league ending (sometime on Sunday evening), you’re not placed in a league that week. ↩

² Portuguese seems like a strange language to prioritize for Stories, given that six other languages (Japanese, Italian, Korean, Chinese, Russian, and Arabic) have higher user counts. Maybe the lexical similarity between Spanish and Portuguese made it easy to port Stories over from one language to the other (this blog post suggests this is the right answer) .↩

³ What, you don’t think PMs care about engineering overhead? Of course we do! It’s all about opportunity cost. Time spent maintaing two distinct experiences is time spent not building features that will move the needle on the product’s KPIs. ↩