15 Apr 2021
“If the answer was more information, we’d all be billionaires with perfect abs.” -Derek Sivers
If there’s one thing I know about myself, it’s that I tend to favor infomation over action. When given a task, chances are I’m going to spend more time thinking of the optimal way to complete it than just jumping into the weeds and arduously working my way through it.
Anecdotally, however, I’ve found that I learn more by doing the latter instead of the former. On the surface, this makes sense. Researching things allows you to passively pick up the best of what people already know. You don’t have to go through the painful process of discovery to figure things out, so you don’t feel as wed to what you learned as you would if you came up with the discovery yourself.
But this process of picking up the best of what other people already know falls short in places where action and execution matter more than the right idea.
In other words, there are games decided by action, and games decided by information. It’s your job to figure out what the right combination is for the game you’re playing, and move on from what you favor once you hit a point of diminishing marginal returns.
As an example: people close to me know that I thru-hiked the Appalachian Trail. Ahead of my thru-hike, I read books here and there about how to prep, and spoke with former thru-hikers about things they would have done differently. But I would’ve been crazy to think that more information was all that I needed to get from Georgia to Maine. At some point you have to start walking.
I used to think that “Everything I want to know I can learn through some combination of Googling and reading”. But I’ve come to realize that simply isn’t true. At some point you have to roll up your sleeves and get to work.
There are no shortcuts.
20 Oct 2020
There were a lot of entrepreneurship groups at college. People were always talking about business ideas: apps, websites, products, services. Anything that could potentially be used to make a buck.
These groups made me give entrepreneurship a bad name. I got a sense that the people involved were more interested in what being an entrepreneur looked like—owning a business, making money, getting accolades—than actually solving people’s problems.
In reality, the only way you’re ever going to convince somebody to give you money for a product is if that product solves a problem that’s top of mind for them. That idea is what I think entrepreneurship is really about. A good entrepreneur ought to be more interested in other people’s problems than their own.
Further reading:
Update, 2022-03-07: another great blog post discussing this idea is Your Product is Not Their Problem.
Update, 2022-03-28: Paul Graham got this point across in a single tweet:
19 Sep 2020
I’m still working out what it means to be good at your job, but we can probably all agree that there’s a strong correlation between people who are good at their job and people who are constantly honing skills related to their job.
If that’s the case though, how does a PM hone their skills? The PM skillset is often so broad it’s hard to pin down any one thing to focus on that will make a tangible difference—you need to understand certain technical concepts, sure, but you also need to communicate well, learn what your customers are truly interested in, form opinions on what’s most important for your product, etc.
Software engineers and data scientists are lucky in that their day jobs rely on skills that they can hone outside of the 9-5. For SWEs, opportunities for side projects/open source projects abound. Data scientists can learn new methodologies, go deeper into statistical theories, heck, they can do Kaggle competitions. But what are parallels for a PM job?
Although there isn’t a straightforward answer, I’d say the best answer is starting a software business. What’s a better way to develop customer empathy, ruthlessly prioritize features, and develop a product vision than boostratpping a software company? I see a number of benefits:
- You develop an owner’s mindset
- You learn to obsess about a customer’s problems (because how else are you going to sell them a product?)
- You gain experience in multiple disciplines, like sales, marketing, finance, engineering, etc.
What do you think? Is there a better way for PMs to upskill?
06 Sep 2020
I’ve spent one year learning Chinese on Duolingo. It’s a well-designed product that effectively encourages users to build a habit around language learning. As any reasonable PM would say, though, no product is perfect—there is always room for improvement!
Here are some improvements I’d look into making if I were a PM at Duolingo.
Replacing Leagues with XP Challenges
There is a lot of gamification in Duolingo. There are a few different motivators Duolingo uses to increase app usage:
- XP: points earned for completing a lesson.
- Crowns: earned by completing five lessons from a particular skill/topic/vocabulary group.
- Streaks: earned by completing lessons . This used to be reaching a daily XP goal every day, but some time in the past year it changed.
- Gems/lingots: earned by reaching a daily XP goal or watching ads. Gems can be used to buy certain items, namely streak freezes (which allow you to keep your streak even if you miss a day) and outfits for the Duolingo mascot.
- Achievements: earned by doing certain tasks (e.g. uploading a profile photo, doing lessons on the weekend) or reaching a certain quantity of XP
Another motivator is leagues. Leagues are groups of 50 users who compete to get the highest XP totals over the course of a week. The top 10 users with the highest XP counts are promoted to the next league; the bottom 5 get demoted. There are 10 leagues in total (from lowest to highest: Bronze, Silver, Gold, Sapphire, Ruby, Emerald, Amethyst, Pearl, Obsidian, Diamond).
Problems and Resultant Anti-Patterns
Leagues have motivated me to binge the app before, but I suspect this sort of bingeing isn’t sustainable. There’s something incredibly frustrating about pouring your heart into a league for a week only to get bumped out of the top 10 on the last day by somebody else at the last second. Even worse is missing out on an achievement (i.e. the “Legendary” achievement, which requires you to get 1st place in the Diamond league) because somebody else just so happened to have more time on their hands.
When a friend started on Duolingo, they were curious about leagues. I told them they were a recipe for burnout—the higher your league, the more competitive your opponents will be, and the more you’ll have to do to keep up. There are strategies to get around the rat race, but they actually result in using the app less. That’s an anti-pattern.
Here’s an example: I wanted to get the Legendary achievement, because it’s conceivably the only remaining achievement that I won’t automatically get through continued use of the app (the remaining achievements I have to unlock are a function of XP and words learned, which I’ll passively get as I progress through the Chinese course). Given that the self-selecting group of people who make it to Diamond League are already pretty competitive, getting the top spot is uber-competitive, especially considering a rare achievement is on the line (don’t believe me? Read this post).
My workaround to avoid the rat race? Don’t complete any lessons between Monday and Saturday morning1, and then binge for the entire weekend. Passing on lessons for the majority of the week puts you into the less-active side of the Diamond-league user spectrum, so overall XP counts will be much lower (and therefore much easier to conquer with concentrated effort). This strategy worked really well for me (if we’re defining “well” as minimal effort for maximal reward): the 2nd place user in the Diamond league that I won finished the week with about 1600 XP. Compare that to stories on Reddit where greater than 1000 XP in Diamond is needed to not get demoted.
Solutions
If one of a product’s KPIs is retention, and a feature rationally encourages less usage of the product, then chances are it’s a suboptimal feature. I’m guessing Duolingo has some data suggesting Leagues encourage retention, but my question is: “Over what time frame?”. I bet Leagues lead to short-term bingeing at the expense of long-term retention.
Are there any examples of features that encourage users to take a break in the short-term for the benefit of the long-term? Yes. Apple’s Screen Time is a perfect example:
Apple was not concerned with people using their devices less as a result of Screen Time, because it aims to provide the best usage experience, not the longest. “We don’t need to make you use it every minute of every day,” said Joswiak. “Our business model doesn’t depend on how much you use your devices.” (source)
Now, I think it is in Apple’s best interest for their users to use devices more, at least a little bit. Greater iPhone usage means more digital purchases for which Apple gets a 30% cut, plus an accelerated depreciation of the phone that leads to device repair or replacement. But what Apple is doing by introducing features like Screen Time is playing the long game—they think they can make more money from loyal customers by caring about their health and wellness compared to letting customers binge use products and grow disenchanted. In other words, they think sustained usage is better than binge-y usage.
For Duolingo, what would a good league replacement look like? Ideally, it would reward a user’s effort in a vacuum—that is, your reward would not be a function of how your effort compares to others around you. Additionally, it would encourage long-term retention instead of short-term binges in app usage.
I think something like “XP challenges” would fit this description: something where the user is encouraged to hit a certain amount of XP in a given week in order to advance to higher levels. You can still make hard-to-earn achievements with XP challenges—just set a wild XP target (5K, maybe?) for a given week. Users can go after the achievements when they want and not worry about gamesmanship from others.
Feature Parity Across Courses
When you read Duolingo forums or talk with friends who are learning a different language, one thing becomes clear: not all courses are created equal. Some features exist only in certain courses. Take Stories, for instance, which are a way to earn XP through answering questions associated with a (you guessed it) story.
As of this post’s writing, Stories are available in the Spanish, Portuguese, French, and German courses for English speakers, which reaches 54.4M speakers2 but not for the other 32 courses (who have a collective 47.7M users).
I’m sure there is a ton of work that goes into launching new experiences like this, and I trust that the Duolingo team prioritize languages which with the new experience will have the greatest impact. But hearing a certain feature isn’t available for your course leaves a bad taste in your mouth. It disincentivizes retention—which is why feature parity across courses is so important.
Side note: Stories are generally regarded as easy XP. If the previous section didn’t persuade you that leagues are bad, this point should help. If certain users have access to easy XP and others don’t, then users aren’t on a level playing field when competing to get the highest XP totals in a given week, which is what leagues are all about!
Standardizing the Advertising UI
Just like many other tech companies, Duolingo monetizes non-subscription users through ads. Optional viewing ads (i.e. ads that you can exit out of within a few seconds) show up at the end of each lesson. Users can also choose to view ads to double gem rewards, though these ads generally require viewership for 15 or 30 seconds before you can claim your reward.
Each ad I see on Duolingo has a slightly different UI: there is no consistency with where the video content, call-to-action, and cancel button are placed. I have a hunch this lowers click-through rates for ads—and thereby lowers ad revenue for Duolingo—because users have to take on the cognitive load of parsing the page before being persuaded by the content.
Duolingo doesn’t run their own ad network–they instead serve as a property for Google and Facebook-served ads—so there may not be much they can do about this. But it would be an interesting scenario to A/B test, if possible.
Normalizing Sizes for Matching Exercises
This recommendation is a nit-pick, but I may as well list it here. At least for Chinese, there are five different exercises Duolingo uses to teach:
- Foreign-native translation: translate a written Chinese sentence using an English word bank.
- Native-foreign translation: translate a written English sentence using a Chinese character bank.
- Character selection: choose the character for the presented pinyin.
- Character-pinyin matching: match the Chinese character to its pinyin spelling.
- Listening: write out what’s spoken in a Chinese audio clip out using a Chinese character bank.
- Speaking: recite a Chinese sentence.
My problem is with (4): users have to click on very small tiles to do the matching. There’s a ton of space left on the screen that would make the experience easier on the fingers and eyes—just take a look at this screenshot. I’m optimistic this will get fixed, though, as I’ve seen a re-designed experience a few times now. I’m guessing they’re A/B testing the re-design before releasing it for good.
Unify the Web and Mobile Experience
Duolingo has both a mobile app and a website. Each platform has a slightly different experience and feature set, which I bet leads to a lot of unnecessary overhead for the engineering team to support3.
One example: users have a different currency on each platform (lingots on the web compared to gems on mobile), each with different accumulation rates: gems are won in random quantities after you reach your daily XP goal, whereas you get one lingot for every multiple of ten you extend your streak to (i.e. a 370 day streak gets you 37 lingots).
Certain features of the web app aren’t fully built out either. For instance the web app has keyboard shortcuts for some of the exercise types listed in the previous section, but not all of them—that’s an accessibility problem. I’d be a die-hard web user if there were keyboard shortcuts for each exercise type. It’d let me iterate through lessons so much faster!
1 The league you’re placed in is determined by the time you finish your first lesson for the week. If you finish your first lesson for the week fewer than 24 hours prior to the league ending (sometime on Sunday evening), you’re not placed in a league that week. ↩
2 Portuguese seems like a strange language to prioritize for Stories, given that six other languages (Japanese, Italian, Korean, Chinese, Russian, and Arabic) have higher user counts. Maybe the lexical similarity between Spanish and Portuguese made it easy to port Stories over from one language to the other (this blog post suggests this is the right answer) .↩
3 What, you don’t think PMs care about engineering overhead? Of course we do! It’s all about opportunity cost. Time spent maintaing two distinct experiences is time spent not building features that will move the needle on the product’s KPIs. ↩
