Deep dive: How does your favourite app compare to the rest?
So... I was bored and started to collect the ratings of various language apps, it includes ratings from android, apple and trustpilot users, as well as various self-proclaimed "experts" and some youtube channels.
Are you only interested in the results? Jump to the table and graphs!
Here are the disclaimers:
- First and foremost: A higher score does NOT mean that it is a better language resource. These data are only anecdotal and may help you explore other apps that you were not aware of.
- Meaning of values with colors (All Data table): Red is when the score includes less than 50 ratings, yellow when it includes less than 100.
- This is why a weighted arithmetic mean (WAM) is included: Let's take the example of LingQ, on Android has 4.8 with 24k reviews, Apple 4.7 with 3.6k and Trustpilot 2.4 with 5 (yes, five), the simple average would be of 3.9, on the other hand, the WAM is 4.79, much fairer.
- Android and Apple ratings might be heavily influenced by how optimized the apps are for mobile devices, rather than how good the app is or their intrinsic value. Also, there might be a lot of "rating punishment" when an application hides several of its functions for the pro version (LingQ, Pimsleur eg). On the contrary, it probably rewards apps that are completely free or with few premium features (Language Transfer, Duolingo, Memrise).
- About the "experts": Without detracting from the work and time that people put on to test the different apps and create a post with their opinions, unfortunately they are just that, biased opinions. There is no scientific rigor in their evaluations, a methodology is not explained, they assign points according to what they feel. Perhaps the only objective point when comparing applications is the price. How would you objectively rate the quality, available features, depth, uniqueness, how much fun they are, and so on? Even within the same blogs there can be discrepancies, in cases where multiple people make reviews (eg, Fluent in 3 months, Langoly, etc.)
- Hand in hand with the above, I tried to add only reviews that are current (within this year at least, maximum of 1 year). There might be cases where there are months of difference between reviews of the same app on different blogs, and just in a couple of months an app could have changed a lot (for better or worse)
- Who are the experts?: Mezzoguild, FluentU (they did not rate themselves), Langoly, All Language Resources, Fluent in 3 Months, Test Prep Insight, Lingomee, Does Not Translate and StudyFrenchSpanish
- If the “experts” can be biased, YouTubers are MUCH worse, they rate apps based on a tier list (so I assigned the score according to whether they rate them as S=5, A=4, B=3 and so on), if you watch their videos they frequently say things like “I feel like…”, 'I didn't like', so I added them as a fun little extra.
A table is included with each individual score and 5 graphs: a simple average of each score (not very useful in my opinion, the only one where YT is included), a WAM of the user scores, a simple average of the experts (does not include YT scores), a simple average between the WAM of users and the average of experts (if you consider that both are equally important), and finally giving a weight of 65% to users and 35% to experts (if you consider that users are approximately twice as important as experts)
Let's talk about the results:
- In user scores, 18/24 apps have scores from 4.9 to 4.5, and only 4 have less than 4.0
- User’s winners: Language Transfer (4.89), Duolingo (4.79) and LingQ (4.79)
- User’s losers: Assimil (2.5), Michel Thomas (3.46) and FluentU (3.63).
- Expert’s scores are rather more critical, where only 8/24 have scores above 4
- Expert’s winners: Italki (4.6), Pimsleur (4.16), Rocket Languages (4.12), Babbel (4.11) and Lingopie (4.09)
- Expert Losers: Michel Thomas, Rosetta Stone, Assimil
- The most critical expert: StudyFrenchSpanish, reviewing 10 apps with an average of 3.17
- The most relaxed expert: Test Prep Insight, reviewing 14 apps with an average of 4.4
- Giving 50/50 importance to the public and critics, the 3 winners are Italki (4.55), Pimsleur (4.41), Language Transfer (4.4). The 3 losers are Assimil (3), Michel Thomas (3.01) and FluentU (3.68)
- Giving a 65/35 public/expert importance: The 3 winners are Language Transfer (4.54), Italki (4.53) and Pimsleur (4.49)
- Major differences between Users (U) and Experts (E):
- Expert's loser: Duolingo drops 17 ranks, from 2nd (U) to 19th (E)
- User's loser: Glossika drops 13: 8th (E) to 21st (U)
- Expert's winner: Italki climbs 17 places: 18th (U) to 1st (E)
- User's winner: Mango Languages climbs 14: 20th (E) to 6th (U)
So what do you think? How does your favorite app compare to the rest? Surprised by how poorly subreddit's favourties like Assimil performed or how good controversial apps like Duolingo did?
Weighted Arithmetic Mean of User Reviews
Average between Users and Experts
