Problem sorting lists and selections
This is not a new idea but a bug report. When sorting lists/directors/actors etc according to the average rating, the results are not what one expects.
An example: http://letterboxd.com/director/doris-wishman-1/by/rating/
When this is sorted by Average Rating > Highest First the result shows first SEX PERILS OF PAULETTE with 2 four star ratings. On position five is THE PRINCE AND THE NATURE GIRL with no rating at all. On position six is TOO MUCH TOO OFTEN with 1 three star rating. Only much later follows BAD GIRLS GO TO HELL with an average 3.1 rating.
I have seen this problem many times. For some reason films with no rating or a very low number of ratings appear on top of the list, while films with a sufficient number of ratings appear at the end of this list.
And while I am here: It would be nice if I could sort lists not only according to the average rating, but also according to my own rating.
Our weighted rating system pushes high-rated films to the start of a list, and low-rated films to the end of the list. Films with few/no ratings may appear in the middle of the list given this approach.
Yes, it’s definitely more obvious when the numbers are very low. And as films with low numbers effectively receive the average rating for the site, if you’re looking at a director whose films are all rated poorly, then all of their popular films will appear below all of their unknown/unrated films in the ranked list. Thanks for following up. We’ll keep in mind.
I understand your strategy to weigh the ratings, but something in the algorithm is not right. Here is a more extreme version of the problem:
Ranked according to the average rating > highest first the list shows first the film "Billy Frankenstein", a film with no rating, 2 views, 6 listings and zero likes. In contrast, the well-known "Hollywood Chainsaw Hookers" with an average rating of 2.9, 455 views, 202 listings and 110 likes appears on page 3. Page, not position. This can't be intended effect of your calculations. You might note also that the ranking according to popularity reflects reality well.
This strange behaviour of the average rating ranking is not an exception, I have seen this quite often. Maybe it mainly appears in lists where some films have very low numbers of user interactions? An unusual behaviour of your algorithm when some of the numbers are zero, maybe?
A couple of things to note: weighted average is calculated every six hours, so adding a new rating to a film won’t necessarily change the weighted average for that film until it’s recalculated.
The more ratings a film receives, the more we weight the average towards the true mean value of those ratings. We feel it’s incorrect for a film with a single 5-star rating to rank ahead of a film with 99 5-star ratings and a 4-star rating, which is what would happen without our weighted algorithm.
Of course I don't know what your weighting criteria are. But shouldn't films with no rating always end up at the bottom of the list when sorted according to average rating? (I have seen lists where films with no rating and marginal view numbers topped the list.) When I sort the list according to user rating, I am looking for film recommendations from the community. Otherwise I would sort the list according to popularity/views.
What I also don't understand: When only I gave the film a 3.5 star rating, the weighted average was 3.4. When another user gave the film a 3 star rating later, the weighted average was still 3.4. How can the weighted average with one rating be different from this rating and not change, when another rating is introduced into the equation?
Anyhow, I am now aware of this problem and check the rating for each film in the list individually. Thanks for looking into this.
The highest-rated film appears to be correct now, it’s showing “The Thing”. Average ratings are calculated using a weighted rating system, so do not show the “true mean rating” if only a handful of members have rated the film.
There is definitely something wrong with the way average ratings are calculated and compared.
I am the only one who rated MOJ STAN with 3.5 stars: http://i.imgur.com/0lcT2x6.png
But on my 2016 summary I see the following picture: http://i.imgur.com/WD3Prp9.png
The average rating is now calculated as 3.4, what is impossible, since there is only n=1.
Furthermore this film is shown as the one with the highest rating, when the most popular film has a 3.8 rating.