The "Uber driver rating" problem of star-voting

Thinking about the poll offered earlier today, I was put in mind of a web serial author's complaint about a similar rating system (Found in their author notes at the bottom of the chapter). Pastafarian says:

As a side note, I wanted to mention something about RR's review/rating system. Some people aren't aware, but a story's Rating and Rank on RR have a bunch of effects on the story's growth, particularly in the long term. As a result, you can model a rating or review as being either an endorsement (5 stars) which makes the practical statement "this story should have more readers", or a range of magnitudes of the practical statement "this story should have fewer readers" (anything below 5 stars).

I describe this as being out of sync with the intended statement by many reviewers/raters, just as people who give sub-10/10 ratings to customer service representatives don'tΒ intend for their rating to be a condemnation (whereas the CSR's manager will generally consider anything below 9/10 on every metric to be a failure on the CSR's part). This is part of why I would prefer an up/down 2-point scale, which would be in alignment with the intent of the reviewer/rater.

And I was thinking about exactly this issue when I rated (https://civitai.com/models/118756?modelVersionId=135523) 4 stars. I said

Quite effective at responding to even esoteric prompts. My only concern is that some of the women rendered, absent other context, can have faces that are read as a little young.

I also have had quite a lot of trouble with multiple/malformed legs. This model has earned the distinction of making images good enough and correct-to-prompt enough that I want to fix them in img2img instead of just shrugging and moving on.

And I felt guilty. This model is interesting enough to post images of and to explore. Models that aren't interesting, I don't post. But because of the persistent issues of "legs" I was having, I wanted to indicate that it was worthy-of-attention but not quite perfect.

This is not how, in practice, the star rating system seems to work right now. Everything is 5 stars, which means that it's approval rating, where anything less than 5 stars is a punishment.

I hope these thoughts spur some further reflection on rating systems, and I'd be delighted to continue this discussion if people find it useful.

Please authenticate to join the conversation.

Upvoters
Status

Implemented

Board

πŸ’‘ Feature Request

Date

Over 2 years ago

Author

Flibertygibbet

Subscribe to post

Get notified by email when there are changes.