The concept of ratings

Balancing the wants of users and artists

https://www.wincustomize.com

Every few months a firestorm wages on WinCustomize.com. The site hosts thousands of pieces of artwork in the form of skins, themes, icons, wallpaper, etc.

And some people can give a star rating to a submission (1 through 5 starS)

On the one hand, often times the skinners (the ones who create the content) have a problem with the whole rating thing. Usually skinners whose works get below 3 stars. Solutions such as eliminating anonymous ratings have been proposed over the years.  Of course, what many would like to see is the elimination of ratings entirely.

And on the other hand, users very much want the ratings. And in fact plenty of users find WinCustomize to be full of "garbage" because we don't moderate very much content. People are free to upload their half-baked skin and unless it's totally broken or really horrendous it's made available. So the casual user finds themselves wading through a lot of stuff they don't like. This hurts the site and the ones who pay for the bulk of the site's costs -- Stardock.

So WinCustomize.com doesn't want to offend skinners but doesn't want to casual users to have to wade through sub-par stuff.

That's where the rating sytsem comes in.  Only users who have contributed to WinCustomize.com (by getting a subscription or having elite access through their works over time) are allowed to rate skins. So most people can't rate skins.  Moreover, the higher the access, the more the rating is weighted.  So if an admin rates a skin, it is weighed far more heavily than a level 1 user who has a subscription.  This way, those who have spent a lot of time doing this and have seen thousands of skins and themes and icons and wallpapers can make a relatively objective rating. 

The second part of the rating system is that those who are not logged on won't see skins,themes, etc. that have 2 stars or fewer ratings. If they log on, they can see it all. Eventually we hope to have per-library rating filters that users can set.

It's a tough thing. The desire to keep everyone happy is an impossible task. Many people who get 2 star ratings or 3 star ratings are simply unaware that perhaps they need to work on their skin. I regularly will go through the galleries and if I see a skin or theme with a rating that seems way too low (or way too high) I'll check the ratings to see if it's being manipulated. It happens. But the reality is, the average skin is a 2.5 star skin which means most skins hover around 2.5 to 3 stars (including my own stuff).

At the end of the day, the skinning community has to realize that it's a symbiotic relationship.  We all need each other.  We need the users AND the skinners. And we can't placate one without alienating the other. So we have to reach a compromise. The rating system is that compromise.  The alternatives -- not allow anything but the very best on would tick off skinners. Or let everything be on with no ratings would make the site unwieldly to use.

37,926 views 39 replies
Reply #1 Top
I like the ratings and it is nice to have an explaination here for those who don't always understand them. However I liken it to American Idol in a way. There are tons of people who audition for that who clearly can't sing and most seem to have no idea that they can't sing. But you can't fault them for trying.
In the same way there are skinners who really can't skin but nothing is stopping them from trying. Just a ratings system helps keep the good ones from drowning in a tidal wave of the bad ones.
I don't even use the ratings system for the most part since I don't always like the skins everyone else likes.


Posted via WinCustomize Browser/Stardock Central
Reply #2 Top
As an artist on this site... I don't really pay attention to ratings and downloads... it's more about feedback... I like to read comments about my work whether it be compliments or critisizm, which is a word I have no idea how to spell right now for some reason...
Reply #3 Top
criticism

...and I'm with you. I would much rather see some kind of constructive criticism or a positive comment on my work that mess with the ratings; but I don't really care too much about those. Maybe a little
Reply #4 Top
Ahhhhh!!! Thank you so much for explaining who gets to rate skins and who doesn't. I have been puzzling over this for the last few months.
Reply #5 Top
I never pay attention to the rates, except for my own skins. I always sort by most downloads.

A lot of people don't, or can't vote, yet by checking download count, you can usually find popular and professional skins...

Another thing is i think the rating systme isn't really good, i mean some of the skins that i made when i first started (i.e my "game icons" pack) are rated the same, as my "Game Pack 3 (Dasent)" icons.

My first pack i look back on with disgust at its amaturishness, but my latest Game pack is very professional looking, and it is evidence enough that that the rating systme doesn't work, that they're on the same level....

And re you're "3 star average" i think the average is 5 actually, none of my skins are lower then 5, or above 7, even the basic ones or the advanced ones are usually on the same level.

My Smoke sin with 1,900 downloads is a '5', while my 'Ripped; Skin with 1000 downloads is a '7'...
personally i prefer the first one....

anyway, just my thoughts....

EDIT: Yeah as a skinner i always appreciate feedback, it makes me feel like my work is appreciated... If you like a skin, post feedback
Reply #6 Top

And re you're "3 star average" i think the average is 5 actually, none of my skins are lower then 5, or above 7, even the basic ones or the advanced ones are usually on the same level

There are only 5 stars...

Reply #7 Top
Will Hunter - I think you'll find the article refers to the current 5 star rating system. You are referring to the old 1-10 scale I think. Sorry if I'm wrong

I don't care at all about ratings and am like Will Hunter where I only look at the ratings of my own skins. I'll look at screenshots first then decide if I want to download someone else's skins. And when it comes to the ratings of my own skins, I don't really care what they get. I create skins that I like to use. If other people want to use them as well, then good. If my skins don't suit others, fine again.
Reply #8 Top
First off, ghos, get your own avatar! I had it first.

Moreover, the higher the access, the more the rating is weighted. So if an admin rates a skin, it is weighed far more heavily than a level 1 user who has a subscription. This way, those who have spent a lot of time doing this and have seen thousands of skins and themes and icons and wallpapers can make a relatively objective rating.

It also means that the higher the access level, the more of a responsibility the person has when rating a skin. If someone with a powerful rating level rates a skin, it is their duty to do an honest job. Its also the duty of fellow admin to watch for abuse from other admin. Just because they are admin it doesn't mean that they are non biased raters. You can't deny it doesn't happen, I have proof. Someone with high rating power that is found to be playing rating games should have their ability to rate downgraded just as anyone else. An apology would be nice also. Frogboy having the same average ratings as myself doesn't do much for comforting when someone has unfairly rated your work.
Reply #9 Top
I get the concept of rating. What I don't get it why no one has come close to giving me a reason why the *anonymous* part of the rating system has to stay. I have seen many skinners ask to see WHO is rating their skins and what their rating is. Also, the fact that you can rate before downloading. And that you don't HAVE to comment after you rate.

It seems very, very reasonable to me to have to download to rate a piece of artwork, and your rating really should come with a mandatory comment. And most importantly, the public should be able to see who is rating what and how low/high they are rating it. Can anyone explain to me why WinCustomize isn't working towards that?
Reply #10 Top

Linking rating ability to download is certainly a possibility....[not necessarily on the screenshot section though].

The theory regarding publicly viewable ratings is that one of two things would happen...a person would rate 'high' so as not to cause repercussions...or would simply not rate at all...in fear of repercussions.

It's better that it remains hidden as it promotes free opinion....

Reply #11 Top
"It's better that it remains hidden as it promotes free opinion"

And abuse, and threads like these.

Is there any way that just the skinner could see the screenames of those rating him/her?
Reply #12 Top
Is there any way that just the skinner could see the screenames of those rating him/her?


Refer back to Jafo's comments.

Ratings are a necessary evil and those who abuse should have thier ability taken away. However those that whine should realize that if they got a 2 star rating then they need to either accept the piece as personally good but, well, crap to the public. To me that's what it comes down to, I skin for myself and turn it out to the public as a gesture of sharing. If I get a low rating I really don't care too much. I would guess that most of the whiners are skinning for the public instead of themselves, thus making it easy to get thier feelings hurt.....
Reply #13 Top

And abuse, and threads like these.

Can't do a lot about the threads...but the abuse is dealt with quite effectively.  It doesn't go unnoticed for long....

Reply #14 Top
And abuse, and threads like these.


Of course the threads would be completely different if people could see who provided what types of scores:

Examples if ratings were available, even to just skinners:

1) User XYZ is a complete twit and obviously can't recognize true art like I create
2) I can't believe user XYZ thinks my new skin is only 3 stars.... ADMINS HELP ME GET EVEN!!!

Or, if the user didn't start a thread they'd simply find the raters home page or skin library and start rating every skin at 1 start because that is obviously the only thing that could mend their wounded ego. And if you don't think this would happen, look back into the past at some of the fights that have taken place over ratings, some people take these things VERY personally and have zero ability to listen to reason.

Just my .02
Reply #15 Top
However those that whine should realize that if they got a 2 star rating then they need to either accept the piece as personally good but, well, crap to the public.


Thats all fine and dandy but what about when something like this happens?

The following is taken from emails between myself and an unnamed admin.

I was finally able to look at the ratings on your skins. I saw that an admin had given your N8T skin a very low rating a couple days ago. It was way out of line with the other ratings it had received, so I deleted that one.

Next I looked at Batman. It was indeed approved with a 3, but not by the same admin. (rating of 3/10, not 3 stars)

How does one deal with that? Do I email admin? I did and got poop. Actually, what I got in return was more like "tough cookies, deal with it".
Reply #16 Top
I saw that an admin had given your N8T skin a very low rating a couple days ago. It was way out of line with the other ratings it had received, so I deleted that one.

Next I looked at Batman. It was indeed approved with a 3, but not by the same admin. (rating of 3/10, not 3 stars)



So... we have admins abusing the system!
An awful powerful sword to be wielding me thinks. Especially since their ratings carry more weight.
Just dandy...
Reply #17 Top

Depends on the skin. A low rating doesn't mean absuse. Let's not let ego take control here.  2.5 stars is the average rating.

IF an Admin is abusing the ratings, the moderator should have contacted me so that we can deal with it.

Reply #18 Top
IF an Admin is abusing the ratings, the moderator should have contacted me so that we can deal with it.


Been there, done that.
Reply #19 Top

Well I don't recall getting an email on the subject about a particular moderator. I would want to see who this was and then investigate as to why that was the case.

Take your N8T skin for instance. I am currently am the one who rated it lowest (3 stars). I don't dislike the skin, I find it okay. So I rated it as such. Just not my type of skin. Other skins of yours I've rated higher.  On the other hand, there are those who rated it 5 stars. Is my 3 star rating an "abuse"? I don't think so.  I would call a 1 star rating to be abusive though.

Reply #20 Top
this system is very bad and really unjust for many skinner which sees
their work destroy by very bad rathing, personally I do not use it any
more because I do not like the injustices !!!
and off course i not say bla!!!bla!!!bla!!!bla!!!bla!!!bla!!!
it's so long !!!
Reply #21 Top
Just drop it. I am not looking to tar and feather anyone, nor fight things that are above the law. I have traded emails with you personally Frogboy and when I didn't get any answers or apologies, I chose to let it go. This thread just caused an old scar to itch. You opened the door , I entered.

About my N8T skin. I can handle that no problem. I am glad you had the courage to stand behind your rating. Yes, a 1 rating on almost all wb skins would be a shame. I don't see a 3 star rating from you as anything bad. I would think you have been around the block more than anyone else. If you think a skin is a 2 star or 3 star skin, it most certainly probably is. Maybe I can at least help you to see that showing what and who rates things can possibly do more good than bad. I would be so happy to know what my mentors really think of my work. It gives me something solid to work off of.


Poor Xav73, I think you have had it the worst. Hopefully things will change. Keep skinning!

Reply #22 Top
Most of the people who skin for the fun of it, I suspect, are more interested in constructive feedback than in ratings per se. Especially for those of us who do few skins for publication. I would hope even the higher volume skinners would mostly ignore the ratings but if they are important to some people, at least the ground rules and methods are known & discussed. A system that would satisfy everyone and be spam-proof is probably more trouble than it is worth, if not impossible, but I agree that it would be rude, to say the least, to rate a skin without downloading and actually using it. WC can determine if you've downloaded a skin but can't tell if you've actually loaded up & used the skin, so it would be false re-assurance to use the "can-rate-only-if-downloaded" approach. Any rating system will depend on the integrity of the raters and we'll just have to trust them to be fair.

Cheers,
Daiwa
Reply #23 Top
Question: How does one rate something as a 3/10 on a 5 star scale? Do admins have the ability to rate on the 1/10 scale then? 3 stars = 6 on the 1/10 scale, yet NT said that it had a 3 on the 1/10 scale.
Reply #24 Top
Xav is a great skinner, I enjoy almost all of his work, that is just plain too bad if I understand him correctly and he is leaving..he puts everything he has into all his work and thats pretty much obvious..Ah well.
Just a note of sorrow at the loss..

Zero.
Reply #25 Top

Phoon. I rated it as 3 stars. There is no more 10 scale anymore.  It's a 5 star scale.

I don't agree that people should have to have their names show on who rated. That woudl only encourage retaliatory ratings and rating trading and all kinds of other nonsense. Not to mention it would increase the already significant problem of rating inflation that was present before we instituted weighted ratings.