The Chess Variant Pages




[ Help | Earliest Comments | Latest Comments ]
[ List All Subjects of Discussion | Create New Subject of Discussion ]
[ List Latest Comments Only For Pages | Games | Rated Pages | Rated Games | Subjects of Discussion ]

Comments/Ratings for a Single Item

Later Reverse Order EarlierEarliest
Chess Variant Pages Rating System. Missing description[All Comments] [Add Comment or Rating]
Fergus Duniho wrote on 2013-04-19 UTC
I am deprecating this. In the years it has been around, it has hardly been used. The new system of selecting favorites, now integrated into the menubar, replaces this. The new system is much simpler, highly visible, and not limited to selected variants.

Fergus Duniho wrote on 2009-01-20 UTC
It does seem that this rating system has been little used. The highest rated game has remained the same for the longest time, and it has been rated only twice. The two main problems with it right now are visibility and competition. The link to rate a game appears near the bottom of the page. Meanwhile, there is a link at the top of each game page for rating the page. People who want to rate a game usually use the top link, which is for entering a comment and a brief rating. With two systems in place for rating games, people are favoring the one that is more visible and easier to use. David, if you want people to use this system instead, my recommendation is to (1) make it more visible and (2) use only a single rating system. Either remove ratings from comments or integrate this system into the comment system.

John Smith wrote on 2009-01-20 UTCPoor ★
This system seems much too tedious. How about having numerical ratings in the comments and a rankings page based on that? Perhaps editors can review ratings for being official or not, having the same requirements as this system.

Michael Lubin wrote on 2007-12-23 UTCAverage ★★★
This idea is good in general, but your category system strikes me as weak. Your four categories other than 'Overall' do not, between them, get at what is most important about a game. How well the variant achieves its stated goals? Whether that's even a good thing depends on the goals. The number and quality of novel concepts introduced in the variant? I don't think I'm alone in believing that a large number of good chess variants -- probably easily a majority -- rank relatively low in this category, while many bad ones rank very high. It seems to me that the main thing is PLAYABILITY. 'Aesthetic appeal' has something to do with playability, but doesn't really cover it. And then 'Presentation' is just the web page. I can easily imagine disliking a chess variant and yet ranking it equal or higher in each of these categories to a game I really liked.

David Howe wrote on 2006-06-23 UTC
Hi Joe,

>> Next, how do I sign up?

Go to any of your game pages (the quickest way to do that is to go to your
information page, and click on the 'Items this person invented' link. Any
of the items under the 'Game' section can be submitted for entry in the
rating system. Just display the web page for the game you want to submit,
and you'll see near the bottom, right above the author/inventor line, a
link to 'Submit this game to be available for rating!'. Click on that
link and follow the instructions. You have 7 games you invented, so it
shouldn't be too much of an effort for you to submit them individually. I
just hope Ralph Betza doesn't try to submit all his! ;-)

Note that only GAME pages may be submitted for rating. If you have a game
courier or zillions item that you want to submit, just create a game page
for it first, and then submit the game page. I am limiting ratings to game
pages to avoid one person rating game X's zillions page, while another
person rates game X's game courier preset and yet another person rates
game X's game information page.

All for now. Please continue with the feedback, and thank you for the kind
words. My experience tells me I will be making lots of changes to this
system, so don't be shy about suggestions.

Joe Joyce wrote on 2006-06-23 UTCGood ★★★★
This seems to be making everybody who wants a tighter and more informative
rating system happy, but it may take a while for the games to actually
make it to the 'rated' stage, as it requires 5 individual ratings. 
Next, how do I sign up? As far as I'm concerned, every game I post I want
rated, so is there an automatic sign-up? I've currently got 4 public
presets, 2 of them with a rook option. How do I get them put into the
rating pool? And how do the 2 presets with the optional rook get rated?
Does someone have to play all [preset] versions of a game to rate it? And
any other games I get posted? Can they go in automatically?
I guess that's enough questions for now. Thanks. I'm interested in
seeing how this works out. I hope it does well.

Fergus Duniho wrote on 2006-06-08 UTC

David Howe wrote:

It may lead to a rater 'buying' points by rating some large number of entries as 'Okay', not because they want to rate those entries, but because they want to gain points.

True enough. But this would be a problem only if those 'Okay' ratings were dishonest. To put a different spin on this, it gives people incentive to rate games they would normally neglect. The overall effect of this would be to make ratings more representative of a game's overall appeal. Also, the need to buy points to give high or low ratings might encourage a person to take a closer look at games he has passed over before, and he might find some new favorites before getting the extra points he needs.


Gary Gifford wrote on 2006-06-07 UTC
Here is another idea:  Have no ratings at all.  Just two rows (A) and (B):

(A) This is what I like about this game:

(B) This is what I don't like about this game:

David Howe wrote on 2006-06-07 UTC
It seems a desirable goal to give more meaning (or weight) to the more extreme ratings. However, the technique you describe might have unintended consequences. It may lead to a rater 'buying' points by rating some large number of entries as 'Okay', not because they want to rate those entries, but because they want to gain points.

Another possible system might involve awarding points over time (everyone gets N points every M days). The points would be of the 'use it or lose it' type, so that point hording would not be possible.

Or perhaps a 'free market' system, where each rater starts out with N points, which the rater uses to 'buy' the right to rate a particular entry from the entry's owner. Eg. Dave has 10 points. He buys the right to rate Peter's Anti-King Chess, which Peter is selling for 3 points. Dave now has 7 points left, and Peter has 13 (assuming he had 10 before the transaction).

The down side to these schemes is that they discourage free expression by putting certain artificial limits on expression.


Fergus Duniho wrote on 2006-06-07 UTC
I just thought of a different idea for rating games. Give members a number of points they can use for rating games. More extreme ratings would cost more points, and less extreme ratings would cost fewer. I'll use the entirely non-comparitive set of terms (Poor, Mediocre, Okay, Good, Excellent) for discussion of this idea. For example, an Okay rating could cost one point, a Good or Mediocre rating two points, and a Poor or Excellent rating 3 points. This would give more meaning to Excellent and Poor ratings, as people would reserve them for the games they truly regarded as among the best or worst. It would also allow an expansion of the scale for better recognizing the best and worst games. For example, ratings of Awful and Superb might cost four points. So as not to prevent people from rating games at all, the number of points available could be a function of how many games a member has already rated. This could be as simple as three plus the number of games previously rated times two. Someone who has rated no games would have three points available for rating a game. After rating one game, he would have five points minus any he already used. After rating each new game, he would have two more points minus any already used.

Fergus Duniho wrote on 2006-06-07 UTC
Overall, this is a good idea. So I will just quibble about the details, specifically the terms used for the ratings. Some terms are comparitive and others are not. A game is average or below average only in relation to other games. A game is poor or good independently of what is average. Depending on the quality of most games, the average game might be either poor or good. The use of both sets of terms can be confusing. I recommend going with a set of terms that is either entirely comparitive or entirely non-comparitive. An entirely comparitive set could be Among the Worst, Below Average, Average, Above Average, and Among the Best. An entirely non-comparitive set could be Poor, Mediocre, Okay, Good, and Excellent.

Gary Gifford wrote on 2006-06-07 UTCExcellent ★★★★★
It is good to see a well-thoughtout proposal which actually requires that
the raters play at least one game of what they are to evaluate... of
course I already imagine there will be complainers.  

Of the system's steps, I really like number 4, which says a lot, and I
quote:

4. Any ratings submitted would have to be accompanied by a textual
explanation explaining the reasons for the particular ratings. Only after
review by the editorial staff would a rating be accepted. Raters must
have
played at least one game of the variant they are rating.

Very nice David.  Well done and well written.

Namik Zade wrote on 2006-03-25 UTCGood ★★★★
The five-point system is widely used in many areas and yields exact enough results

Sam Trenholme wrote on 2006-03-24 UTC
I only rate a game poor if the game's rules do not adequately describe how to play the game in question. Sometimes, someone has a half-baked idea for a variant, and they don't take the time or energy to even fully describe the variant, much less make a Zillions ZRF or Game Courier preset for the variant. If a game is fully fleshed out, or has a Zillions rule file, or even a game courier preset, I will plain simply not rate the game poor.

If I think there is an issue with the mechanics of a fully described game, I will discuss my issue in an unrated game comment. This gives me an opportunity to discuss whether my concern is really a legitimate concern with other chessvariant.org editors and the game's inventor. I will usually suggest how I would fix the game when bringing up the concern. As just one example, I feel Hex chess may have problems. The consensus, however, is that these are probably not legitimate concerns.

Speaking of Game Courier, could someone please index my game courier preset for my take on Carrera chess.

- Sam


David Paulowich wrote on 2006-03-24 UTC
'i know i am not going to ever rate a game poor'

'oh yes i agree, 'zhouxia' is crap'

Really, Christine, you need to go easier on your own inventions and be tougher on other people's games :>)

I reserve 'Excellent' ratings to games that are clearly in the top five percent, 'Good' ratings to those in the next twenty percent that have some points of interest, and 'Poor' ratings to ... oops!. We are all aware that the majority of games posted here are poorly designed, will never be played on Game Courier Preset, and may not even get a single comment on their pages. 'Poor' ratings are sort of warning signs posted in a more or less random fashion about this web site. We should spend our precious time studying and playing games of proven quality.


Christine Bagley-Jones wrote on 2006-03-24 UTC
yeah it was pretty funny :)

Jeremy Good wrote on 2006-03-24 UTC
I'm glad my tongue-in-cheek proposed worded descriptions amused some
people as they did me. While I agree that using words like 'beneath
contempt' to describe a variant may hurt people's feelings, I do think
there is a serious reason to use a number system that includes 1 through
10. I do not think that 1 through 5 allows as much flexibility in
analysis. A '4' in a 1 - 10 system is less harsh than 'Poor' in a 1 -
5 system. 

I think there is a serious purpose in having a rating system which allows
a certain depth of analysis. That is so we can list pages according to
their rating. This will allow visitors to the site, including ourselves,
to sort through variants according to apparent quality. Of course,
popularity will not always translate into quality, but at least we can
have some sorting mechanisms in place that will help guide us through an
increasingly prolific site. 

I would like to see all 10s rated together, all 9.4's rated together, all
5.32's rated together, etc (averages derived from cumulative ratings).
This would encourage people to really take seriously the art of critiquing
games. Does a game really deserve the rating of 10? People can go to the
page on which 10s are listed and say one way or the other, thus
influencing the way in which the games are listed.

I would also like to ask that we take seriously the idea of separate
rating systems for different aspects of games 'playability'
'originality' and 'appearance.' Again, the advantage is to evolve
sorting mechanisms (sorting according to playability, etc). It would also
help the designer to know what people did or didn't like about the
games.

I would also like to hear some feedback for a separate rating system in
place for Game Courier post-game analysis, so that people who have
actually played games can then have a chance to rate them. At the end of a
game, there could be an option, 'Do you want to rate this variant?'

Christine Bagley-Jones wrote on 2006-03-24 UTC
yes it is just the word, i don't think it is precise, it could lead to
confusion or whatever.
thinking about the rating system, i just reckon there needs to be
something between poor and good, probably average is fine.
if you look at how people rate, 95% rate excellent or good, that is
probably because there is no average rating. most people don't bother
about rating a game poor. this year, members have rated games 30
excellent, 20 good and 5 poor, and that poor rating recently got 3 poors
because of people naming games 'grand chess 2' and 'grander chess'. 
non-members rated in a similiar fashion, bit more excellents, bit less
goods, around same poors.
i know i am not going to ever rate a game poor, it could be my taste in
games, or i can't bother to rate a game i don't like. also i would never
rate a game average, and wouldn't be surprised if people mainly didn't
either, but maybe i'm wrong with that. 
about 'popular' ...
are you saying the 'most popular' thingy is worked out by amount of
comments? ... that can't be good, that would mean 'gridlock' would be a
very popular game, and i'm pretty sure/unsure no one has ever played it. 
(woops sorry mr leno, just remember i did rate your game excellent)

Thomas McElmurry wrote on 2006-03-24 UTC
Good point. It's true that ambivalence is only one possible cause of neutrality. Perhaps I should have said '... to express ambivalence or indifference,' although one who is indifferent toward a game is perhaps less likely to want to comment at all than one who is ambivalent.

If it's the choice of words that bothers you, I readily concede that 'Neutral' is not the best possible word for this new rating. 'Average' is significantly better, but still not perfect.

On another topic, I'm not sure I like the proposed popularity ranking. I'm guessing that under this system, the most 'popular' items on these pages would be the Rules of Chess FAQ, an old flame war regarding the Gothic Chess patent, and Navia Dratp.


Christine Bagley-Jones wrote on 2006-03-24 UTC
i am guessing we are keeping 'none', so as to make comments on a game but
not rate.
now this is no big deal, but ..
Thomas, i understand 'average', but as far as 'neutral' is concerned,
it does not mean 'ambivalence', which means ...
1.The coexistence of opposing attitudes or feelings, such as love and
hate, toward a person, object, or idea. 
2.Uncertainty or indecisiveness as to which course to follow. 
where as neutral means ..
1.Not aligned with, supporting, or favoring either side in a war, dispute,
or contest. 
2.Belonging to neither side in a controversy: on neutral ground. 
3.Belonging to neither kind; not one thing or the other. 
so technically, i cannot see how 'neutral' is a rating, or showing
anything at all towards a game.

Thomas McElmurry wrote on 2006-03-24 UTC
In the previous thread, Christine Bagley-Jones wrote:
i think the rating system in place is just fine, what is the point of 'neutral', what is that, it isn't even a rating, and isn't 'none' pretty much the same.
I will address the second, fourth, and fifth independent clauses of this sentence.

2. The point of a 'neutral' or 'average' rating is to allow users to express ambivalence.

4. Correct. As the system now stands, anyone can skim the rules of a game, think about it for thirty seconds, and proclaim that it's the best thing ever. But someone who has played a game numerous times, or analyzed in detail, and comes to the conclusion that it's nothing special (neither very good nor very bad) has no way to express that opinion and have it taken into account in the computation of the game's average rating. The fact that 'Neutral' or 'Average' is not a rating in the current system is the primary reason why change is desirable.

5. Correct. 'None' is not at all the same as 'Neutral' or 'Average'. 'None' is not a rating, communicates nothing about the user's opinion of a game, and has no effect on the average/overall rating. But a neutral rating would communicate something (i.e. that the user considers the game to be neither very good nor very bad) and does affect the overall rating.

In my opinion, in order for a rating system to be useful, it must include one rating which corresponds to neutrality, and at least two ratings on each side, so that both positive and negative opinions can be expressed with various degrees of intensity. The current system satisfies neither of these criteria and is therefore not useful. The proposed system satisfies both, and I think it could be useful if used properly (a big 'if'). It's interesting to note that the labels (ranging from 'Poor' to 'Excellent') are still biased toward boosting people's egos, but this is relatively unimportant: the meaning of the ratings is carried by their underlying numerical values, as long as it is clear which is the neutral rating.


Christine Bagley-Jones wrote on 2006-03-23 UTC
'How about:

0 Poor
1 Below average
2 Average
3 Good
4 Excellent'

well that is the best i've heard. using numbers is good idea too, so as
not to upset people, then i guess you could rate 1, 2, 3, 4 or 5.

Adrian Alvarez de la Campa wrote on 2006-03-23 UTC
I am in full agreement with both Fergus' and Derek's comments.

Fergus Duniho wrote on 2006-03-23 UTC
I think the ratings system should be divorced from the comments system. It should be setup so that any user can rate a game only once, and it should make it clear that it is for rating the game, not the page or the game's presentation. To this end, ratings for anything but games should be eliminated. This will make the ratings system useful for generating lists of the top-rated games. There should also be the option for a user to change his rating for a game.

Derek Nalls wrote on 2006-03-23 UTC
For that matter, any rating scale at all can offend game inventors
(including the 4-tier one we are using right now).  Still, we need a
rating scale.  It can be comparable to helpful advice.

I don't think adjectives should be used at all (including the ones in our
current system).  When someone's game receives a below-average rating, bad
words trigger people to get upset and feel insulted moreso than numbers.

25 comments displayed

Later Reverse Order EarlierEarliest

Permalink to the exact comments currently displayed.