H. G. Muller wrote on Wed, Nov 30, 2016 02:18 PM UTC:
"For example, I want to evaluate Pegasus, then change 2 Rooks by 2 Pegasus and launch long series. If I get a result of winrate of 60% (for example), how to convert it to centipawns?"
You can then delete the f-Pawn of the side that scored 60%, and repeat the test. If it now scores 45%, you know the Pawn was worth 15%, so the 60% translates to an advantage of 2/3 Pawn, which would be 33cP for a sigle Rook-Pegasus difference. If you do this for many piece comparisons,and each time deleting a Pawn makes the score drop ~15%, you can expect that to be always the case, and skip the Pawn-odds test if the score was close to 50%.
"And how long match need to played for accuracy of 0.25 pawn? 0.1 pawn? 0.05 pawn?"
The standard deviation of the score percentage in a match of N games is approximately 40%/sqrt(N). For N=100 that would be 4%. If a Pawn is indeed worth ~16%, that would be 25cP. But in 32% of the cases the error would be larger than that. To be within error bars 95% of the time you should count on an error of 2 standard deviations. To get that to 25cP you would need 4 times as many game, i.e. 400. To get the standard deviation to 0.1 Pawn (2.5x smaller) you would need 6.25 times as many games, i.e. 2500. For 5cp it would be 10,000. But I doubt that piece values can be defined with this precision anyway; the whole idea that pieces have aconstant value is only an approximation, and in practice the value could differ dependent on what the opponent has, or where they start in the opening. (As is most dramatically demonstrated by the fact that in 'Charge of the Light Brigade' 7 Knights crush 3 Queens.)
"Is it works if I set a fast timing limits of 1 sec per turn, or maybe better to use another timing limits for such tests?"
It is better to use time cotrolslike 40 or 60 moves per minute, than a fixed time per move, so that the engine can allocate the time where its needs it, and is not forced to abort a depth iteration that took unexpectedly long, and let the effort goto waste. But 40 moves/min should be fine. I never tried faster games.
"For example, I want to evaluate Pegasus, then change 2 Rooks by 2 Pegasus and launch long series. If I get a result of winrate of 60% (for example), how to convert it to centipawns?"
You can then delete the f-Pawn of the side that scored 60%, and repeat the test. If it now scores 45%, you know the Pawn was worth 15%, so the 60% translates to an advantage of 2/3 Pawn, which would be 33cP for a sigle Rook-Pegasus difference. If you do this for many piece comparisons,and each time deleting a Pawn makes the score drop ~15%, you can expect that to be always the case, and skip the Pawn-odds test if the score was close to 50%.
"And how long match need to played for accuracy of 0.25 pawn? 0.1 pawn? 0.05 pawn?"
The standard deviation of the score percentage in a match of N games is approximately 40%/sqrt(N). For N=100 that would be 4%. If a Pawn is indeed worth ~16%, that would be 25cP. But in 32% of the cases the error would be larger than that. To be within error bars 95% of the time you should count on an error of 2 standard deviations. To get that to 25cP you would need 4 times as many game, i.e. 400. To get the standard deviation to 0.1 Pawn (2.5x smaller) you would need 6.25 times as many games, i.e. 2500. For 5cp it would be 10,000. But I doubt that piece values can be defined with this precision anyway; the whole idea that pieces have aconstant value is only an approximation, and in practice the value could differ dependent on what the opponent has, or where they start in the opening. (As is most dramatically demonstrated by the fact that in 'Charge of the Light Brigade' 7 Knights crush 3 Queens.)
"Is it works if I set a fast timing limits of 1 sec per turn, or maybe better to use another timing limits for such tests?"
It is better to use time cotrolslike 40 or 60 moves per minute, than a fixed time per move, so that the engine can allocate the time where its needs it, and is not forced to abort a depth iteration that took unexpectedly long, and let the effort goto waste. But 40 moves/min should be fine. I never tried faster games.
More later.