I wonder if increasing the number of 10 positions would decrease the results' span.
Sure, in the limit of an infinite number of positions this should converge to a fixed value. I suppose this limit could even be calculated from the probabilities that squares will be occupied (and by what) according to the rules used by the model to generate the positions. It was just less work to have the program estimate it.
The variation between attempts is not larger than the accuracy of the method anyway. (And a bit of randomization helps to avoid deterministic play.) Different distributions of black vs white pieces might result in a different limit: the more your pieces gravitate to your own side of the board, the larger will be the reward for 'forwardness'. Perhaps the move counts should have been weighted to take account of the probability the piece will be there (which would be smaller in the enemy camp). Using a different filling fraction would affect the relative values of leapers, sliders and hoppers. (Strong chess programs do use separate sets of piece values in end-game and opening phase. The Diagram's AI is not that advanced, which probably means it handles hoppers stupidly. But if I would ever imrove that I would probably have it determine the values with a 50% and 10% filling fraction, respectively.)
There are also many 'global' aspects of the move patterns that are not taken into account. It seems that there should be some bonus for attacking orthogonally adjacent squares ('concentration'), which is the most likely explanation for the unexpectedly high value of the Archbishop. There probably should be a penalty for color binding, especially for severe forms of color binding. As it is, the algortithm would value an Alibaba about equal to a Knight, because it is not aware it can only access 1/4 of the board. Having only step moves (i.e. Shogi generals) should probably be penalized.
Sure, in the limit of an infinite number of positions this should converge to a fixed value. I suppose this limit could even be calculated from the probabilities that squares will be occupied (and by what) according to the rules used by the model to generate the positions. It was just less work to have the program estimate it.
The variation between attempts is not larger than the accuracy of the method anyway. (And a bit of randomization helps to avoid deterministic play.) Different distributions of black vs white pieces might result in a different limit: the more your pieces gravitate to your own side of the board, the larger will be the reward for 'forwardness'. Perhaps the move counts should have been weighted to take account of the probability the piece will be there (which would be smaller in the enemy camp). Using a different filling fraction would affect the relative values of leapers, sliders and hoppers. (Strong chess programs do use separate sets of piece values in end-game and opening phase. The Diagram's AI is not that advanced, which probably means it handles hoppers stupidly. But if I would ever imrove that I would probably have it determine the values with a 50% and 10% filling fraction, respectively.)
There are also many 'global' aspects of the move patterns that are not taken into account. It seems that there should be some bonus for attacking orthogonally adjacent squares ('concentration'), which is the most likely explanation for the unexpectedly high value of the Archbishop. There probably should be a penalty for color binding, especially for severe forms of color binding. As it is, the algortithm would value an Alibaba about equal to a Knight, because it is not aware it can only access 1/4 of the board. Having only step moves (i.e. Shogi generals) should probably be penalized.