 |
Last update: 14/01/98 |
| Program | CPU/Mhz | Hash table | Move | Value | Plys/Max | Time | Notes |
|---|---|---|---|---|---|---|---|
| Chess Master 2175 | P100 | 2 Mb | Ka1-b1 | +1.00 | 21 | 00:00:17 | |
| Chess Genius 1.0 | P100 | 320 Kb | Ka1-b1 | +2.16 | 17/29 | 00:00:02 | seen at 00:00:01 |
 Comet-A.75 |
P100 | 13786 Kb | Ka1-b1 | +4.03 | 27 | 00:00:16 | 445.063 nodes |
| Rebel Decade 1.2 | P100 | 192 Kb | Ka1-b1 | +3.87 | 20 | 00:00:04 | 100.278 nodes |
Rebel Decade 2.0 |
P100 | 512 Kb | Ka1-b1 | +4.39 | 32 | 00:01:25 | 1.302.628 nodes |
 Rebel 8 |
K6/233 | 60 Mb | Ka1-b1 | +4.04 | 30 | 00:00:00 | instantly |
 Rebel 9 |
Pentium Pro 200 Mhz | 60 Mb | Ka1-b1 | +3.91 | 35 | 00:00:21 | seen at 00:00 |
| Crafty 12.7 | P100 | 12/5 Mb | Ka1-b1 | +3.85 | 22 | 00:04:29 | |
 Crafty 13.3 |
P6 | ? | Ka1-b1 | +4.086 | 26/27 | 00:00:06 | see notes |
 Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Ka1-b2 | +0.31 | 23 | 00:03:41 | see notes |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Ka1-b1 | +5.83 | 28 | 00:00:16 | |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Ka1-b1 | +7.26 | 31 | 00:00:55 | |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | ? | ? | 22 | ? | not found |
 Patzer 2.99y |
Sun Ultra1/167 Mhz | 32 Mb | Ka1-b1 | +4.65 | 18 | 00:00:01 | |
 Green Light Chess 2.04g |
6x86 P200+ | 24 Mb | Ka1-b1 | +2.85 | 24 | 00:00:02 | |
|
Notes: This kind of endgame can only be solved if the program has transposition tables, also known as hash tables. Otherwise, as Monroe Newborn states in Chess SKill in man and machine, pag. 129, a 30-ply search is necessary, and that would take several thousand hours of CPU time. On the other hand, a hash table easily copes with this, as, at a depth of 6 plies, there are only some 130 different positions reachable from the initial one. Once they are evaluated as wins or draws, the search proceeds much faster. Notice, anyway, that even so, Rebel Decade 1.2 has to look at no less than 100.278 positions to reach a depth of 20 plies. Without hash tables, however modest (RD1.2 is limited to 192k), that same number of plies would mean looking at trillions and trillions of positions. Rebel Decade 2.0, the newest version, features a larger hash table, 512 Kb, and finds the correct move very quickly, at a depth of 19 plies, evaluated at +1.77, in some 3 seconds. Letting it run till it reaches 30 plies increases the evaluation to +4.39, and still takes very little time. When this depth is reached, it has evaluated more than 1.3 million positions. Comet-A.75, another strong freeware program, uses a large 13 Mb hash table, and it reaches a depth of 27 plies in a very short time, finding the correct move with a good +4.03 evaluation. Thanks to the hash table, it had to evaluate only 445.063 positions. Rebel 8 running on very powerful hardware, and using an incredibly big 60 Mb hashtable, reaches 30 plies in no time and finds the correct move with a large +4.04 evaluation. Unfortunately, Rebel 8 is limited to 30 plies, so it can't go any deeper. The newest version, Rebel 9 increases this limit to 60 plies, so here it can reach 35 plies in a very short time, and finds the same move with almost the same value. It could see the correct move instantly, too. Crafty 12.7, a freeware program by Hyatt (Cray Blitz programmer), using a hash table of 12 Mb (plus another 5 Mb for pawn structure), goes to 22 plies and also finds the correct move, assigning it almost the same value as RD 1.2. Kai Luebke sent me the results for this test on his Pentium Pro at 200 Mhz. His machine has 64 Mb of RAM, and he run Crafty 12.6 using 24 Mb for the main hash table plus 16 Mb for the pawn hash table, under Windows 95. He also used all 3-man and 4-man endgame tablebases. However, it did not find the correct move, playing instead Kb2, which does not win. I asked Robert Hyatt about this, and he was kind enough to run Crafty 13.3 on this position, using a P6, large hash tables, and endgame tablebases. The program searched to 26/27 plies in a few seconds, finding both the correct move and a large gain. In the Addendum below, you can see that it found the winning move when it reached 22 plies, in less than a second, but evaluated only as +0.395. Notice too that, although the timing is very good, it could be much better, as the program only got 52% CPU time. Finally, 4.379 probes to the endgame tablebases were made, all successful !. Kai Luebke also tested Chess Master 5500 in his hardware, and it did extremely well, searching up to 31 plies in quite a short time, and finding the greatest gains. It accomplished that feat more than 10 times faster than Crafty 12.6, on a par with Crafty 13.3. On the other hand, most surprisingly, MChess Pro 5.0, on the same hardware and with a big 10 Mb hash table, failed to find the correct move, even when searching to 22 plies. A new gentle contributor, Roland Pfister is the author of a strong non-commercial program, Patzer 2.99y. He ran most of the problems of this suite at 7000 seconds per position and kindly sent me the results he got. In this position, running on a powerful Sun Ultra-1 at 167 Mhz under SunOS 5.5.1 (Unix), 64 Mb of RAM, and using 32 Mb hashtables total (including 1 Mb for positions hash, 256 Kb for material hash, and 256 Kb for pawn structures hash), it went to 18 plies in just one second and found the correct move with a healthy +4.65 evaluation, after examining 53.686 nodes. Good performance.
Addendum: I sent an e-mail to Robert Hyatt commenting on Crafty 12.6's strange result, not finding the correct move despite the very large hash tables. He kindly sent the following reply: " ... something is wrong somewhere, because every version I have finds this very quickly. Crafty 13.3 for example finds 1. Kb1 at 20 plies, and gets to ply=26 in 1 second with a score of +4.000... I have never seen a version that didn't solve this (this is called Fine #70 by many as it is in Fine's Basic Chess Endings book). Here is my Crafty 13.3 output. It might be that Crafty 12.6 was somehow broken... Safest Crafty 12.x version is the last one which has no known problems at present... Crafty 12.9 finds it in 20 plies, 15 seconds. Newer hashing in Crafty 13.3 does this:"
depth time score variation (1)
22 0.68 0.195 Kb2 Ka8 Kc3 Kb7 Kc4 Kb6 Kd3 Kc7 Ke3
Kd7 Ke2 Kd8 Kf3 Ke7 Kg3 Kf6 Kf2 Ke7
Kf3 Ke8 Kg3 Kf7
22 0.81 0.395 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6
22-> 0.86 0.395 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6
23 0.98 0.395 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6 Kd3
23-> 1.03 0.395 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6 Kd3
24 1.26 ++ Kb1!!
24-> 1.30 0.694 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6 Kd3
25 1.37 -- Kb1
25 1.85 ++ Kb1!!
25-> 1.87 0.993 Kb1 Kb7 Kc1 Kc7 Kd1 Kd8 Kc2 Kc8 Kd2
Kd8 Kc3 Kc7 Kd3 Kb6 Ke3 Ka6 Ke2 Kb6
Kf3 Ka6 Ke3 Kb6 Kd3
26 2.42 ++ Kb1!!
26 5.90 4.086 Kb1 Kb7 Kc1 Kc7 Kd1 Kb7 Ke2 Kc7 Kf3
Kd7 Kg3 Ke7 Kh4 Kf6 Kh5 Ke7 Kg6 Ke8
Kxf5 Kf7 Kg5 Ke7 f5 Ke8 f6 Kf7 Kf5
26-> 5.98 4.086 Kb1 Kb7 Kc1 Kc7 Kd1 Kb7 Ke2 Kc7 Kf3
Kd7 Kg3 Ke7 Kh4 Kf6 Kh5 Ke7 Kg6 Ke8
Kxf5 Kf7 Kg5 Ke7 f5 Ke8 f6 Kf7 Kf5
27 7.47 4.086 Kb1 Kb7 Kc1 Kc7 Kd1 Kb7 Ke2 Kc7 Kf3
Kd7 Kg3 Ke7 Kh4 Kf6 Kh5 Ke7 Kg6 Ke8
Kxf5 Kf7 Kg5 Ke7 f5 Kd7 f6 Ke8 Kg6
Kd7
27-> 7.54 4.086 Kb1 Kb7 Kc1 Kc7 Kd1 Kb7 Ke2 Kc7 Kf3
Kd7 Kg3 Ke7 Kh4 Kf6 Kh5 Ke7 Kg6 Ke8
Kxf5 Kf7 Kg5 Ke7 f5 Kd7 f6 Ke8 Kg6
Kd7
time: 10.79 cpu:52% mat:1 n:566282 nps:100049
ext-> checks:26985 recaps:331 pawns:33997 1rep:18534
predicted:0 nodes:566282 evals:32901
endgame tablebase-> probes done: 4379 successful: 4379
|
| Program | CPU/Mhz | Hash table | Move | Value | Plys/Max | Time | Notes |
|---|---|---|---|---|---|---|---|
| Chess Master 2175 | P100 | 2 Mb | Nb2-d3 | +2.68 | 13 | 00:00:27 | sees to 3 ..K-Q6 |
| Chess Master 2175 | P100 | 2 Mb | Nb2-d3 | +2.83 | 19 | 01:08:40 | sees the same |
| Chess Genius 1.0 | P100 | 320 Kb | Nb2-d3 | +3.45 | 10/22 | 00:00:17 | sees to KxP |
Comet-A.75 |
P100 | 13786 Kb | Nb2-d3 | +3.78 | 10 | 00:00:39 | |
| Rebel Decade 1.2 | P100 | 192 Kb | Nb2-d3 | +3.18 | 11 | 00:00:40 | 545.254 nodes |
Rebel Decade 2.0 |
P100 | 512 Kb | Nb2-d3 | +3.27 | 11 | 00:00:27 | seen at 12 sec. |
Rebel 8 |
K6/233 | 60 Mb | Nb2-d3 | +1.86 | 10 | 00:00:02 | low evaluation |
Rebel 9 |
Pentium Pro 200 Mhz | 60 Mb | Nb2-d3 | +3.86 | 14 | 00:00:32 | +3.31 at 12 ply, 9" |
| Crafty 12.7 | P100 | 12/5 Mb | Nb2-d3 | +3.315 | 11 | 00:00:49 | |
Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Nb2-d3 | +3.34 | 11 | 00:00:15 | see notes |
Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Nb2-d3 | +4.24 | 12 | 00:01:21 | see notes |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Nb2-d3 | +3.90 | 10 | 00:00:13 | |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Nb2-d3 | +5.22 | 12 | 00:01:02 | |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | Nb2-d3 | +4.97 | 8 | 00:00:20 | |
Patzer 2.99y |
Sun Ultra1/167 Mhz | 32 Mb | Ke3-f3 | +1.87 | 20 | 01:49:14 | can't see it |
Green Light Chess 2.04g |
6x86 P200+ | 24 Mb | Nb2-d3 | +3.02 | 12 | 00:03:29 | see notes |
|
Notes: This position requires quite a deep search before the rook pawn's promotion threat can be acknowledged, thus forcing white to trade its knight for that pawn, losing the game. However, the improved algorithms of Chess Genius 1.0 allow him to see more with a 10-ply search than Chess Master 2175 with a 19-ply one. This is accomplished through the additional 12-ply quiescence search that it carries at terminal nodes. The effective 22-ply search thus sees the exact first 7 plies, against only 5 of CM2175, and does it almost 300 times faster. On the other hand, Rebel Decade 1.2 also does well, seeing the gain with just an 11-ply search, which requires looking at a little over half a million positions. But the newest version, Rebel Decade 2.0 does even better. Thanks probably to its larger hash table (512 Kb vs 192 Kb), it finds the correct move with a slightly higher evaluation (+3.27 vs +3.18) almost 2 times faster. It sees the correct move even sooner, at 10 plies, in 12 sec., but evaluated only as +1.86. Freeware Comet-A.75 performs very similarly. It finds the correct move at a depth of 10 plies, with a healthy +3.78 evaluation, very quickly. Letting it search deeper, it reaches a depth of 12/14 plies in 1 min. 57 sec., considering 3.038.612 positions, but the evaluation increases only very slightly, +3.87. Rebel 8 also finds the correct move with a 10-ply search, in just a couple of seconds, but with the very low +1.86 evaluation, which doesn't reflect much apart from the +1.00 material advantage. A deeper search would surely improve the evaluation a lot. Rebel 9 does perform the deeper search, 12 plies instead of 10, taking just 9 seconds, and finds the correct move evaluated at +3.31 . When it reaches 14 plies, the evaluation raises to +3.86, also in a very short time. However, it compares badly to Chessmaster 5500 running in exactly the same hardware, as CM5500 needs to look at only 10 plies (instead of 14) to reach the same evaluation (+3.90 vs +3.86), and does it more than 2 times faster. The comparison against old Chess Genius 1.0 is even worse. Crafty 12.7 searches also to 11-ply, in almost the same time that Rebel Decade 1.2, and finds the same move with nearly the same value too, though it has much larger hash tables. Crafty 12.6, running on powerful hardware and with very large hashtables and all 3 and 4 pieces endgame tablebases, finds the correct move and sees the win very fast, in a par with CG1.0. Chess Master 5500 performs very like Chess Genius 1.0, finding the same move in a 10-ply search, and similar evalution too, but taking into account the vast difference in hardware, it does it some 2 to 3 times slower, though still very fast. A 12-ply search discovers even greater gains in quite a short time too.MChess Pro 5.0 only needs 8 plies to see a similar gain, and it does it quite fast, nearly 3 times faster than the equivalent 12-ply search of Chess Master 5500. Patzer 2.99y is the only program among those tested that fails to discover the correct move. It looks 20 plies deep taking nearly 2 hours, examines 232.411.394 nodes, yet it fails to deliver the goods. Perhaps some bug ? Green Light Chess 2.04g finds the correct move at 10 ply, in 20.38 seconds, but not with any particularly high evaluation. It doesn't improve the evaluation until 11 ply, in 2 min. 02 seconds (evaluation: +2.98), and gets the correct Principal Variation at 12 ply, as seen in the table. All in all, it looked at almost 8 million nodes.
|
| Program | CPU/Mhz | Hash table | Move | Value | Plys/Max | Time | Notes |
|---|---|---|---|---|---|---|---|
| Chess Master 2175 | P100 | 2 Mb | Bb6-d8 | +0.00 | 15 | 03:20:24 | can't see it |
| Chess Genius 1.0 | P100 | 320 Kb | Rd2xb2 | +0.57 | 12/24 | 01:42:17 | sees the win |
Comet-A.75 |
P100 | 13786 Kb | Bb6-d8 | +0.15 | 13 | 01:27:23 | can't see it |
| Rebel Decade 1.2 | P100 | 192 Kb | Bb6-d8 | +0.25 | 13 | 05:00:00 | can't see it |
Rebel Decade 2.0 |
P100 | 512 Kb | Bb6-d8 | +0.23 | 13 | 02:00:58 | can't see it |
Rebel 8 |
K6/233 | 60 Mb | Bb6-d8 | +0.27 | 14 | 00:16:10 | not found |
Rebel 9 |
Pentium Pro 200 Mhz | 60 Mb | Bb6-d8 | +0.23 | 12 | 00:05:19 | not found |
| Crafty 12.7 | P100 | 12/5 Mb | Rd2xb2 | +1.804 | 12 | 00:08:47 | seen at 5m06s |
Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Rd2xb2 | +1.90 | 12 | 00:02:09 | see notes |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | c4-c3 | +0.05 | 13/16 | 01:09:38 | doesn't see the win |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | Rd2xb2 | +1.38 | 8 | 00:03:13 | seen at 2:56 |
Patzer 2.99y |
Sun Ultra1/167 Mhz | 32 Mb | Rd2xb2 | +0.93 | 15 | 00:55:05 | see notes |
Green Light Chess 2.04g |
6x86 P200+ | 24 Mb | Rd2-d4 | -0.11 | 9 | 00:05:00 | doesn't see the win |
|
Notes: In this incredible position, black sacrifices first its rook, then its bishop, and manages to win the game with only a pair of doubled, passed pawns against white's rook and knight. To see that those amazing sacrifices are really winning ones, a great depth of search is required, made even more difficult due to the large number of captures and promotion threats present. Anyway, Chess Genius 1.0 finally sees the winning sequence, after quite a long time, while Chess Master 2175 simply cannot, even using double the time. Comet-A.75, despite its very large hash table compared to CG1.0's (13786 Kb vs 320 Kb) goes to a depth of 13 plies in a similar time, examines 125.529.331 positions, yet it fails to discover the winning move. I'm pretty sure the much better and/or accurate search extensions are the key in this case. Rebel Decade 1.2, using triple the time as CG1.0, also fails to see the winning move, even though it looks at no less than 238 million positions (237.932.551 to be exact !), which shows this problem is no piece of cake. Its newest incarnation, Rebel Decade 2.0 does exactly the same: it also searches 13-ply deep, and finds the same move with almost the same value, +0.23. Even though it examined 121.341.241 positions it couldn't find the correct move. Notice it did almost the same as Rebel 9, only more than 20 times slower. Even taking into account the faster hardware, no doubt why they call it a downgraded version. Rebel 8, its commercial, full-featured sibling, looks one ply deeper, 14 plies, some 20 times faster, but cannot see the solution either. It just finds the same move, with nearly the same evaluation, +0.27. Most dissapointingly, Rebel 9, the newest and strongest version, also fails to see the winning move, though it searches to 12 plies, as non-commercial, free Crafty 12.6 and Crafty 12.7, which can find it in a couple of minutes. The comparison against commercial MChess Pro 5.0 is even worse, as MC needs to look at only 8 plies to see the win. Freeware Crafty 12.7 shows its strength with this one, searching also 12 plies, as CG1.0, but 12 times faster, and finding the correct winning move. No doubt the large hash table helped quite a lot here. Crafty 12.6 does even better, and thanks to its powerful hardware, large hashtables, and endgame databases, finds also the winning move in the shortest time. It even sees it at 00:00:50, but evaluated at only +0.44. Chess Master 5500 cannot find the correct move even though it looks at 13/16 plies, running a little over 1 hour on powerful hardware. It doesn't see the win at all. But MChess Pro 5.0, running on the same hardware and with a hash table of 10 Mb, discovers the winning move while looking at just 8 plies in a very short time, though it takes a little longer and sees less gain than Crafty 12.6. Patzer 2.99y searches to a depth of 15 plies taking nearly one hour, examines 95.868.135 nodes, and finds the correct move, with a +0.93 evaluation. The timing is much worse than that of Crafty or commercial MChess Pro 5.0, but at least it does find the correct move in this difficult position. Green Light Chess 2.04g searches to a depth of 9 plies in exactly 5 minutes, but though it examines 10.188.369 nodes it can't see the winning Rook sacrifice.
|
4.- Unknown players
| Program | CPU/Mhz | Hash table | Move | Value | Plys/Max | Time | Notes |
|---|---|---|---|---|---|---|---|
| Chess Master 2175 | P100 | 2 Mb | Nc5-d3+ | +3.09 | 15 | 00:30:28 | |
| Chess Genius 1.0 | P100 | 320 Kb | Nc5-d3+ | +3.09 | 9/21 | 00:00:32 | |
Comet-A.75 |
P100 | 13786 Kb | a5-a6 | +2.51 | 12 | 00:24:04 | can't see it |
| Rebel Decade 1.2 | P100 | 192 Kb | a5-a6 | +2.24 | 12 | 00:29:33 | can't see it |
Rebel Decade 2.0 |
P100 | 512 Kb | a5-a6 | +1.91 | 13 | 00:38:09 | can't see it |
Rebel 8 |
K6/233 | 60 Mb | Nc5-d3+ | +3.55 | 14 | 00:11:01 | |
Rebel 9 |
Pentium Pro 200 Mhz | 60 Mb | Nc5-d3+ | +2.90 | 10 | 00:00:17 | |
| Crafty 12.7 | P100 | 6 Mb + 640k | Nc5-d3+ | +3.950 | 9/13 | 00:00:33 | |
Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Nc5-d3+ | +4.06 | 10 | 00:00:11 | see notes |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Nc5-d3+ | +3.61 | 9 | 00:01:08 | seen at 19 sec. |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | Nc5-d3+ | +4.01 | 7 | 00:00:30 | seen at 11 sec. |
Patzer 2.99y |
Sun Ultra1/167 Mhz | 32 Mb | Nc5-d3+ | +3.70 | 13 | 00:01:46 | see notes |
Green Light Chess 2.04g |
6x86 P200+ | 24 Mb | Kg1-f2 | +2.40 | 9 | 00:04:38 | can't see it |
|
Notes: White sacrifices a knight, so that he can promote a pawn. A program must perform at least a 15-ply search to see the unavoidable promotion, and the more advanced quiescence search of Chess Genius 1.0 does it with a 9-ply search, complemented with 12 additional plies of captures and other threats. Chess Master 2175 requires a full 15-ply search to see the same, rendering it some 50 times slower. Rebel Decade 1.2, searching to a depth of 12 plies in the same time as CM2175, fails to see the correct move. It had to evaluate 25.224.808 positions to reach that depth, and probably a larger hash table (RB1.2 is limited to 192 Kb, against the 2 Mb used by CM2175) would have helped no end in this case. The newest version, Rebel Decade 2.0 also fails to find the correct move. It looks one ply deeper, 13 plies, but it doesn't see the correct sacrifice. Reaching that depth took almost 40 min., evaluating 31.183.667 positions. By the way, if Rebel Decade 2.0 is really a downgraded version of Rebel 9, perhaps it's been downgraded too much, judging from Rebel 9's performance in this same test (it finds the correct move at a depth of only 10 plies in 17 seconds). Comet-A.75 fails too, in almost exactly the same way as RD 1.2. It also looks at 12 plies, and finds 1. a5-a6 instead of the correct Knight sacrifice, with a similar evaluation and in the same time. However, it evaluates significantly more positions in that time, 37.190.210, but doesn't see the correct move either, despite the very large hash table. Rebel 8, the commercial version, allegedly some 200 ELO points stronger than RD 1.2, does find the correct move, but after searching to 14 plies it evaluates it at +3.55, which is more or less the same as Chess Master 5500's evaluation (+3.61) at 9 plies. Also, the timing is not very good despite the powerful hardware. Rebel 9, the newest version, looks at 10 plies and finds the correct move, with a somewhat lower evaluation, though it does it very quickly. However, both non-commercial Crafty 12.6 and commercial MChess Pro 5.0 find significantly more gain (+4.0 vs +2.90) in comparable times. Freeware Crafty 12.7 does use two hash tables: a huge 6 Mb one for the transpositions table, and a smaller 640k for the pawn structures. This results in finding the correct sacrifice in very little time, searching a whopping 1.090.120 nodes, but efficient pruning means that it needs to apply the evaluation function to only 113.707 of them. And, as usual, Crafty 12.6, thanks to its large hash and endgame tablebases, not to mention the CPU, finds the correct winning move after looking at 10 plies in the shortest time. Chess Master 5500 finds the winning move with a 9-ply search, but it's 6 times slower than Crafty 12.6 and Chess Genius 1.0, taking into account the hardware difference. MChess Pro 5.0 does much better, and only needs a 7-ply search, in half the time than Chess Master 5500, to see more or less the same. Patzer 2.99y does quite well. It searches to a depth of 13 plies, 4 plies more thant Chess Master 5500, in a comparable time, and finds the correct move with a very similar evaluation after examining 3.029.904 nodes. Green Light Chess 2.04g can't find the winning move. It looks at 9 plies, examines some 10 million nodes, but fails to see the correct Knight sacrifice. Tim Foden ran these tests at just 5 minutes maximum time. Perhaps some extra time would have helped.
|
| Program | CPU/Mhz | Hash table | Move | Value | Plys/Max | Time | Notes |
|---|---|---|---|---|---|---|---|
| Chess Master 2175 | P100 | 2 Mb | Bd4xf6+ | Mate8 | 11 | 00:00:29 | sees mate in 8 |
| Chess Genius 1.0 | P100 | 320 Kb | Bd4xf6+ | Mate8 | 6/18 | 00:00:02 | sees mate in 8 |
| Chess Genius 1.0 | P100 | 320 Kb | Rh3-h6 | Mate5 | 9/10 | 00:00:03 | level=mate5 |
Comet-A.75 |
P100 | 13786 Kb | Bd4xf6+ | Mate7 | 9 | 00:00:31 | not shortest |
| Rebel Decade 1.2 | P100 | 192 Kb | Rh3-h6 | Mate5 | 9 | 00:00:10 | level=mate5 |
| Rebel Decade 1.2 | P100 | 192 Kb | Bd4xf6+ | Mate8 | 6 | 00:00:01 | sees mate in 8 |
Rebel Decade 2.0 |
P100 | 512 Kb | Bd4xf6+ | Mate8 | 6 | 00:00:02 | seen at 1 sec. |
Rebel 8 |
K6/233 | 60 Mb | Bd4xf6+ | Mate8 | 7 | 00:00:00 | instantly |
Rebel 9 |
Pentium Pro 200 Mhz | 60 Mb | Bd4xf6+ | Mate8 | 7 | 00:00:01 | not shortest |
Crafty 12.6 |
Pentium Pro 200 MHz | 24 Mb + 16 Mb | Bd4xf6+ | Mate7 | 9 | 00:00:01 | see notes |
Chess Master 5500 |
Pentium Pro 200 Mhz | ? | Rh3-h6 | Mate5 | 8 | 00:00:09 | |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | Rh3-h6 | Mate6 | 8 | 00:01:23 | see notes |
MChess Pro 5.0 |
Pentium Pro 200 Mhz | 10 Mb | Rh3-h6 | Mate6 | 8 | 00:00:15 | level=mate6 |
 Fritz 3.10 |
Cyrix P166/48Mb | 18 Mb | ? | Mate7 | 9/15 | 00:00:19 | |
Fritz 4.1 |
Cyrix P166/48Mb | 18 Mb | ? | Mate7 | 5/13 | 00:00:08 | |
Fritz 5 |
Cyrix P166/48Mb | 18 Mb | ? | Mate7 | 9/14 | 00:00:01 | fastest |
Hiarcs 6.0 |
Cyrix P166/48Mb | 18 Mb | ? | Mate7 | 8/25 | 00:00:09 | |
Patzer 2.99y |
Sun Ultra1/167 Mhz | 32 Mb | Bd4xf6+ | Mate7 | 1 | 00:00:00 | see notes |
Green Light Chess 2.04g |
6x86 P200+ | 24 Mb | Bd4xf6+ | Mate6 | 8 | 00:00:06 | sees mate in 6 |
|
Notes: This is a problem intended for human chess players, and as such it's quite difficult. Testing a number of programs reveals that it makes a large difference if one tries to solve it in an specific Mate level or using the Infinite (indefinite more so) level. In this example, the Mate levels are much faster than the regular one. Also, note that Chess Master 2175, which does not have special Mate levels, does not find the required mate in 5 but a slower mate in 8. Chess Genius 1.0, in its indefinite level also finds that same mate in 8, just some 10 times faster, while in its special Mate in 5 level it finds the correct move, also very quickly. Comet-A.75 is freeware, as Crafty, but in this test it does much worse. Both look at the same depth, 9 plies, find the same move evaluated as mate in 7 in both cases, none of them finds the correct move which gives mate in 5. But Comet does it many times slower than Crafty or any of the other programs tested, wit the exception of old Chess Master 2175 and new MChess Pro 5.0. It searched a total of 1.113.190 positions before finding the mate. Rebel Decade 1.2, in its infinite level, also finds the same slow mate in 8 as the rest, though it finds it amazingly fast, 3 times faster than CG1.0, looking at only 20.469 positions !. The correct mate in 5 is found when using its special Mate in 5 level, this time looking at 618.312 positions in a short time, though still 3 times slower than CG1.0. New Rebel Decade 2.0 does exactly the same as Rebel Decade 1.2. The mate is seen really fast, after less than 1 sec. calculation, and is played before 2 sec. have elapsed. It examined only 27.759 positions, 35% more than RD1.2. Both Rebel 8 and Rebel 9 behave the same as their weaker cousins, and find also the slow Mate in 8, but thanks to the very powerful hardware, do it instantly. They need to look 7 plies deep, one more than Rebel Decade 1.2. Crafty 12.6 looks at 9-ply and finds almost instantly a longish mate in 7, instead of the correct mate in 5. It's somewhat odd, as 9 plies would be sufficient to find a mate in 5, it seems. Chess Master 5500 does fine, finding the correct move with an 8-ply search, and evaluating it as a mate in 5, all in quite a short time. But MChess Pro 5.0 doesn't do as well. First, it finds instantly Bxf6+, and evaluates it as a longer mate in 8. Some eigth seconds later, it reevaluates the same move as a still too long mate in 7. And finally, an 8-ply search discovers the correct Rh6 but evaluated as a mate in 6, not in 5. Anyway, the timing is much worse than Chess Master 5500, say. All Fritz-family engines did fairly well, though none found the shortest mate, but a mate in 7. Fritz 3.10 was the slowest, looking at roughly the same depth (9/15 vs 9/14) as Fritz 5, which was the fastest, only equalled by Crafty 12.6. Surprisingly, the Fritz 4.1 engine only needed to look at 5/13 plies, but its timing was midway the other Fritzs. The Hiarcs 6.0 engine had a timing comparable to that of Fritz 4.1 though it looked more deepely (8/25 vs 5.13). By the way, Mike Cooter gently provided all these data, but forgot to mention which moves were selected !. Patzer 2.99y finds the mate in 7 instantly. It does not find the shortest mate in 5, but the mate in 7 is found at ply 1, almost in no time at all, which is the best result among those programs which found this mate in 7 with Bd4xf6+. Green Light Chess 2.04g performed quite well. It couldn't find the shortest mate in 5 but a mate in 6, in a few seconds, while looking 8 plies deep. It examined only 337.605 nodes before finding the forced mate.
Addendum: I sent an e-mail to Robert Hyatt commenting on why Crafty failed to find the shortest mate. This is an extract of my questions and his kind answers: VA: " ... Crafty 12.6 looks at 9 plies and finds a mate in 7, but a shorter mate in 5 does exist. Why it doesn't find that shortest mate ? It seems that 9 plies would be just enough to see a mate in 5. Isn't it dangerous to play a mate in 7 instead of the correct one ? I mean, perhaps after playing it, at its next move it finds a mate in 12 and plays it, then a mate in 8 and plays it, continuing playing mate-in-xx moves without *actually* giving mate. Can this happen ?" RH: "No. See below. It will always search until mate-in-(N-1) after making a move that leads to mate-in-N." VA: "... something similar happens for Test 6, where it finds a mate in 8 and, one ply deeper, a mate in 6, but not the mate in 5." RH: "This is not a problem, really, because once it finds a mate in N moves, after the opponent makes a move, it won't stop searching until it finds a mate in N-1 moves. However, in a game, due to the way search extensions work, it is common for it to find a deeper mate than is really there, particularly when the shorter mate has non-checking moves in the PV and the deep mate is all checks ..."
|
