Monday, August 24, 2015

Correspondence Chess - Reform is Long Overdue


In a previous blog post, I mentioned GM Arno Nickel's open letter concerning the excessive draw problem in correspondence chess.

GM Nickel promised to distribute a summary of responses to all who responded to his survey. Today my copy arrived. I found the results most interesting and wish to thank GM Nickel for his efforts. The survey was about his proposal to alter the scoring of drawn games. However, his conclusions about engine detection must be considered opinion not supported by facts. GM Nickel's comments are shown below in quotation marks.

"A radical measure claimed over and over again is the engine ban. It has to be distinguished from the additional offer of "engine - free play" which is supported, for example, by the BdF and by many free chess servers, namely as a kind of competition based on voluntary arrangement."

Not being able to read German well, I cannot discover a relationship between Bdf and the server, but I did locate a non-engine event. Here is a game between two strong players. Top 3 analysis below is with Stockfish 6 to 30 ply depth, multi-pv=3. "Book" or database opening moves are not counted in Top 3, nor are "forced" moves (f) such as responding to checks, or moves that recover material during exchanges.

[Event "non-engine"]
[Site ""]
[Date "?"]
[Round "?"]
[White "redacted"]
[Black "redacted"]
[Result "1/2-1/2"]
[ECO "C68"]
[WhiteElo "2300+"]
[BlackElo "2400+"]

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Bxc6 dxc6 5. O-O Bg4 6. h3 h5 7. d3 Qf6
8. Nbd2 Ne7 9. Re1 Ng6 10. d4 {end of book: Chow - Ivanov, Dallas 1996,
1/2 - 1/2 in 45 moves.}

10. ...                           Nf4     T1
11. dxe5    T1              Qg6    T1
12. Nh4     T1              Bxd1   T1
13. Nxg6    (f)              Nxg6    T1
14. Rxd1    (f)              O-O-O T1
15. e6        T1             fxe6     T1
16. Re1     T1             Ne5     T1
17. Nb3     T1             Bb4     T2
18. c3        T1             Bd6     T1
19. Bg5     T1             Rd7     T1
20. Red1   T1             Rf8     T2
21. a4        T1             Rdf7   T1
22. Nd4      T1            Rxf2    T1
23. Nxe6     (f)            R8f7    T1
24. Nd8      T1            Rf8     T1
25. Ne6      T1            R8f7   T1
26. Nd8      T1            Rf8     T1
27. Ne6      T1

White has T1 14/14 = 100% agreement with the engine's top recommendations.
Black has T1 15/17 = 88% agreement with the engine's top recommendations.
So much for engine-free, gentleman's agreement chess. As GM Nickel famously quips, "black sheeps can be everywhere."

Using the Top 3 methodology described in this article, I have never found a correspondence game played  prior to 1980 (i.e., before chess micro devices, before personal computers) where T1 matching exceeded 71%. In most games, the percent was far less, even at the world championship level. That some of today's amateur cc players match engine output 80%, 90% even 100% in non-engine tournaments defies credulity.

Looks like Bdf's "voluntary arrangement" doesn't work any better than the "honor system" here in the States, where engine abuse is rampant in no-engine events.

"A strict 'ban' is more than this, as it goes for active control methods and drastic sanctions in order to guarantee a fair competition and avoid as far as possible grey areas. As control mechanism some players think of copying methods from real time on - line play. This way a player who’s moves are, for example, more than 70% the same as the first choice by engines might be considered as convicted of cheating. In my view such an approach is no more than wishful thinking as it totally ignores the differences between real time on - line play (mostly blitz and rapid chess) and correspondence chess ... Already the use of databases in correspondence chess would be a problem to find the right move, where to start with controls. While in on - line chess time usage is an important criterion for proving cheating (players spend the same time on easy moves as highly difficult moves), this aspect does not apply to correspondence chess." 

Top 3 analysis is based on frequency counts and their percentage of total moves played. It does not use "secret methods," "time" or any of the methods of on-line server admins to police cheating in fast play (blitz) chess. Active control methods are a necessary part of engine detection, but "drastic sanctions" aren't the best solution. What business can survive by driving away its customers? Organizations should simply create a separate division for advanced chess, just like they do for Chess960. New players are free to choose which type of chess they wish to play, traditional (no engine) chess or advanced (engine-assisted) chess. Separate rules, events and rating lists would be maintained. Traditional players guilty of engine abuse are permanently moved into the advanced chess group. Players may not participate in both groups; advanced chess players may never move to the traditional (no engine) group.
     As explained above, Top 3 calculations do not include "book" or database opening moves - these moves are ignored. Players in traditional no-engine chess may follow any published (publicly available) chess game played before 1980, and any published otb game played after 1980. Published correspondence chess games played after 1980 are "verboten;" copying such moves in traditional cc games is committing Top 3 suicide because they will drive up % matching.

"Besides it is completely unclear how to value a correspondence of the played move with an “engine move”, when there are many equivalent computer moves like in stages of mainly positional play. This could happen just by chance, same as in case of forced moves a correspondence with “engine moves” would have to exist as the player would otherwise just lose (a piece or more)."

As explained above, "forced" moves are not included in Top 3 calculations. It's also clear combinations to win material and mating attacks are something players can find without an engine's help. These sequences are exceptions and we don't count "forced" moves. The majority are "quiet" positions where several moves have nearly equal evaluations. In positions where the evals are very close to one another, unaided human players can't differentiate differences of a few hundredths of a pawn. Yet some players' moves match an engine's top recommendations well in excess of 70%.
      In positions where two or more moves are tied at a 0.00 eval, and the engine can't break the tie no matter how many plies are examined, it's a theoretical draw. The engine-assisted player will unerringly steer the game to a draw, even if it takes 100 or more moves. Natural players often lose such positions because they play like humans, failing to find the precise drawing line.
     Strong players can estimate a half-pawn advantage in a position (engine eval for this is 0.50). Human players cannot discern a few hundredths of a pawn. When the engine determines the top 3 moves are + 0.08, + 0.04 and +0.01, the odds are strongly against a natural player making the T1 move, especially matching such minuscule T1 evals many times in a game. Alarmists are always claiming Top 3 will wrongly convict innocent people who play "sharp" (tactical) chess, or that players could match computer output "by chance." Such critics display ignorance of Top 3 methodology and/or a total lack of understanding of mathematical probability. In contrast to natural chess play, one rarely sees combinations and mating attacks in advanced chess because engines don't ever fall into such predicaments. Contrary to "conventional wisdom", it is those non-tactical, "quiet" positions that will be most revealing of engine abuse.

"There are many other arguments, why the concept of an enforced engine ban is definitively condemned to failure, no matter how we think about usage of engines in correspondence chess. Support of engine - free play makes only sense as an additional offer and it requires that all players strive for nothing more than fun and honour, without precious prices and qualifications, as otherwise cheating would dramatically increase. As has been reported in forums even fun tournaments with engine - free play are by no means absolutely safe from cheating. Black sheeps can be everywhere."

Cannot agree engine detection is "condemned to failure" but separating traditional chess from advanced chess is a long overdue reform! The two types of chess can co-exist (just don't co-mingle) peacefully in an organization. There is no need to promote one at the expense of the other. Players are customers and organizations need to respect the wishes of all players.

GM Nickel concludes: "However, we are still waiting for long-term plans how to deal with the challenge posed by the increasing strength and domination of chess programs, and correspondence chess databases that are getting bigger and bigger. The answers given by ICCF appear to be evasive and defensive; to my mind they also fail to include the ICCF members in a discussion about the question how correspondence chess sees and presents itself. The answers to the survey show above all one thing: in view of the rising number of draws and the dominance of computer engines a huge number of ICCF members and a lot of chess fans feel the need for concrete action to keep correspondence chess attractive or, if there is no other choice, to reinvent its attraction."

Almost immediately following the ChessBase open letter, survey analysis and conclusions, ICCF rushed to post an interview with Ron Langeveld, ICCF's 26th World Champion.

In this interview, GM Langeveld offers some opinions supportive of ICCF's status quo regarding advanced chess rules. He states: "A high draw ratio in itself is not a problem. It’s not that game replay would suffer due to an increased draw ratio." Not sure typical players and fans agree with that. Playing through 75-100 (and more) move games that end in draw after draw after draw is pretty boring stuff. Gone are the brilliant combinations and slashing attacks of yesterday. GM Langeveld calls any player in disagreement "mediocre." The overwhelming majority of chess players are not super GM's and it is their entry fees that sustain an organization. It is incongruent for GM Langeveld to belittle average players and then conclude his interview with a wish for lower entry fees. 

Friday, July 24, 2015

Another "Fix" for Advanced Chess ?

Last month ICCF GM Arno Nickel published an open letter on the popular Chessbase web site, lamenting the "draw death of correspondence chess" and proposing a new scoring system for certain types of drawn games. You may read the complete article and commentary at .

The thread running through many responses has nothing to do with the proposal, but rather the question of whether use of chess engines (in the selection of moves in live games) should be permitted at all. The editorial remarks preceding the article are telling: "For some the growing use of computers is a modern day curse ... the main problem is the resulting increase of draws."

The real problem is a growing dissatisfaction (with engine-assisted correspondence chess) leading to world-wide decline in participation. Closely related is the problem of multiple tie scores in tournaments and how to declare winners in an equitable fashion. The traditional Sonneborn-Berger tie-break method (SB points) is liable to fail as more and more competitors finish in a dead heat. ICCF has already prioritized the Bamberger Rule ahead of S-B; Bamberger counts number of actual wins instead of total scores (wins plus draws.) However, as chess engines (and the processors they run on *) continue to improve, the draw rate will exceed the current 90 % and the number of wins from actual play will decrease. Bamberger may be deciding event winners on the basis of sporadic time forfeits. In the 28th WC Finals, GM Nikolai Papenin (highest-rated player at start of the event) finished in last place with an unprecedented six losses. Due to the war situation in his native Crimea, he wrote he had "no time for chess." Had he been playing under normal circumstances, scoring six more draws instead of losses (one of these by time forfeit),  ICCF's 28th World Championship would be at 94 % draws.


Commenting in the above-mentioned article, ICCF GM Tansel Turget stated: "Until recently, human could add 200-300 points to the strength of the chess engine, but this additional human input is decreasing with the improvement of chess engines (against a computer with no human input, I think we (human+computer) can still score 70-80%). If my theory is correct (may not be), then there is still a lot of human influence in cc, which is very encouraging. We may not be able to effect the result of a game every game, but we still have enough influence to change the result in a significant number of games."

One ICCF official opined: "The reason ICCF allows them is that there is not a good way to enforce a no computers rule." So much for centaur chess being the highest art form; it exists by default, not design. The inevitable march of the machines toward 100% draws doesn't leave much room for "significant" human contribution. ICCF's band-aid approach (Bamberger Rule, Nalimov adjudications, Nickels' proposal etc) will not rescue Advanced Chess from the draw death. What is needed is for ICCF to acknowledge Advanced Chess was a mistake and to return to traditional cc, following FIDE's lead in implementing Dr. Ken Regan's (or similar) statistical engine detection system.

Postscript. In the world of traditional (no-engine) correspondence chess, "engine abuse" is a huge concern, but the problem is largely of the organizations' own making. Unethical players utilize chess computers in events where they are expressly forbidden by the rules. Such shenanigans have gone on for thirty-five years, ever since 1980 and the introduction of stand-alone micro devices followed by software (engines) for the exploding pc market. Sadly, these "no engine" groups have inconsistent rules enforcement or, more likely, make absolutely no effort to detect "engine abusers." Draws are not the problem here, it's "wins" by players rated 500 or more ELO points below their opponents, or "wins" where moves by the victor match engine output at a rate approaching 100%. Chess ratings, titles and prizes have become meaningless in these venues. Declining membership in traditional cc organizations is the elephant in the room.

Thursday, February 5, 2015

It's All About the Ply Depth

In recent posts, we have revealed some truths about so-called "Advanced Chess" (computer-assisted play) as well as advice for players who are new to this venue; see links below.

In his syndicated column, GM Lubomir Kavalek annotated  Ivanchuk-So, Wijk aan Zee 2015, a game featuring a recent TN (Theoretical Novelty) in the Ruy Lopez credited to a chess engine. Kavalek writes:

"It seems So surprised Ivanchuk with a knight sacrifice in the delayed Marshall Attack in the Spanish opening. The Ukrainian just followed Aronian's analysis from his Candidates game against Anand. But it became clear that Aronian's analytical team unplugged the computer too soon, leaving the piece sacrifice undiscovered. Suddenly, there it was on the board, the work of a number-crunching monster, too foreign to a human mind. And Wesley So knew about it."

From What is Advanced Chess ? "There is nothing revolutionary about opening preparation helping to win chess games - chess masters have been unleashing opening surprises on their opponents for 500 years. What is revolutionary is the ability of chess software to find TN's so deeply hidden that an unassisted human player could never discern them".

[Event "77th Tata Steel GpA"]
[Site "Wijk aan Zee NLD"]
[Date "2015.01.18"]
[Round "8.3"]
[White "Ivanchuk, Vassily"]
[Black "So, Wesley"]
[Result "0-1"]
[WhiteElo "2715"]
[BlackElo "2762"]
[ECO "C88"]

1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Ba4 Nf6 5.0-0 Be7 6.Re1 b5 7.Bb3 0-0 8.h3 Bb7 9.d3 d5 10.exd5 Nxd5 11.Nbd2 Qd7!?

"Levon Aronian's novelty, connecting the rooks and planning to bring the queen rook into play ... it looks like Aronian's analytical team didn't let the computer work long enough to find the little combination" (Kavalek).

This refers to Anand-Aronian, WC Candidates 2014, which continued 12.Nxe5 Nxe5 13.Rxe5 Nf6? 14.Re1 Rae8 15.Nf3 Bd6 16.Be3 Re7 17.d4 Rfe8 18.c3 h6 19.Ne5 Bxe5 20.dxe5 Rxe5 21.Qxd7 Nxd7 22.Red1 Nf6 23.c4 c6 24.Rac1 R5e7 25.a4 bxc4 26.Bxc4 Nd5 27.Bc5 Re4 28.f3 R4e5 29.Kf2 Bc8 30.Bf1 R5e6 31.Rd3 Nf4 32.Rb3 Rd8 33.Be3 Nd5 34.Bd2 Nf6 35.Ba5 Rde8 36.Rb6 Re5 37.Bc3 Nd5 38.Bxe5 Nxb6 39.Bd4 Nxa4 40.Rxc6 Rd8 41.Rc4 Bd7 42.b3 Bb5 43.Rb4 Nb2 44.Bxb5 axb5 45.Ke3 Re8+ 46.Kd2 Rd8 47.Kc3 1-0 

Note: Komodo 8 and Houdini 4 engines don't find 11... Qd7 at 30 ply. Stockfish 5 is all over it at 38 ply, finding not only 11... Qd7 but also the sacrifical sequence 13... Nf4 and 14... Nxg2:

[Komodo 8 64-bit: Depth: 30 00:19:03  5355MN] 11... Nf4 12.Ne4 Na5 13.Bxf4 exf4 14.d4 Nxb3 15.axb3 f5 16.Nc5 Bxc5 17.dxc5 Qf6 18.Qe2 Qg6 19.Qe6+ Qxe6 20.Rxe6 Bxf3 21.gxf3 a5 22.Re5 a4 23.c6 axb3 24.Rxa8 Rxa8 25.cxb3 Ra6 26.Rxb5 Rxc6 27.Rxf5 Rg6+ 28.Kf1 Rb6 29.Rxf4 Rxb3 30.Rc4 Rxf3 31.Kg2 Rf7 32.Kg3 g6 33.Rc6 Kg7 34.f4 Rd7 35.Kg4 h5+ 36.Kg5 Rd5+ = (0.10)

[Houdini 4 x64B: Depth: 31/74 00:26:40  12588MN] 11... Nf4 12.Ne4 Na5 13.Bxf4 Nxb3 14.axb3 exf4 15.d4 f5 16.Ned2 Re8 17.c3 Qd5 18.Re5 Qf7 19.Qc2 Bd6 20.Rxf5 Qg6 21.Nh4 Qh6 22.Nhf3 Qg6 23.Nh4 = (0.00)  

[Stockfish 5 x64 Depth 38/56 00:42:40  12693MN]    11... Qd7 12.Nxe5 Nxe5 13.Rxe5 Nf4 14.Nf3 Nxg2 15.Kxg2 a5 16.c3 Ra6 17.d4 Rg6+ 18.Kh2 Bd6 19.Nh4 a4 20.Nxg6 hxg6 21.Qg4 Qxg4 22.hxg4 axb3 23.f4 Bxe5 24.fxe5 Ra8 25.a3 f6 26.Bf4 g5 27.Be3 Kf7 28.Kg3 Kg6 29.Re1 Re8 30.Bd2 Bd5 31.Rf1 = (0.05)

In Advanced Chess - Some Hints to Get Started  readers were admonished that "Advanced chess isn't speed chess. Never get in a hurry ... let the chess engine do its job!"  This advice also applies to home analysis; it  turns out Grandmasters and "teams of analysts" are not immune from basic computer analysis mistakes. 13... Nf6? demonstrates GM Aronian had not discerned the deep sacrificial possibilities (... Nf4 and ...Nxg2). If he had that analysis hidden away, was not a game against the previous world champion, in the candidates' tournament for the world's championship, the place to use it?
12.Nxe5 Nxe5 13.Rxe5

Position after 13. Rxe5

13 ... Nf4!

"This knight leap is more in the spirit of the Marshall gambit and leads to a piece sacrifice" (Kavalek).

[Stockfish 5 x64] 36:-0.37] 13...Nf4 14.Nf3 Nxg2 15.Kxg2 a5 16.c3 Ra6 17.d4 Rg6+ 18.Kh2 a4 19.Bc2 Bd6 20.Bf4 Rf6 21.Bxh7+ Kxh7 22.Qd3+ Kg8 -+.

14.Nf3 Nxg2!

"The Dutch grandmaster Anish Giri pointed out this sacrifice in the third issue of  New In Chess  last year. It was picked up by a few players, but the credit goes to computers.

"Komodo 8 suggests a different piece sacrifice: 14... Nxh3+ 15.gxh3 Bf6 16.Rh5 g6 with roughly equal chances" (Kavalek).

The rest of the Ivanchuk-So game: 15. Kxg2 a5 16. Rxe7 Qxe7 17. c3 Ra6 18. d4 Rf6 19. d5 a4 20. Bc2 Rd8 21. Qe1 Qd7 22. Ng5 h6 23. Ne4 Rg6+ 24. Kh2 f5 25. Ng3 Qxd5 26. Qg1 Qf3 0-1

Interested readers may also want to examine: Guliyev-Gustafsson, Baden-Baden 2014, 1/2-1/2 in 37 moves, and Jolly-Gozzoli, Erts 2014, 0-1 in 33 moves.