How I Managed to Finally Beat LeelaKnightOdds With Black

Tell me, O Muse, of the man of many devices, who wandered full many ways...
(Homer, Odyssey)

A few months ago, during a team event from my chess club in The Netherlands, someone drew my attention to a lichess bot called LeelaQueenOdds. It's an online chess bot, based on the neural network engine Leela Chess Zero, trained to play without a queen. Sounds easy to beat, right? We laughed a bit. How hard could that be? Some team members and I (most have a FIDE rating of 2200 or higher) decided to give it a go. Taking turns, we tried to beat it at bullet and blitz. We lost the first game. We lost the second game. We lost the third.

How was this possible? How could we not win with a full queen up? We are not weak players. Some of us have played competitive chess for over 30 years. We've ocassionally beaten grandmasters in serious games and scored modest successes on the international tournament scene. Surely we should be able to win with such a material advantage? But no. Somehow, we fell for tricks all the time, we got unexpectedly fooled by passed pawns in the endgame, or spotted a winning mating attack by Leela way too late. We felt like we were witnessing a David Copperfield performance. But better.

Beating Queen And Rook Odds

What made it even more infuriating is that Leela played very risky and opportunistic, as if it was trained to annoy us on purpose. It went for unorthodox, even ridiculous-looking openings, sometimes running quickly with its g-pawn, sometimes simply withdrawing pieces to the backrank in order to avoid exchanges (1.e4 Nc6 2.d4 f6!?! 3.d5 Nd8 or 1.e4 c5 2.Nf3 g5!?! are just two examples) and sometimes giving up even more material in order to develop quickly and obtain beautiful outposts for its pieces. Somehow, its pieces always seemed to be much better placed than ours.

Leela managed to win positions which, we were convinced, we'd even beat Magnus Carlsen with. Even in clearly lost endgames, being down lots of material, the bot proved extremely resourceful and resilient. It seemed to us we were playing against God.

After the first shock, we calmed ourselves and, using blitz time controls with increment (3+2) instead, some of us managed to get a few wins in. We realized that above all, it was important to remain calm, to keep a simple, open position, to avoid weakening our king, and to aim for piece exchanges. Easier said than done when you're playing an engine with an estimated rating of 3800. I went home and played 1-minute games against it for a long time, instead of sleeping, like normal people do. I tried again the next day. After several sessions playing until late at night, I managed to beat LeelaQueenOdds at 1-minute bullet after all.

I immediately stopped playing this terrible opponent and moved on to LeelaRookOdds (playing without a rook on a8 or a1).

This proved an even tougher nut to crack. Leela appeared to use dark sorcery to find ways to complicate seemingly simple positions, avoiding time and again what appeared to be certain defeat. It often made a completely unexpected king move in the middle of the board, as if Steinitz was still alive. I couldn't believe how difficult it was to get even playable positions. My extra rook was typically entirely useless (in fact both my rooks were often remarkably passive), as if I didn't know how to activate it, while Leela's single rook supported in a devastating attack.

Occasionally, it was possible to score a draw (especially because the bot sometimes allowed a threefold move repetition), but usually it was simply hopeless. I became increasingly desperate (and obsessed), though I noticed that many of my chess friends, all of whom are strong players, also failed incessantly. Watching them stumble was like watching a classic slapstick movie. It was less funny when it happened to me.

What to do? I started resigning positions that were objectively still winning, but were so difficult to play (e.g. closed positions, or endgames with a 'mere' pawn up) that it seemed like a waste of time to even try. But after almost 100 bitter attempts, to my own astonishment, I actually succeeded in defeating the many-eyed monster with black in a kind of reversed-Sicilian. For once, the tactics worked out for me. Here's the game:

BOT LeelaRookOdds vs CM ArnieChipmunk: Analysis board • lichess.org

I honestly thought it couldn't get any worse. Hadn't I just completed the ultimate test of human endurance in the face of silicon resistance? But of course, it wasn't over yet. There was still the challenge of playing and beating Leela (or should I say Hydra, the mythological creature with multiple heads?) with an extra knight...

A Different Kind Of Game

So I challenged LeelaKnightOdds. Bobby Fischer is reputed to have said that he could give knight odds to any female chess player in the world, and Paul Morphy scored very well with knight odds against rather strong opposition in the middle of the 19th century. But modern engines are even stronger than Fischer and Morphy, of course. How would I do when my opponent didn't have a knight on g8 (or g1)?

I soon realized playing against knight odds was a different kind of game. Leela played more or less 'normal' chess now, not even bothering to avoid mainline theory. It simply didn't care that it was down a piece and with Black it went for mainstream Sicilians, the French Defence and Pirc/Modern kind of setups. But somehow, my theoretical knowledge of these openings didn't help me much. I couldn't get my game going, my plans were all thwarted while Leela relentlessly profited from every inaccuracy. But relatively early on, I noticed something important: when it went for a fianchetto setup on the kingside, Leela often castled quickly (without a knight on g8 that goes one move faster), but it then allowed a quick h2-h4-h5, which offered chances for a rapid, if somewhat desperate attack.

I achieved my first draw using this device, and subsequently, I almost exclusively tried to get positions that allowed this kind of setup with White. I now shamelessly resigned sometimes as early as move 2 or 3 just to get the type of game I wanted. I felt it was possible... and after a whopping 150 attempts, it was there at last: a victory as White - with the 'slow attack' on the kingside I was so desperately looking for. I'm especially proud of the position after Black's 25th move: allowing a pawn promotion with check doesn't happen every day, and if it happens against an engine, one feels briefly in heaven.

CM ArnieChipmunk vs BOT LeelaKnightOdds: Analysis board • lichess.org

By now, I felt like the only sane option was to quit this game for good. It wasn't healthy. I lacked sleep. On lichess, I didn't have any appetite for 'normal' chess anymore, and the games I did play, I played badly. I was distracted, my mind was with the Beast.

And so I had to pick up the ultimate challenge: beating LeelaKnightOdds with Black. This is not an easy task even for super grandmasters, as none less than Hikaru Nakamura recently experienced in a match with black against the bot. After winning the first game, he went on to lose the session 14-2. In the video, despite being increasingly annoyed, Nakamura makes some very insightful remarks, such as that he wants to avoid closed positions (but can't) and that it's incredibly difficult to exchange pieces. Moreover, winning won endgames is nearly impossible even for a top 3 player in the world against perfect and endless opposition.

Playing with black was even more problematic for me than with white, as it was almost impossible to get any kind of attacking or even normal playing position. There were endless amounts of Trompowsky or London setups that I came to detest intensely. My beloved Rubinstein French proved toothless with an extra knight. Even the sharp Winawer with a piece up was a bridge too far. My Scandinavians crumbled, my Sicilians came crashing down. I played hundreds of games, drawing only a handful. I now aborted or resigned most games on the first couple of moves. I felt like Odysseus, trapped on Calypso's island of Ogygia, unable to escape.

(Calypso and Odysseus, by Sir. William Russell FIint)

On The Edge Of Madness

All this happened last week. It couldn't last much longer. It had to stop. But still I went on. Resigning the bulk of all my games without even trying, I played only three types of position seriously:

1. A relatively open position from the Tarrasch Defence in which Leela usually wins my isolated pawn on d5, but I at least I get all my pieced out and can get some exchanges. I came close a few times, but I never managed to beat it. Here's a typical game, exemplary of Leela's unbelievable tenacity:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

2. A line in the semi-Tarrasch (1.d4 Nf6 2.c4 e6 3.Nc3 d5 4.cxd5 Nxd5 5.e4 Nxc3 6.bxc3 c5) which Leela repeated several times. Gradually, by trial and error and looking at the positions a bit with Stockfish, I learned how to obtain a favourable endgame here - which then proved impossible to win as Leela managed to set up some kind of magical fortress. See for youself:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

I improved this line even further, reaching another kind of 'winning' endgame, but again I couldn't convert:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

The downside of playing this line was that Leela only rarely went for it, preferring setups with 4.Bg5 or an early g2-g3. This seemed increasingly like a waste of time as well. That left only one other option:

3. The Center Game (1.e4 e5 2.d4 exd4 3.Qxd4). Ironically, I wrote a book about this opening last year (The Center Game, published by New in Chess). Now I had to face this opening with an extra piece. I had already noticed that Leela played this variation a lot with knight odds (Matthew Sadler has also remarked on this fact in his review of my book in New in Chess Magazine) and it seemed to me that it played the opening too optimistically. I found a tricky line in which Leela played some weird moves and got a bad position. But time and again, I managed to mess up towards the end:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

This was already a much-improved version of earlier attempts, but here too, I couldn't deliver the final blow. I had many near-victories in this line, but I ended up either drawing or even losing many of them out of sheer frustration. I came to hate Leela. I came to hate chess. I came to hate myself. The madness had to stop.

The Endgame

There was, however, one last glimmer of hope: a funny little ending, also from a line in the Center Game, in which I would return my extra piece for just a pawn. But in this case, converting the ensuing position to a win seemed a matter of technique.

Wouldn't you say this endgame must be winning for Black? Well, good luck trying!

My first handful of attempts with this position were unsuccessful, not to say painful. I lost all of them, sometimes in amazing ways. Leela's king in particular seemed to come from a different planet than my own. I gradually came to understand that the black rook needs to go to the 6th rank as soon as possible, so that it can be exchanged (or attack h2). But this knowledge wasn't enough in itself. I also needed to ensure my pawns on the queenside must be placed on dark squares as soon as possible, or disasters like the following might occur (though the final position is actually not lost for Black):

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

My next insight was that there wasn't a single masterplan I could rely on. I needed to activate my king either via the kingside or via the centre, depending on how White would set up his pieces.

Fortunately for me, Leela kept playing the same plan as White, which wasn't even the optimal defence (at least according to Stockfish). This is actually a feature, not a bug. Leela's algoritm is trained to play against human inperfections, which is especially useful in queen and rook odds, and explains the weird opening choices. But it does mean that it plays opportunistically even in simple endgames, assuming people don't play optimal moves.

Then it dawned on me. A simple, yet profound truth.

Though I felt caught between the mythical sea monsters Scylla and Charybdis, I realized that I could just keep on trying. Leela would play the same moves and I would improve, step by step, slowly but steadily. I just needed to get the position I wanted in every game that I played. Even if I improved only one move per game, drawing or losing them all, in the end it simply wouldn't matter. I would succeed, because I knew the position was winning. All I needed was a little patience. I could beat it by exploiting its only weakness. With trial and error, perfecting the endgame tirelessly from my side, I could finally triumph over the monster.

Though, of course, there's dreaming and there's being awake. I sometimes got seemingly winning positions but then failed to convert due to Leela's superior tactical insights, such as here:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

Here at two more failed attempts to break White's fortress:

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

But I tried and tried again. I now felt I was very close, but I needed to be even more precise. In the following game, Leela again shows what an amazing escape artist it is. I resigned in a drawn position because I couldn't be bothered to play on for 50 more moves. I had long stopped caring for individual game results.

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

Homecoming

It was over 2am by now, and I badly needed sleep. All I needed was a single victory. One win to rule them all. But I needed to get The Endgame again. And that meant patience: half-heartedly playing more games, waiting until Leela played the Center Game again. Most of my next games I resigned instantly. Surprisingly, I got the Semi-Tarrasch twice in this series, but my heart wasn't into it and I even made a silly mouse-slip. But I knew that all I had to do was wait stoically for the next turn.

And there, around three in the morning, it suddenly was. This was it. The moment I'd been waiting for. I activated my king, as Leela had done so many times against me, not afraid to give up my bishop in order to promote my pawn. And this time, Leela was powerless to stop it.

BOT LeelaKnightOdds vs CM ArnieChipmunk: Analysis board • lichess.org

Checkmate. I couldn't believe it. I'd done it.

As my hands were still shaking, I sent the game link to some friends (who were all asleep, of course) and played over the moves one more time. I was completely exhausted. After what felt like 20 years, my journey had come to an end. After playing almost a thousand games against LeelaKnightOdds, I'd barely scored 1%. But none of that mattered. I had achieved the impossible. I had beaten an engine with only an extra knight.

And now I was finally home.

*
(The Return of Odysseus, by Claude Lorrain)