praxtube / chess-ai Goto Github PK

This would require an insane amount of work to implement, without even being sure that the performance payoff will be worth it. I wouldn't work on this for the next few milestones, probably not at all in the given time period of the project.

The evaluation functions takes up a significant amount of time in the whole search process (second only to probably the legal move generation). The current implementation takes about 0.050 ms, which I believe could be cut down to 0.025 ms if we would use C to evaluate the board position.

It appears as though numpy also has a C API which could be used to use numpy in C. This would also make it a lot easier and faster to call the transposition table (which is also implemented in C) and given that these two mainly need to communicate with each other, this might be a significant performance boost.

The unit tests should test the following situations:

• Castling (all possible sides) and also undoing the castle move
• Promotion and undoing promoting

The current engine calculates the following amount of moves:

Depth 3: 9322

Depth 4: 206603

However the actual numbers are

Depth 3: 8902

Depth 4: 197281

See Shannon number

So the current engine somehow calculates too many moves.
My suspicion is that moves like en passant aren't woring correctly
or that moves regarding check are broken.

This might also be because of the 3 move repetition draw rule, which is not checked in the chess engine.

Compare different boards that have different evaluations (one higher then the other). For instance, taking a piece is usually better then just moving to an empty square.

We should automatically generate a .csv file when running benchmarks to format the results more easily into tables. In order to create a .csv structure that makes sense, think about how we would like the tables to be generated, see #11.

I think it would be pretty fun to see the AI play against itself at different depths. This should show that our AI becomes smarter with each depth.

Pawns get always promoted to queens no matter what. We should change to check all possible promotions.

Currently, we simply set checkmate=False (as well as stalemate and in_check). It would be much better if we actually calculate if that is the case. Though this has absolutely no influence on the actual engine in the end. It would however improve the quality of tests.

It's not very rare to see an endless loop in the late game, where both sides repeat the same move. Implementing the 3 move repetition rule should stop this.

The caste rights are currently stored in a class. This is extremely unnecessary boilerplate and awkward to use. Instead, we could use a numpy.array of shape (,4) that stores booleans. The changes that would need to be made are also only inside of the chess_engine.py file.

It would be very useful to create the .tex (latex) file for the benchmark table automatically. We would want the following output

\begin{table}[ht!]
  \caption{}
	\centering
	\begin{tabular}{cccc}
		\toprule
		Tested Category & Early-Game & Mid-Game & Late-Game \\ \toprule
    Fen Conversion & 12 $\upmu$s / |  & 14 $\upmu$s / | & 9 $\upmu$s / | \\ %\midrule
		Legal Move Gen & 145 $\upmu$s / 3 $\upmu$s & 900 $\upmu$s / 3 $\upmu$s & 175 $\upmu$s / 3 $\upmu$s \\
		Making Move & 5 $\upmu$s / 11 $\upmu$s & 5 $\upmu$s / 11 $\upmu$s & 5 $\upmu$s / 11 $\upmu$s \\
		Evaluate Board & 21 $\upmu$s / 88 $\upmu$s & 24 $\upmu$s / 87 $\upmu$s & 20 $\upmu$s / 59 $\upmu$s \\
		Best Move Search & 1.2 ms / 9.3 ms & 2.4 ms / 34.3 ms & 0.6 ms / 2.0 ms \\ \bottomrule
	\end{tabular}
\end{table}

The .csv file that get's created from the benchmarks should be used to create the above table. See #7 for move info on the .csv file.

The move strings are ambiguous, for instance:

5N2/6P1/5K1k/8/5N2/8/8/8 w - - 0 1

will generate the exact same move twice (the two knight moves). It should however take into account that it must label them differently (in this case with the number 8 and 4).

It would be very useful to have a line plot with the depth (1 to 5 or something like that) on the x axis and the time it takes to find the best move in second on the y axis. Then we can have the different versions of the AI in the same plot to compare them much easier.

Castling rights seem to be ignored and always -.

If a stalemate is encountered the AI just crashes. One way to handle this, is to check if a stalemate can be reached and if that's the case and the evaluation is in our favor (i.e. we are winning), then evaluate it negatively, else evaluate it positively.

The way we are using the dict debug_info inside of log.py is extremely bad. It would be much better to use a class here. Not quite sure where the class should live though.

One way to train the hyper parameters like, piece values, piece tables, possibly how much weight should a is in check have, etc., is to train these through Reinforcement Learning where they play against an AI that can think in much higher depths (i.e. one that is written in Rust or C). This way the AI could potentially learn tricks that could help when playing against AI's that should completely destroy it.

Though I don't think that this is work too well, I do think that it could at least help a bit. It also seems like a fun little experiment and it also gives me an excuse to write some more Rust (or potentially C).

One big issue here is the question of how to reward the AI. Obviously it will fail most of the time, if not all. So a reward purely on winning and loosing is pretty meaningless. Instead, I think it would be smart to reward the AI for "surviving" longer, i.e. playing more moves, and punish it when it loses very quickly. It should also be rewarded when it looses with a smaller difference in evaluation compared to a huge difference.

I don't think this will be too great of an improvement, but I think it could be very fun and teaching.

It would be very useful to have plots that compare the AI of different version (i.e. Dummy AI, Basic AI, Advanced AI, etc.). My idea would be to use a bar chart to show the different categories (evaluation, best move gen, legal move gen, etc.) on the horizontal axis and the time used on the vertical axis. The categories would have all the bars from the different versions.

It would also be great to have line diagrams that show the time of certain categories across all versions, but that can be done later.

The following features should be included:

• Keep track of the available time
• Determine how much time to use (for each move)

The main loop should also be tested. This should make sure that the whole functionality of the code base works.

The following cases should be tested for as well:

• checkmate
• stalemate
• promotion
• castling
• en passant
• capturing
• checking
• pins

It seems like the AI generates illegal moves when it's thinking in higher depths.
For example:

FEN: rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1
Depth: 5
Moves:
g3 e6 Ng5 hxg5 a3

The third move is already illegal. It's not only illegal, it's just straigt up nonsense.
This happens as early as depth = 4, possibly even earlier.

My supsicion is that the search algorithm is somehow breaking either the board of the
GameState, or the engine is not working correctly.

Although the evaluation function #16 could potentially cover this up,
this must be resolved.

Potential things to try:

• Look at the previous search with minimax and see if it was broken there already
• Debug getValidMoves
• Check if the string representation of Move is working
• Check that the debug info is working as expected