Implementations of Counterfactual Regret Minimization (CFR) for solving a variety of Holdem-like poker games

• Supports both CFR+ and MCCFR.
• Supports resolving of endgames
• Supports creation of card and betting abstractions
• Supports head-to-head evaluation and real-game best-response
• Supports multiplayer

You should have a version of gcc that supports C++ 17. gcc 7.3 is known to work.

Set up a "static" directory and a "cfr" directory with enough disk space available. Specify these paths in files.cpp.

Build whatever binaries you need from the top level. For example:

make bin/build_hand_value_tree

A simple example using a two street game solved with a small betting abstraction and no card abstraction.

cd runs
../bin/build_hand_value_tree ms1f3_params
../bin/build_betting_tree ms1f3_params mb1b1_params 
../bin/run_cfrp ms1f3_params none_params mb1b1_params cfrps_params 8 1 200
../bin/run_rgbr ms1f3_params none_params mb1b1_params cfrps_params 8 200 avg raw

Run external sampling on a four street game using the full 52-card deck but a crude card abstraction and a simple betting abstraction.

cd runs
../bin/build_hand_value_tree holdem_params
../bin/build_betting_tree holdem_params mb1b1_params 
../bin/build_null_buckets holdem_params 0
../bin/build_null_buckets holdem_params 1
../bin/build_rollout_features holdem_params 2 hs 1.0 wmls 0.5
../bin/build_rollout_features holdem_params 3 hs 1.0 wmls 0.5
../bin/build_unique_buckets holdem_params 2 hs hs
../bin/build_unique_buckets holdem_params 3 hs hs
../bin/run_tcfr holdem_params nhs2_params mb1b1_params ecfr_params 8 0 1 100000000 1

../bin/run_rgbr ms1f3_params none_params mb1b1_params cfrps_params 8 200 avg raw

First run MCCFR for another batch of 100 million iterations:

../bin/run_tcfr holdem_params nhs2_params mb1b1_params ecfr_params 8 1 2 100000000 1

You can compare the two checkpoints using head_to_head:

../bin/head_to_head holdem_params nhs2_params nhs2_params mb1b1_params mb1b1_params ecfr_params ecfr_params 0 1 10000 -1 false false

Alternatively you can use play:

../bin/play holdem_params nhs2_params nhs2_params mb1b1_params mb1b1_params ecfr_params ecfr_params 0 1 100000000

Solve all the subgames and produce a new system that merges the base system with the resolved subgames.

../bin/solve_all_subgames ms1f3_params none_params none_params none_params mb1b1_params mb1b1_params cfrps_params cfrps_params 1 200 200 unsafe cbrs card zerosum avg null mem 8
../bin/assemble_subgames ms1f3_params none_params none_params none_params mb1b1_params mb1b1_params cfrps_params cfrps_params cfrpsmu_params 1 200 200 unsafe

Compare a base system to a system with resolved subgames, sampling only some boards.

../bin/head_to_head holdem_params nhs2_params nhs2_params mb1b1_params mb1b1_params ecfr_params ecfr_params 0 1 10000 3 false true none_params mb1b1_params cfrps_params

TODO

Multiplayer (i.e., more than two players) does not work with all features of this package. For example, multiplayer is only supported by MCCFR and not by CFR+. And you cannot compute real-game best responses on multiplayer games.

Because the betting tree can explode with multiplayer games, it is often advisable to use a reentrant betting tree with multiplayer. See the "Betting Abstraction" section below.

TODO

TODO

CFR+ is the preferred algorithm when it is feasible, ideally with no card abstraction. It exhibits the best convergence to equilibrium. However, CFR+ makes a full iteration over the game tree on each iteration and may take too long for large games.

MCCFR is a sampling variant of CFR which makes it possible to come up with pretty good approximations of equilibrium for large games. We support External Sampling and Targeted CFR which are good for large games. Each iteration of MCCFR traverses a small fraction of the game tree.

CFR can be configured through several parameter files which control the game being solved, the card abstraction, the betting abstraction, and solving parameters.

For an example, see runs/holdem_params or runs/ms1f3_params.

Specifies the basic rules of the game including the number of streets (betting rounds), the size of the deck, the number of players and how many community cards are dealt on each street.

Notes:

• Although there is a parameter for the number of hole cards, the code assumes two hole cards in many places.
• The turn and river are assumed to involve one community card being dealt. Only the flop is configurable, and the flop may only involve one, two or three community cards.
• Games involving "draws" like 5-card draw are not supported.

Specifies the card abstraction. For each street we specify a bucketing. A bucketing is a way of reducing the size of the card space by mapping raw hands into buckets which are collections of very similar hands.

There are two reserved names for bucketing:

• "none": CFR+ can operate without a card abstraction. To specify no abstraction, use "none" for the bucketing.
• "null": the null abstraction only groups together groups of cards that are functionally equivalent (sometimes called "isomorphic"). For example, the preflop hands AsKs and AhKh can safely be grouped together in the same bucket. MCCFR requires a card abstraction so use the "null" abstraction if you want a "lossless" abstraction.

See the "Card Abstraction" section later for more information on card abstraction.

For an example, see runs/mb1b1_params.

Specify things like the stack size, and what bet sizes are allowed per street and how many bets.

For an example, see runs/cfrps_params or runs/ecfr_params.

Specify whether you are using CFR+ or MCCFR and set other parameters that control solving.

Use build_null_buckets to create the "null" bucketing".

You can also create your own card abstractions.

Use build_rollout_features to create a map associating hands with certain hand strength related features.

Use build_unique_buckets or build_kmeans buckets to create a bucketing from features.

You may also wish to look into prify and combine_features.

Once you have create a betting abstraction parameter file, build the betting tree with build_betting_tree:

../bin/build_betting tree <game params> <betting params>

You can also view your betting tree:

../bin/show_betting tree <game params> <betting params>

For an example, see runs/mb1b1r3_params.

TODO