Code Monkey home page Code Monkey logo

postflop-solver's Introduction

postflop-solver

Important

As of October 2023, I have started developing a poker solver as a business and have decided to suspend development of this open-source project. See this issue for more information.


An open-source postflop solver library written in Rust

Documentation: https://b-inary.github.io/postflop_solver/postflop_solver/

Related repositories

Note: The primary purpose of this library is to serve as a backend engine for the GUI applications (WASM Postflop and Desktop Postflop). The direct use of this library by the users/developers is not a critical purpose by design. Therefore, breaking changes are often made without version changes. See CHANGES.md for details about breaking changes.

Usage

  • Cargo.toml
[dependencies]
postflop-solver = { git = "https://github.com/b-inary/postflop-solver" }
  • Examples

You can find examples in the examples directory.

If you have cloned this repository, you can run the example with the following command:

$ cargo run --release --example basic

Implementation details

  • Algorithm: The solver uses the state-of-the-art Discounted CFR algorithm. Currently, the value of γ is set to 3.0 instead of the 2.0 recommended in the original paper. Also, the solver resets the cumulative strategy when the number of iterations is a power of 4.
  • Performance: The solver engine is highly optimized for performance with maintainable code. The engine supports multithreading by default, and it takes full advantage of unsafe Rust in hot spots. The developer reviews the assembly output from the compiler and ensures that SIMD instructions are used as much as possible. Combined with the algorithm described above, the performance surpasses paid solvers such as PioSOLVER and GTO+.
  • Isomorphism: The solver does not perform any abstraction. However, isomorphic chances (turn and river deals) are combined into one. For example, if the flop is monotone, the three non-dealt suits are isomorphic, allowing us to skip the calculation for two of the three suits.
  • Precision: 32-bit floating-point numbers are used in most places. When calculating summations, temporary values use 64-bit floating-point numbers. There is also a compression option where each game node stores the values by 16-bit integers with a single 32-bit floating-point scaling factor.
  • Bunching effect: At the time of writing, this is the only implementation that can handle the bunching effect. It supports up to four folded players (6-max game). The implementation correctly counts the number of card combinations and does not rely on heuristics such as manipulating the probability distribution of the deck. Note, however, that enabling the bunching effect increases the time complexity of the evaluation at the terminal nodes and slows down the computation significantly.

Crate features

  • bincode: Uses bincode crate (2.0.0-rc.3) to serialize and deserialize the PostFlopGame struct. This feature is required to save and load the game tree. Enabled by default.
  • custom-alloc: Uses custom memory allocator in solving process (only available in nightly Rust). It significantly reduces the number of calls of the default allocator, so it is recommended to use this feature when the default allocator is not so efficient. Note that this feature assumes that, at most, only one instance of PostFlopGame is available when solving in a program. Disabled by default.
  • rayon: Uses rayon crate for parallelization. Enabled by default.
  • zstd: Uses zstd crate to compress and decompress the game tree. This feature is required to save and load the game tree with compression. Disabled by default.

License

Copyright (C) 2022 Wataru Inariba

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

postflop-solver's People

Contributors

b-inary avatar bkushigian avatar dpmaloney avatar jbcourtois avatar matthewkennedy5 avatar mhluska avatar ydm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postflop-solver's Issues

Multi thread example

Hi,

Thanks for the great work!

It seems that the example run much slower than in the desktop app.

I notice that multi-thread is implemented in the desktop application but not in this code.

Can you give an example of how to do that?

Announcement: Suspension of development of open-source poker solver project

As mentioned in the README, from October 2023, I have started developing a poker solver as a business and have decided to suspend development of this open-source project. Therefore, the following repositories will no longer be updated and I will no longer respond to issues and pull requests.

This issue is for developers and users to discuss these repositories. I am very sorry that I will no longer be able to provide support, but I sincerely hope that such support can be replaced on a community basis.

I am proud to be the first open-source developer of a decent and reliable poker solver. However, at the same time, this project heavily relied on my personal motivation and was not sustainable. One of the reasons for this decision was that I thought it might be better to announce the suspension of development rather than to leave the project without any updates.

I do not know how this decision will turn out. However, this simply means that I will no longer update these repositories, and therefore they will remain available to the public, and of course it is perfectly fine to fork, redistribute, modify, etc., under the AGPL license. Also, if a worthy successor comes along, I may update the README of these repositories and provide links to them as an exception.

Lastly, I would like to thank everyone who has supported, contributed to, or used this project. If there were not so many people interested in this project, I would have quit development earlier and in a more ambiguous way. I am sad that I will not be able to be involved, but I am very excited to see what comes next for the poker development community through the spirit of open source.

Add game tree visiting logic?

There are various reasons why we might want to traverse a game tree after a solve. My present usecase (mentioned in #14) is to build a strategy simplifier by pruning low-utility branches. Another use case would bet to serialize the game tree efficiently (rather than dumping to memory, we could serialize relevant pre-river state), and if mutability was allowed we could even modify trees during a visit.

Would this be something you'd want to support? I'd maybe be willing to take a stab at the implementation though I can't guarantee quality.

Edit: I over-edited the above to the point of being nonsense: I want some isolated logic to make traversing a game tree dead simple, something that does all the traversal logic and lets me hook into the right spots with lamdas or whatever.

Easiest way to compute intermediate EVs and MESs during solve

First, thank you so much for this library! It's letting me run an empirical eval that I've been wanting to do for a while, and the code is so nice it's an absolute pleasure to work with!

One thing that would help would be to inspect intermediate EV and MES values for each player during a solve (e.g., every 10 iterations). Best way I can see to do this is to clone the tree and run finalize(game), but this is obviously not great (memory usage, inefficient, etc). Is there any way around this?

Exploitability Calculation - ICM

Hi,

first of all very nice and clean project, the GUI in the other project is very impressive.

I have been working on trying to implement custom ICM payouts, which seems easy enough if I just change the evaluate function to call my custom ICM function and then have amount_lose and amount_win be ICM values. In a first test, the convergence was starting and the results look promising.

Now the problem that I have is the exploitability calculation, as the solver will stop after much few iterations than with chipev and I can't seem to figure out how the computation works. It says:

/// Computes the expected values of the MES (Maximally Exploitative Strategy) of each player.
///
/// The bias, i.e., (starting pot) / 2, is already subtracted to increase the significant figures.
/// Therefore, the average of the return value corresponds to the exploitability value if not raked.
#[inline]
pub fn compute_mes_ev<T: Game>(game: &T) -> [f32; 2] {

But I can't find where this subtraction is happening. Admittedly I have never coded in Rust or C++ before so I apologize if I am missing something very obvious. How would I go to set the initial value for the exploitability?

Release Ram after solving

Hi,

one very cool change that would allow some experimenting with having multiple different solves open and compare them, would be to release RAM after solving.

Example error

Hi,

When trying to run the example it gives the error

error: expected item, found keyword `let`
 --> example.rs:4:1
  |
4 | let oop_range = "22+,A2s+,A8o+,K7s+,K9o+,Q8s+,Q9o+,J8s+,J9o+,T8+,97+,86+,75+,64s+,65o,54,43s";
  | ^^^ expected item

error: aborting due to previous error.

Or if I try to run cargo run it gives the following

postflop-solver % cargo run              
error: a bin target must be available for `cargo run`

Is there a main.rs available for this project, or some instructions to run the example file?

Thanks!

GPU implementation

Hi,
Have you considered to implement GPU implementation for the solver?
Thanks,
Pavel

Slow time to load saved solution

It looks like it takes around 7.5 seconds to load one of the solutions. And it doesn't seem to parallelize at all. If I load 4 solutions, it takes ~34 seconds.

I'm doing this to save a file:

  let config = config::standard();
  let mut file = BufWriter::new(File::create("test.bin").unwrap());
  bincode::encode_into_std_write(&game, &mut file, config).unwrap();

Which produces a 1.8 GB file after solving. Then I have another executable that does this:

  let mut file = File::open("test.bin").unwrap();
  let config = config::standard();
  let mut game: PostFlopGame = bincode::decode_from_std_read(&mut file, config).unwrap();

The bincode::decode_from_std_read step takes 7.5 seconds. Is that correct? I'm using a MacBook Pro M2 Max.

If I copy the test.bin file four times, and do something like this to concurrently load four files, it takes 4x as long:

./target/release/examples/load test1.bin & 
./target/release/examples/load test2.bin & 
./target/release/examples/load test3.bin & 
./target/release/examples/load test4.bin & 
wait

You'd think that with four processes in the background, it would load these concurrently. I'm not too familiar with Rust though.

PLO extension

Hi @b-inary big fan of your work and just wanted to say thank you for open sourcing it.

Is it possible to extend this library to solve PLO and PLO-5

Thanks

Raked calculations

Is it possible to prerform raked calculations (e.g. rake 5% with 1 BB cap)?

Example usage and solver tests are incorrect

Is the target_exploitability arg of the solve method supposed to be a percentage like 0.005 (representing 0.5% EV) or is it supposed to be a big blind amount? Like pot_size * 0.005?

Reading through the code, it seems like it's supposed to just be a raw percentage like 0.005 but basic.rs is suggesting using a big blind amount:

let target_exploitability = game.tree_config().starting_pot as f32 * 0.005; // 0.5% of the pot

This has huge implications on the runtime of the software and would make it seem much faster than it is for people trying it out via the basic example.

In the tests, sometimes a percentage is directly used and other times its a percentage of the pot.

Unclear how to save a file

It's unclear in basic.rs how to save a file, I tried this
let config = config::standard();
let mut file = BufWriter::new(File::create("test.bin").unwrap());
bincode::encode_into_std_write(&game, &mut file, config).unwrap();
It doesn't work, can you add some instructions in basic.rs, how to save a file

node lock

Any plan to support node lock? It looks straightforward from the current code. Specifically, can we call the solve function after we run some game.play(), and then it will start searching from the current state? This may need some modification in solve_step to determine the current player and starting node.

I wish I had a clue how to use this..

I'm pretty new when it comes to this stuff. Installing python stuff is easy but no idea how to even begin here. I've downloaded the Desktop Postflop application. Could you maybe explain to me in beginners terms how I would go about running this and using it along with Desktop Postflop or is it even supposed to be used in combination with that application? Sorry and thank you haha!

train players reach variable may not be used.

In solver, variable reach_actions are multiplied based on the strategy, then passed to the child nodes.
But, I think the reach argument is never used for computetion.

I tried to just pass the result to the child node without multiply, but result did not seem to change.

         // update the reach probabilities
-        #[cfg(feature = "custom-alloc")]
-        let mut reach_actions = {
-            let mut tmp = Vec::with_capacity_in(strategy.len(), StackAlloc);
-            tmp.extend_from_slice(&strategy);
-            tmp
-        };
-        #[cfg(not(feature = "custom-alloc"))]
-        let mut reach_actions = strategy.clone();
-        reach_actions.chunks_exact_mut(num_hands).for_each(|row| {
-            mul_slice(row, reach);
-        });
+        // #[cfg(feature = "custom-alloc")]
+        // let mut reach_actions = {
+        //     let mut tmp = Vec::with_capacity_in(strategy.len(), StackAlloc);
+        //     tmp.extend_from_slice(&strategy);
+        //     tmp
+        // };
+        // #[cfg(not(feature = "custom-alloc"))]
+        // let mut reach_actions = strategy.clone();
+        // reach_actions.chunks_exact_mut(num_hands).for_each(|row| {
+        //     mul_slice(row, reach);
+        // });
-                row(&reach_actions, action, num_hands),
+                // row(&reach_actions, action, num_hands),
+               reach,

Is this unnecessary code?

https://github.com/b-inary/postflop-solver/blob/main/src/solver.rs#L262

Best way to serialize PostFlopGame?

Hi!

This is a very cool library! I've been messing around with it for the last week and it's so cool to run a bunch of solves on my machine :)

I'd like to save a PostFlopGame so I can load it again later and use the functions that let me traverse the tree with different actions, turns, and rivers. I tried using serde to do so, but got stuck with reading/writing the private fields.

I'm new to rust so any advice you have would be helpful.

Thanks again for this awesome library!

Removing different actions from different turns/rivers

I'm building a strategy simplifier that removes unused sizes from different turns/rivers. For instance, in a single raised pot Bn vs BB on J93 after a small cbet, on a brick turn I'll probably want to overbet barrel, but this size won't be used much on a J or a 9. Therefore I'd like to prune my gametree differently for different turn cards.

It looks like ActionTree doesn't support this:

    /// Removes a given line from the action tree.
    ///
    /// - `line` must exist in the current tree.
    /// - Chance actions (i.e., dealing turn and river cards) must be omitted from the `line`.
    /// - If the current node is removed by this method, the current node is moved to the nearest
    ///   ancestor node that is not removed.

Is there any way to get this functionality? I also looked into modifying a PostFlopGame instance directly but couldn't find anything, and I'd imagine there might be issues that arise from treating isomorphic lines differently?

Thanks a bunch!

Decompression speed improvements

Currently when loading a compressed solution file, it takes very long because zstd doesn't support multi-threaded decompression. Only compression.

One way around this would be to instead serialize into multiple files instead of one large file. Particularly the Vec[u8] data from storage1 and node_arena which can then be deserialized in parallel into a single vector in memory.

It looks like pzstd uses a similar strategy.

I can submit a PR if the library author is interested. I have something working now which looks promising:

  • loading a 6GB solution compressed with zstd level 3: 10s
  • loading the same solution split into file chunks compressed with zstd level 3, decompressed in parallel: 3.9s

Add discussion board to repo?

I have a few things (e.g., "what are your plans/goals for this project") that doesn't really fit into an issue. Any interest in enabling discussions for this repo? It would facilitate more open-ended discussions that aren't necessarily feature-oriented.

Extremely slow serialization with bincode v2

I'm noticing throughput of several MB per second when saving to file with some solutions (others work fine, yet others are slow at the beginning and fast towards the end of serialization). I'm using high end EC2 machines so it's unlikely to be a hardware limitation. I also have compression disabled.

I'm pretty sure the issue is the bincode v2 dependency: bincode-org/bincode#618

When I add this to the Cargo.toml file, things serialize much faster:

[profile.release]
lto = true

Print tree actions

I'm trying to print all the possible action combinations of a tree, but I'm running into a problem. The first loop prints correctly: Action history: [0, 0, 18446744073709551615, 0, 0, 18446744073709551615, 0, 0]

The second it should be: Action history: [0, 0, 18446744073709551615, 0, 0, 18446 744073709551615, 0, 1 , 0]

however I get Action history: [1,0], which prints until it collapses. What am I doing wrong here? There are no tests where you experiment with something similar?

My code:

fn traverse_tree(game: &mut PostFlopGame) {
        if game.is_terminal_node() {
            println!("Action history: {:?}", game.history());
            game.back_to_root(); 
            return;
        }
    
        if game.is_chance_node() {
            game.play(usize::MAX); 
        }
    
        let actions: Vec<Action> = game.available_actions();
    
        for action in &actions {
            let action_index = actions.iter().position(|&a| a == *action).unwrap();
            game.play(action_index);
            traverse_tree(game);
        }
    }

traverse_tree(&mut game);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.