Code Monkey home page Code Monkey logo

setback's Introduction

Counterfactual Regret Minimization for Setback

This project uses Counterfactual Regret Minimization to solve the game of Setback (aka Auction Pitch).

Approach

Setback's move tree is not as large as that of some other card games (e.g. Bridge, Hearts), but it is still too large for vanilla CFR on commodity hardware. The approach used here is to create a simpler abstract game that is essentially identical to Setback, but whose move tree is small enough for vanilla CFR. Defining such an abstraction is much more of an art than a science, relying heavily on personal intuition for what information is most salient when playing Setback.

Information set keys

Keys are only created for infosets with more than one legal action. Each key is a string with the layouts described below.

Auction key

Baseline

These keys are used when training the baseline (score-insensitive) model.

Position Name Possible Values Description
0 High bid 0 (pass)
2 (two bid)
3 (three bid)
D (dealer-overridable four bid)
High bid so far.
1-6 Ranks #1 6 rank characters, or . placeholders. E.g. Jxx2.. Ranks present in hand's strongest suit
7-12 Ranks #2 Same as above Ranks present in hand's second-strongest suit, if any, or . placeholders

Suit strength is determined as follows:

  • Convert the rank of each card in the suit to an integer by subtracting one from its face value. (E.g. Two → 1, Three → 2, ..., Ten → 9, Jack → 10, ..., Ace → 13).
  • Sum the values.

The second-strongest suit is included iff the difference between its strength and the strength of the strongest suit is less than 2.

Bootstrap

These keys are used when bootstrapping a score-sensitive model from a baseline model. Unlike the baseline keys, these keys are formatted using / characters, rather than by absolute position within the string, as follows:

Score-need/High-bid/High-bidder/Num-bids/Hand

Where:

  • Score-need: Representation of the current score, consisting of two characters: Them-need and Us-need.
    • Them-need: The number of points the other team needs to win, or x if the other team needs more than four points (i.e. can't win on this deal).
    • Us-need: If the other team is ahead and needs four points or less then !, else ..
  • High-bid: The current high bid as an integer: 0, 2, 3, or 4.
  • High-bidder: Index of the current high bidder, relative to the dealer: 0, 1, 2, 3, or -1 if there is no current high bidder.
  • Num-bids: The number of bids made so far in this auction (0 - 3).
  • Hand: The bidder's hand, as expressed in positions 1-12 of the baseline auction key.

Playout key

Establish trump

These keys are used by the auction winner to establish trump with the first card played.

Position Name Possible Values Description
0 Header E Establish trump.
1-13 Auction key Corresponding auction key.

Normal playout

A normal playout key is a string of up to 24 characters, laid out as follows:

Position Name Possible Values Description
0 High established H (true)
. (false)
Indicates whether the auction winner led a high card on the first trick.
1 Low taken 2
3
4
5
x (rank > 5, or current hand holds a lower trump)
Rank of the lowest trump card taken so far.
2 Jack taken J (true)
. (false)
Indicates whether the Jack of trump has been taken.
3 Game delta + (current team is ahead or tied)
- (current team is behind)
Indicates how many Game points the current team has taken, relative to the other team.
4 Trump voids 0-F A hex digit that indicates which of the other players are known to be void in trump (because they failed follow suit on a trump lead).
5 Rank #1 2-5 (possible low trump)
T-A (has Game value)
x (other ranks)
. (not played yet)
Rank of first card played on current trick.
6 Suit #1 t (trump)
x (non-trump)
. (not played yet)
Suit of first card played on current trick.
7 Rank #2 Same as above. Rank of second card played on current trick.
8 Suit #2 Same as above. Suit of second card played on current trick.
9 Rank #3 Same as above. Rank of third card played on current trick.
10 Suit #3 Same as above. Suit of third card played on current trick.
11 Trick winner 0-3
. (none yet)
0-based index of card play that is winning the current trick.
12-13 Legal action #1 Tr (play trump of rank r)
S. (lead a strong non-trump)
W. (lead a weak non-trump)
Wx (play a winning trump)
Lx(play a losing trump)
wg (follow suit with card worth g Game points to win the trick)
lg (follow suit with card worth g Game points to lose the trick)
T. (contribute a Ten)
G. (contribute a non-Ten card worth Game points)
D. (duck - play a weak card)
Legal action available in this situation.
14-15 Legal action #2 Same as above, or .. placeholder.
16-17 Legal action #3 Same as above, or .. placeholder..
18-19 Legal action #4 Same as above, or .. placeholder..
20-21 Legal action #5 Same as above, or .. placeholder..
22-23 Legal action #6 Same as above, or .. placeholder..

Trailing dots (.) are trimmed from the baseline keys to save space. (There are a lot of them, especially the playout keys.)

Usage

  1. Run the TrainBaseline project to generate a baseline strategy profile (Baseline.strategy) that is optimized to play a hand of Setback without regard to the score of the game. Warning: This requires a machine with at least 16GB of RAM, and will take several weeks/months. I recommend at least 10 million CFR iterations.
  2. Copy Baseline.strategy into the TrainBootstrap project and then run the project to bootstrap a strategy profile (Bootstrap.strategy) that is optimized for score-aware bidding (e.g. by bidding more aggressively if the opposing team is close to winning the game). This requires less time and RAM. I recommend at least 20 million CFR iterations.
  3. Copy Baseline.strategy and Bootstrap.strategy into the LoadDatabase project, and then run the project to create a SQLite database that can be used to play Setback (via the DatabasePlayer module).
  4. Copy the SQLite database Setback.db into the Setback.Cfrm.PlayGui project, and then run the project to play against the trained model (Windows only).

setback's People

Contributors

brianberns avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

setback's Issues

Model trumps in with loser

West: 765♥ 9♦ QJ♣
North: 6♠ 82♥ T98♣
East: A♠ AT♥ A82♦
South: QJ4♠ QT♦ K♣

West: Three
North: Pass
East: Pass
South: Pass

W:Q♣ N:9♣ E:T♥ S:K♣ 
S:Q♠ W:5♥ N:6♠ E:A♠ 
E:A♦ S:Q♦ W:J♣ N:8♣

North's play on trick 3 is surprising.

Model bids four with bad hand

Score is N+S: 8, E+W: 7
North bids four on 654♥ 8♦ T3♣

Deal #562328435
Deal.fs.js:140 Dealer is South
Auction.fs.js:68 West bids Pass
Auction.fs.js:68 North bids Four
Auction.fs.js:68 East bids Pass
Auction.fs.js:68 South bids Pass
Playout.fs.js:115 North plays 6♥
Playout.fs.js:115 East plays 2♠
Playout.fs.js:115 South plays 9♥
Playout.fs.js:115 West plays 2♥
Playout.fs.js:115 South plays 9♦
Playout.fs.js:115 West plays J♦
Playout.fs.js:115 North plays 8♦
Playout.fs.js:115 East plays 2♦
Playout.fs.js:115 West plays A♥
Playout.fs.js:115 North plays 4♥
Playout.fs.js:115 East plays Q♣
Playout.fs.js:115 South plays 4♦
Playout.fs.js:115 West plays Q♠
Playout.fs.js:115 North plays 3♣
Playout.fs.js:115 East plays 5♠
Playout.fs.js:115 South plays 3♠
Playout.fs.js:115 West plays 5♣
Playout.fs.js:115 North plays T♣
Playout.fs.js:115 East plays 2♣
Playout.fs.js:115 South plays 4♣
Playout.fs.js:115 North plays 5♥
Playout.fs.js:115 East plays J♠
Playout.fs.js:115 South plays T♠
Playout.fs.js:115 West plays 9♠
Game.fs.js:83 E+W make 1 point(s)
Game.fs.js:84 N+S make -4 point(s)
Game.fs.js:92 E+W have 8 point(s)
Game.fs.js:93 N+S have 4 point(s)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.