Add testing infrastructure

Draw up a slide by slide plan for the presentation.

Why I am using EFR as opposed to CFR, why haven't more people done this, what are the advantages/disadvantages.

All are good questions, look more into the Morrhill paper.

Ensure that all points have sufficient referencing.

As suggested by Dr Morrill, should use a non fixed point method for external deviations.

If a game like Limit Texas hold 'em is to be used, extensive abstract techniques are to be used. Look for the current state of the art and justify its use.

Current informed deviations are is the action deviation module.

Find the module name for the Poker version and try running some experiments on it.

Informed CF is not performing as expected, I've started a 1000 iteration run to further investigate this. A new issue for a fix should be created if required.

OpenSpiel is currently considered, probably not the best, see if a new system is more suitable or an alternative is better.

Write and submit the progress report

See:

https://warwick.ac.uk/fac/sci/dcs/teaching/material/cs310/components/progress/

Assessed on:

The following criteria will be used by the assessors in marking your progress report.

Technical content:
Student is well read in the project's subject area.
Effective analysis of problems and issues.
Quality of design work.
Good choice of methods and tools.
Project management:
Well conceived project
Unforeseen problems well detected and overcome.
Progress consistent with the project specification.
All necessary research, analysis and design work completed.
Work for next term is well planned out.
Communication skills:
Basic written language skills such as spelling and grammar.
Effective composition and exposition.
Report is of an appropriate length for your particular project.

Refactor code and remove unused functions.

Develop/find a Python module that returns the random opponent for the test purposes.

What Poker ruleset? How many players? How many rounds? What kind of opponents?

ANSWER

What I am doing:

Limit Texas Hold 'em (stretch goal would be the full NL HE but this is almost certainly intractable on DCS machines and 2 days of compute time).

2 player, this is the minimum number of players that we can chose, increasing this number would again add alot of complexity to the project. I have resources limited to the DCS batch compute system.

Computer opponents, this will allow for efficient learning and circuments potential ethical issues and admin issues in administering such tests. See #17 for more details.

    Opponent types will be decided after my algorithm and details are nailed down and before implementation, 4 different types seems like a good range and types should be different enough to test the adaptability of agents.

Originally posted by @Jamesflynn1 in #16 (comment)

EFR produces a theoretical equilibria, how can we translate this into an exploiting strategy using opponent modelling

As described

Understand how current methods are limited and suggested improves that could be made to them

Currently lacking in direction for project, these GitHub issues are a good start but aren't comprehensive.

Watch and make notes on presentation lecture to understand marking criteria.

Add remaining deviation types

Training time will again depend on the details of the algorithm and will be decided after such time.

Originally posted by @Jamesflynn1 in #16 (comment)

Run EFR with for all deviation set functions.

Store avg strategy.

Compute opponent strategy.

Compute expected game value.

Use this thread to keep track of progress (and save it into a log maybe).

Create a beautiful AI presentation and account for the project and 20 slides

How max regret grows sublinearly (leading to an equilibrium)
How it works with all deviations subsets (unlike CFR)
Potential of equilibrium concepts to work better in some opponent cases.
Time selection regret minimisation
more??

I suspect many issues current arise from a broken fixed point solver, the fixed point should be constrained to be a strategy in and of itself.

Run an example program in Python and investigate suitability. Create a mini report with the findings

Asked Dr. Morrill for advice, external deviations can just use a normalisation approach. Internal requires a Jacobi SVD decomposition and then to solve that.

Ensure that I can run the experiments on the DCS machine.

Action transforms
Swap transforms (internal and external)
Deviations
How this leads to memory states
Hindsight rationality
Observable Sequential Rationality and why this is required over standard rationality
Partial Sequence Deviations (correlation, deviation, recorrelation)

All deviation sets definition (start with behavioural devations)

The length of the report is normally between 12,000 and 18,000 words (excluding appendices), and code (if applicable) will be submitted as a separate zip file on Tabula. Check the format page for further details of the requirements for the document's format.

Technical Content:

Use of reference and other information sources.
Problem solving methodologies and tools.
Effective problem analysis.
Innovative design.
Technical achievement.
Critical and fair discussion of the subject matter.
Project Management:

Well conceived and specified project.
Effort and motivation.
Organisation and professionalism.
Communication Skills:

All components present.
Chapters well designed.
Language and technical writing skills.
Composition.
Software and hardware documentation.
Readability.
Appropriate length.

Bug to do with either histories or deviation variables. Should be an easy fix, need to test on blind action deviations first.

Aim understand what I have researched and create a file containing known knowledge to reference and a material for an introduction for the progress report

Compile a list of presentation markscheme points and write as description of milestone.

As title suggests.
https://dl.acm.org/doi/pdf/10.1145/1329125.1329357
Will need to modify for Leduc Poker.

Should include an overview of the theoretic background for the project.

Equilibrium concepts

Best response

Incomplete information

Extensive and strategic forms of games (information sets, actions, players ect)

Add more

Notebook the results graph and store the experiment details, should add more deviation types.

Create a good readme, this could serve as a brief introduction for the progress report

Regret definition

Rationality and the problems with learning against an opponents strategy (exploitation)

What is no regret (no internal, no phi regret ect) learning

Regret matching

Fictitious self play

CFR and primary theorems

Talk about the abstraction that OpenSpiel uses for Leduc Poker.

Combines #20 #10 #2

Write an introduction to EFR, why EFR and the algorithm in Overleaf

Technical content:
Innovative analysis and design.
Methods and solutions employed.
Quality of examples used to demonstrate results.
Overall level of technical achievement.
Conclusions and suggestions for further work.
Project management:
Well conceived project.
Unforeseen problems well detected and overcome.
Progress consistent with project specification.
All necessary research, analysis and design work completed.
Communication skills:
Care and preparation of visual aids.
Quality of oral delivery.
Effective use of time and appropriate length.
Effective response to questions.

the motivation for your project (Why is it worth doing?)
the necessary background (What is already known? What has been done so far? What are the gaps? How does your project fit into this?)
your methodology (How did you go about the project? E.g.: research methods used for investigative type project; development method chosen for system development project.)
your analysis/design/solution/research (the meat of what you actually did in your project)
evaluation
both the contribution of your project and the limitations of its scope
that your project has scope for future development/has shown up further questions and directions to explore
project management
for development projects - a demo to show what your system does
your understanding of the topic and the wider landscape - answering questions.

Locally use a dummy strategy and ensure through testing