tdy / engine-evaluator Goto Github PK

Calling engine-evaluator.php
From the command line:
	php engine-evaluator.php $username [$forceDeepSearch $lichessApiToken $lichessURL]

$username (String): The username that is being investigated. (capitalisation allowed)

$forDeepSearch (Boolean: TRUE, FALSE): True = past 100 games will be looked at and a detailed deepIndex will be returned in the output JSON. False = deepSearch will only be activated if the user’s cheat index is equal to or above 60.

$lichessApiToken (String): The passkey that is used to access secret information from lichess APIs.

$lichessURL (String, URL): The HTTP address of the lichess server.

Output:
	JSON:
	-Normal Output:
	{
	"userId":"falconsag", 	//Username
	"cheatIndex":98,		//CheatIndex from 0->100 higher = likely cheating
	"deepIndex":73,			//DeepIndex of past 100 games from 0->100
	"action":"REPORT",		//Or MARK, or NOTHING
	"reportDescription":	//The report that will be sent to the moderator via user report
		"Cheat Index:  87\/100,  Deep Index:  42\/100\n
		 63 - http:\/\/lichess.org\/kxa5n76m\n
		      Error Rate:  96\/100, Blur Rate:  29\/100\n 
		 59 - http:\/\/lichess.org\/h4dp8ivx\n
		      Error Rate:  74\/100, Blur Rate:  44\/100\n
		 59 - http:\/\/lichess.org\/bn35019d\/black\n
		      Error Rate:  94\/100, Blur Rate:  23\/100\n
		 50 - http:\/\/lichess.org\/ww4egmem\/black\n
		      Error Rate:  67\/100, Blur Rate:  34\/100\n
		 41 - http:\/\/lichess.org\/23zz3sxg\n
		      Error Rate:  66\/100, Blur Rate:  17\/100\n
		 34 - http:\/\/lichess.org\/eaqpjcw6\n
		      Error Rate:  51\/100, Blur Rate:  18\/100\n
		 18 - http:\/\/lichess.org\/5mi19ntr\n
		      Blur Rate:  30\/100, Move-Time Deviation:   5\/100\n
		 16 - http:\/\/lichess.org\/ztst7ex8\/black\n
		      Blur Rate:  31\/100, Move-Time Deviation:   0\/100\n",
	"moveTime":26, 			//The amount of points from move time potentially used for the cheat index (int 0->100)
	"blur":49, 				// “” “” “” (int 0->100)
	"computerAnalysis":49, 	// “” “” “” ""
	"progress":25,			// “” “” “” ""
	"knownEngineIP":0,		// “” “” “” ""
	"Error":0 				//Error code. 0 = normal
	}

	-Player does not have any games:
	{"Error" : 2}

	-Could not reach lichess server (404):
	{"Error" : 1}


	-User Report Example:
Cheat Index:  99/100,  Deep Index:  74/100
100 - http://lichess.org/5y6ztdw2
     Blur Rate: 100/100, Move-Time Deviation: 100/100
100 - http://lichess.org/6rfgtty3
     Error Rate: 100/100, Blur Rate: 100/100
100 - http://lichess.org/uqn6ynpu/black
     Move-Time Deviation: 100/100, Blur Rate: 100/100
 79 - http://lichess.org/ik4hxmrm
     Blur Rate: 100/100, Move-Time Deviation:  58/100
 73 - http://lichess.org/50sw3g5l
     Blur Rate: 100/100, Move-Time Deviation:  46/100
 61 - http://lichess.org/icvg9v9c/black
     Blur Rate: 100/100, Move-Time Deviation:  22/100
 53 - http://lichess.org/j6zgyyer
     Blur Rate: 100/100, Move-Time Deviation:   6/100
 51 - http://lichess.org/akepjxm3
     Blur Rate: 100/100, Move-Time Deviation:   2/100
 45 - http://lichess.org/k3vakrrt
     Blur Rate:  90/100

     The first number is the index (likelihood of cheating) for that game. It is then followed by the URL to the game.
     Below the URL are the indicators that were used to calculate the index.


Aim: To provide a grading system for players with
regards to the likelihood that they are cheating.

Method: There are 5 main points of information that
can be used to aid in identifying cheaters, that are:
	-IP relationship to known cheaters
	-Rating progress
	-Blur rate
	-Perfection of moves (computer analysis)
	-Standard Distribution of move times

The relationship to known cheaters is boolean, so 
points are either allocated or not.

	IPpoints ~ IF IP relationship to known engine

The rating progress is dependant on the win rate,
RD, and opponent ratings. This system is to be improved.
Currently is is a linear relationship with rating increase.

	RIpoints ~ Increase in rating

Points assigned for blur rate is a square relationship
with the proportion of blurs to moves, where the 
proportion is 0 -> 1. This is so players with less
blurs fall off faster.

	BLpoints ~ blur rate ^ 2

Perfection of moves is harder to quantify. 0 inaccuracies,
0 mistakes, and 0 blunders will obviously be assigned the
maximum amounts of points, but at what rate will points
be diminished?

This approach shown here starts each game with 1 point
(the maximum possible), then subtracts the proportion
of each type of error (0 -> 1) from the starting value 
using different coefficients for each.

	CApoints ~ 1 - error rate

The relationship between move time and probability
of cheating is the most complicated process in this system.
By inspection, players who have more consistent move times
have a higher likelihood of cheating. The more consistent
a players move time, the lower the standard distribution (SD)
of move times.

However, for fast games, even a legitimate players SD
will be relatively low. And in long games, a cheaters SD
can be elevated. Using this knowledge, this system
uses the ratio of SD / Mean (average) to determine if a player
is likely cheating. This value is then 

	SDpoints ~ ( 1 - SD / Mean ) ^ 2

These points are then added together with different weights
to each to give an overall Cheat Index.