Hey!

Installation

git clone https://github.com/Chiang97912/bet365.com.git
pip3 install -r bet365.com/requeriments.txt --user

Should I paste those lines in a linux-bash-shell or in a python Terminal?

Now that bet also uses a token, do you know an easy way to make it work? I see their JS code and the data that they send initially is something like this: "#\u0003P\u0001__time,S_3AB9480563C146039AE42BA781900CF3000003,D_Z7kOYA==.WPIMdBoup7+GqCd1goysJx9NwZr4G1cLk8r24V3ETl8=\u0000"

And also CONFIG_1_3,OVInPlay_1_3,Media_L1_Z3,XL_L1_Z3_C1_W3 is CONFIG_1_1,OVInPlay_1_1,Media_L1_Z1 now but this didn't make any difference. Any help is appreciated!

I get the cookie the first time but on repeat runs of the script. It never receives that header.

And on such occasions when the cookie is not sent, the server redirects the request to localhost. I supposed, this was an IP blocking issue but it also happens with my local IP which can still access the website on the browser.

But the problem is more pronounced on my VPS that has a UK IP address.

I have done a lot of tweaking on the headers, yet to no avail.

`class MyFactory(WebSocketClientFactory, ReconnectingClientFactory):

def clientConnectionFailed(self, connector, reason):
    print('failed retry!')
    self.retry(connector)

def clientConnectionLost(self, connector, reason):
    self.maxDelay = 10
    print('lost retry!')
    self.retry(connector)

`

Hi guys, I'm in Brazil and I want to follow some football games here. I used this code today and put it to run without modifying any part of the code. That didn't work. The code is frozen in 'updateGameData'. If that had worked, I would parse and change the code to "pt-br" and use the string "Futebol" instead of "football".

Is there some sort of IP block on bet365 ? Because when I repeatedly restarting script it starts returning nothing after subscribing to channel (OVInPlay). So session key in script always different but in browser it remains the same and refreshed after some time. The question is did you noticed that or problem is somewhere else ?

The get_session_id() throws a KeyError:

KeyError: "name='pstk', domain=None, path=None"

I get the cookie the first time but on repeat runs of the script. It never receives that header.

And on such occasions when the cookie is not sent, the server redirects the request to localhost. I supposed, this was an IP blocking issue but it also happens with my local IP which can still access the website on the browser.

But the problem is more pronounced on my VPS that has a UK IP address.

I have done a lot of tweaking on the headers, yet to no avail.

Trying to install, I'm having the problem below:

Running setup.py install for Twisted ... error Exception: Traceback (most recent call last): File "c:\users\hugo.borges\appdata\local\programs\python\python36-32\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str return s.decode(sys.__stdout__.encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf3 in position 9: invalid continuation byte Como resolver? Traduzir do: Inglês 441/5000 Trying to install, I'm having the problem below: Running setup.py install for Twisted ... error Exception: Traceback (most recent call last): File "c: \ users \ hugo.borges \ appdata \ local \ programs \ python \ python36-32 \ lib \ site-packages \ pip \ compat \ __ init__.py", line 73, in console_to_str return s.decode (sys .__ stdout __. encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf3 in position 9: invalid continuation byte

How to solve?

I know that my question is not exactly related to the repository, but I thought it was worth sharing.

I created a thread on stackoverflow and there extremely detailed what my problem is when web scraping the Bet365 site.

Thanks if you can take a look.

https://stackoverflow.com/questions/70883112/problem-scraping-bet365-with-headless-mode-python-selenium-puppeteer

I would like to share the method used to capture the "X-Net-Sync-Term" header of the HTTP request.
Currently the Bet365 index page does not have "boot.nsu (str1, str2)", they have changed the way the website is displayed.

Encrypted initial token:

Method used to capture: [Javascript]

let generate = (param) => {
	let arr;
	
	if(typeof param == 'string')
		arr = eval('[' + param.split('var a=[')[1].split('];')[0] + ']');
	else if(Array.isArray(param))
		arr = param
	else
		return false
		
	let n = arr[397];
	
	let q = arr[158] + '.' + arr[375] + arr[182] + arr[294];

	let parts = q.split('.')

	let compressed = decodeURIComponent(parts[0])
	
	let decompress = function(o) {
		o = decodeURIComponent(o);

		let s = +atob(n) * -1;
		
		s = s % 64;

		let r = '';
		
		for (var t = 0; t < o.length; t++) {
			var u = o.charCodeAt(t)
			  , v = String.fromCharCode((u + s) % 256);
			  
			r += v;
		}
		
		return r;
	}
		
	return decompress(compressed) + '.' + parts[1];
}

The parameter can be the entire HTML returned by the Bet365 website, or an array, if you choose to pass an array, the array must be the one shown in the image.

Ps: The above function will return the token (encrypted) of the page, which in this case is: idS4bd==.KavFYZJwIEwXZ1nkv6zdXI/FqYJYQiHNxQDhvhnLP1E= with this token, you will need to decode

Method used to decode or encode (encode only to explication) function:

class Bet365 {
    mapLen = 64;
    
	charMap = [["A", "d"], ["B", "e"], ["C", "f"], ["D", "g"], ["E", "h"],
        ["F", "i"], ["G", "j"], ["H", "k"], ["I", "l"], ["J", "m"], ["K", "n"], ["L", "o"],
        ["M", "p"], ["N", "q"], ["O", "r"], ["P", "s"], ["Q", "t"], ["R", "u"], ["S", "v"],
        ["T", "w"], ["U", "x"], ["V", "y"], ["W", "z"], ["X", "a"], ["Y", "b"], ["Z", "c"],
        ["a", "Q"], ["b", "R"], ["c", "S"], ["d", "T"], ["e", "U"], ["f", "V"], ["g", "W"],
        ["h", "X"], ["i", "Y"], ["j", "Z"], ["k", "A"], ["l", "B"], ["m", "C"], ["n", "D"],
        ["o", "E"], ["p", "F"], ["q", "0"], ["r", "1"], ["s", "2"], ["t", "3"], ["u", "4"],
        ["v", "5"], ["w", "6"], ["x", "7"], ["y", "8"], ["z", "9"], ["0", "G"], ["1", "H"],
        ["2", "I"], ["3", "J"], ["4", "K"], ["5", "L"], ["6", "M"], ["7", "N"], ["8", "O"],
        ["9", "P"], ["\n", ":|~"], ["\r", ""]];

    encrypt( txt ){
        let result = "";

        for( let i = 0; i < txt.length; i++ ){
			let c = txt.substr(i, 1)
			
            for( let j = 0; j < this.mapLen; j++ ){
                if( c == this.charMap[j][0] ){
                    c = this.charMap[j][1];
                    break;
                }
            }
            result += c;
        }
		
        return result;
    }

    decrypt( txt ){
        let result = "";

        for(let i = 0; i < txt.length; i++){
            let c = txt.substr(i, 1)
			
            for( let j = 0; j < this.mapLen; j++ ){
                if( ":" == c && ":|~" == substr( txt, i, 3 ) ){
                    c = "\n";
                    i+=2;
                    break;
                }
                if( c == this.charMap[j][1] ){
                    c = this.charMap[j][0];
                    break;
                }
            }
            result += c;
        }

        return result;
    }
}

let tool = new Bet365

Decrypted token: FAcuYA==.4XSpij3T2oThjrKHSwWAh2/pNi3iaF17UanESEK59ro=

Websocket:

After sending the token in the initial request, which looks something like this: "time,S{PSTK},D{DecodedToken}"

In this example:
{PSTK} = 62E1FE57F3BF449681C3E380BE1D986A000003
{DecodedToken} = FAcuYA==.4XSpij3T2oThjrKHSwWAh2/pNi3iaF17UanESEK59ro=

Importante note:

1. Sometimes you will receive responses from the WebSocket containing the message: "_SPTBK_D" and "==.", When you receive this, your requests should be updated to tool.decrypt (received token), example in the image above:
   tool.decrypt('RtA4bd==.qBNfyjxIInIdGMbjt/2ayaTyB8IoL8XKAzJU5uf4D6t=')

2. You should always preserve the characters sent on the Websocket, for this reason it is recommended that you intercept messages sent by the websocket through Base64, an example:
   The initial request is: atob("IwNQAV9fdGltZSxTX1BTVEssRF9ERUNPREVEX1RPS0VOAA==")

With this I have the intact and safe initial message, and only need to execute:

let initReq = atob("IwNQAV9fdGltZSxTX1BTVEssRF9ERUNPREVEX1RPS0VOAA==")

initReq = initReq.replace('PSTK', PSTK)
initReq = initReq.replace('DECODED_TOKEN', DecodedToken)

It would be possible to create a version using node.js?

i think this problem

bet365 update..

P + session_id + '????'

added

姜同学，我在抓取bet365数据的时候，遇到一点困难，之前已经给你发过两封电子邮件到你的email中，希望你看到邮件后能够回复我，谢谢

I have a question, is this data only for in play matches or does it grab data for prematch as well?