Hi, funny you write this. I was just playing with it and noticed the exact same thing. Apparently BoringSSL completely removed some older ciphers which Safari still uses. I'll soon upload a patch to BoringSSL that restores them.

And BTW, I also found out that with BoringSSL there is no need to convert to OpenSSL cipher format. You can use the "regular" names as they appear in Wireshark for example.

from curl-impersonate.

@lwthiker here's a capture of going to google.com I got from my VM, it might be useful.

I've changed the extension to .png so I can upload it to GitHub (I'm lazy). Just change the extension to .pcapng
https://user-images.githubusercontent.com/91278344/154979811-198da2d2-ed3b-4250-96ec-1c75b8aa9305.png

from curl-impersonate.

@bolshoytoster Incredible, thank you!

HTTP headers are important as well, and they must appear in the exact order that Safari sends them out. If you can share them that'll be extremely helpful. Here's how you can get them the most accurately. Takes 2 minutes tops :)

# Generate temporary self-signed key
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.crt
$ cat server.key server.crt > server.pem
# Run socat with SSL server
$ socat -v openssl-listen:8443,reuseaddr,fork,cert=server.pem,verify=0 echo

Then use Safari to browse to https://localhost:8443 and ignore the SSL warning. socat will print all the headers. Make sure you are not in incognito because that affects the headers sometimes.

Thanks again.

from curl-impersonate.

Looking at the pcap from @bolshoytoster, the Client Hello message of Safari is very similar to Chrome's. Need to disable some extensions and change some ciphers. If someone wants to give it a shot, it's probably not very difficult using the already existing Chrome build and the info from the blog post. I'm a bit busy writing automated tests to ensure we don't break the TLS signatures in the future.

from curl-impersonate.

@lwthiker

$ socat -v openssl-listen:8443,reuseaddr,fork,cert=server.pem,verify=0 echo

returns

E SSL_accept(): error:140809C:SSL routine:ss13_get_record:http request

and safari says

Safari Can't Open the Page

Safari can't open the page "localhost:8443" because the server unexpectedly dropped the connection. This somtimes occurs when the serveris busy. Wait for a few minutes, and then try again.
when I go to it.

Probably because it's in a VM.

from curl-impersonate.

Ahh, I got this also the first time. Make sure to use https in the URL bar, i.e. https://localhost:8443. This worked for me with Chrome.

from curl-impersonate.

@lwthiker

GET / HTTP/1.1\r
Host: localhost:8443\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r
Accept-Language: en-US,en;q=0.9\r
Connection: keep-alive\r
Accept-Encoding: gzip, deflate, br\r
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15\r
\r
< 2022/02/21 20:39:35.144192  length=337 from=0 to=336
GET / HTTP/1.1\r
Host: localhost:8443\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r
Accept-Language: en-US,en;q=0.9\r
Connection: keep-alive\r
Accept-Encoding: gzip, deflate, br\r
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15\r
\r

from curl-impersonate.

Thank you @bolshoytoster

from curl-impersonate.

Hey, cool project. Have you actually tried to implement safari so far? I can't get it to use the same ciphers as my browser.

I used this list to get the OpenSSL cipher format. For example, for TLS_AES_128_GCM_SHA256 there is no OpenSSL reference at all. Seems like Safari is pretty tricky to me.

This is my current state only for the ciphers and the headers. I have not dealt with the extensions at all.

#!/bin/bash

# Find the directory of this script
dir=`echo "$0" | sed 's%/[^/]*$%%'`

# The list of ciphers can be obtained by looking at the Client Hello message in
# Wireshark, then converting it using this reference
# https://wiki.mozilla.org/Security/Cipher_Suites
"$dir/curl-impersonate" \
    --ciphers 
    TLS_AES_128_GCM_SHA256,
    TLS_AES_256_GCM_SHA384,
    TLS_CHACHA20_POLY1305_SHA256,
    ECDHE-ECDSA-AES256-GCM-SHA384,
    ECDHE-ECDSA-AES128-GCM-SHA256,
    ECDHE-ECDSA-CHACHA20-POLY1305,
    ECDHE-RSA-AES256-GCM-SHA384,
    ECDHE-RSA-AES128-GCM-SHA256,
    ECDHE-RSA-CHACHA20-POLY1305,
    ECDHE-ECDSA-AES256-SHA384,
    ECDHE-ECDSA-AES128-SHA256,
    ECDHE-ECDSA-AES256-SHA,
    ECDHE-ECDSA-AES128-SHA,
    ECDHE-RSA-AES256-SHA384,
    ECDHE-RSA-AES128-SHA256,
    ECDHE-RSA-AES256-SHA,
    ECDHE-RSA-AES128-SHA \
    -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
    -H 'Accept-Language:  en-US,en;q=0.9' \
    -H 'Connection:  keep-alive' \
    -H 'Accept-Encoding: gzip, deflate, br' \
    -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15' \
    --http2 --false-start --tlsv1.2 --compressed \
    $@

I've added a newline between the ciphers for better readability.

from curl-impersonate.

Okay. I thought the problem was on layer 8 :-D

Thanks for the quick reply.

from curl-impersonate.

Here it is :) #15
Let me know if some ciphers still don't work.

from curl-impersonate.

Update: I have made some progress with this. I didn't want to hardcode everything into the binary (and create a separate build system) so I made the relevant TLS extensions configurable via command line / libcurl options (#16).

For now the following command line gets us closer (but still not there yet), using the Chrome build:

curl-impersonate \
--ciphers TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA:TLS_RSA_WITH_AES_256_GCM_SHA384:TLS_RSA_WITH_AES_128_GCM_SHA256:TLS_RSA_WITH_AES_256_CBC_SHA256:TLS_RSA_WITH_AES_128_CBC_SHA256:TLS_RSA_WITH_AES_256_CBC_SHA:TLS_RSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA:TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA:TLS_RSA_WITH_3DES_EDE_CBC_SHA \
--curves X25519:P-256:P-384:P-521 \
--signature-hashes ecdsa_secp256r1_sha256,rsa_pss_rsae_sha256,rsa_pkcs1_sha256,ecdsa_secp384r1_sha384,ecdsa_sha1,rsa_pss_rsae_sha384,rsa_pss_rsae_sha384,rsa_pkcs1_sha384,rsa_pss_rsae_sha512,rsa_pkcs1_sha512,rsa_pkcs1_sha1 \
--http2 --false-start --compressed \
--tlsv1.0 --no-npn \
--cert-compression zlib

from curl-impersonate.

Update: Using #16 it is now possible to impersonate Safari from the TLS perspective. There's still some work to be done on the HTTP2 headers. The command is:

curl-impersonate \
--ciphers TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA:TLS_RSA_WITH_AES_256_GCM_SHA384:TLS_RSA_WITH_AES_128_GCM_SHA256:TLS_RSA_WITH_AES_256_CBC_SHA256:TLS_RSA_WITH_AES_128_CBC_SHA256:TLS_RSA_WITH_AES_256_CBC_SHA:TLS_RSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA:TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA:TLS_RSA_WITH_3DES_EDE_CBC_SHA \
--curves X25519:P-256:P-384:P-521 \
--signature-hashes ecdsa_secp256r1_sha256,rsa_pss_rsae_sha256,rsa_pkcs1_sha256,ecdsa_secp384r1_sha384,ecdsa_sha1,rsa_pss_rsae_sha384,rsa_pss_rsae_sha384,rsa_pkcs1_sha384,rsa_pss_rsae_sha512,rsa_pkcs1_sha512,rsa_pkcs1_sha1 \
--http2 --false-start --compressed \
--tlsv1.0 --no-npn --no-tls-session-ticket

from curl-impersonate.