neykov / extract-tls-secrets Goto Github PK

Decrypt HTTPS/TLS connections on the fly with Wireshark

License: Apache License 2.0

Java 83.26% Shell 16.74%

decrypt ssl tls wireshark java https

extract-tls-secrets's Introduction

extract-tls-secrets

Decrypt HTTPS/TLS connections on-the-fly. Extract the shared secrets from secure TLS connections for use with Wireshark. Attach to a Java process on either side of the connection to start decrypting.

Usage

Download from extract-tls-secrets-4.0.0.jar. Then attach to a Java process in one of two ways:

Attach on startup

Add a startup argument to the JVM options: -javaagent:<path to jar>/extract-tls-secrets-4.0.0.jar=<path to secrets log file>

For example to launch an application from a jar file run:

java -javaagent:~/Downloads/extract-tls-secrets-4.0.0.jar=/tmp/secrets.log -jar MyApp.jar

To launch in Tomcat add the parameter to CATALINA_OPTS:

CATALINA_OPTS=-javaagent:~/Downloads/extract-tls-secrets-4.0.0.jar=/tmp/secrets.log bin/catalina.sh run

Attach to a running process

Attaching to an existing Java process requires a JDK install with JAVA_HOME pointing to it.

To list the available process IDs run:

java -jar ~/Downloads/extract-tls-secrets-4.0.0.jar list

Next attach to the process by executing:

java -jar ~/Downloads/extract-tls-secrets-4.0.0.jar <pid> /tmp/secrets.log

Decrypt the capture in Wireshark

To decrypt the capture you need to let Wireshark know where the secrets file is. Configure the path in Preferences > Protocols > TLS (SSL for older versions) > (Pre)-Master-Secret log filename.

Alternatively start Wireshark with:

wireshark -o tls.keylog_file:/tmp/secrets.log

The packets will be decrypted in real-time.

For a step by step tutorial of using the secrets log file (SSLKEYLOGFILE as referenced usually) refer to the Peter Wu's Debugging TLS issues with Wireshark presentation. Even more information can be found at the Wireshark TLS page.

Requirements

Requires at least Oracle/OpenJDK Java 6. Does not support IBM Java and custom security providers like Bouncy Castle, Conscrypt.

Building

git clone https://github.com/neykov/extract-tls-secrets.git
cd extract-tls-secrets
mvn clean package

Running the integration tests requires Docker to be installed on the system:

mvn verify

Troubleshooting

If you get an empty window after selecting "Follow/TLS Stream" from the context menu or are not seeing HTTP protocol packets in the packet list then you can fix this by either:

Save the capture as a file and open it again
In the Wireshark settings in "Procotols/TLS" toggle "Reassemble TLS Application Data spanning multiple SSL records". The exact state of the checkbox doesn't matter, but it will force a reload which will force proper decryption of the packets.

The bug seems to be related to the UI side of wireshark as the TLS debug logs show the message successfully being decrypted.

Reports of the problem:

If "Follow/TLS Stream" is not enabled the server is probably on a non-standard port so Wireshark can't infer that the packets contain TLS traffic. To hint it that it should be decoding the packets as TLS right click on any of the packets to open the context menu, select "Decode As" and add the server port, select "TLS" protocol in the "Current" column. If it's still not able to decrypt try the same by saving the capture in a file and re-opening it.

extract-tls-secrets's People

Contributors

Stargazers

Watchers

Forkers

polarctos frohoff benjamin-bin-shen paulyd210 ericcong nguyenkhong 3x-m4ch1na bostko avmohan schlauerwolf wheelerlaw pryma zqheng lukas88662 fachher gr3yr0n1n o2e 5l1v3r1 cansou daker2016 0xbadcoffe lecogiteur jas502n crackercat hongruixing scottfield gavz 32291382 timb-machine-mirrors stevenseeley liuwenru scriptidiot futurebody weiwhy bharadwajyas neodark-7 alvinkwok1 shrekwang sunshinelin12138 levelupsoft mdurand76 jeff-cn jm-g gtjbtits sinhaman1909 yili001 daboss7627 fishjam nephewtom atomitl oliverfun raboof pirarucu manvscode singhabhishek4u t-santoshsahu longyoudong ykd007 tsaarni download8866 kgibm danman40th makamuy sleepie666 palfans fariouche jarekwaszczuk

extract-tls-secrets's Issues

Failed to attach to java process xxxx. Cause: Unable to open socket file: target process not responding or HotSpot VM not loaded

Hello, thank you for the software!

Sadly, I am getting the following error:

Failed to attach to java process xxxx. Cause: Unable to open socket file: target process not responding or HotSpot VM not loaded

What are the pre-requisites for the target process responding? Can I target Glassfish server?

How to deattach?

Hello,

I can successfully attach to my running Java application and I get a steady stream of session keys in the logfile.
Tracing and decrypting packages works fine, so the main goal achieved.
But how do I de-attach? I could not find a running java pid for extract-tls-secrets to kill and I do not want to leave the key logging enabled.
I am probably missing something obvious.. new to this.. :-)

Best Regards,
Thomas

Published 4.0.0 package is not installable on Linux

The pom.xml file contains reference to the path of the package that's specific to the system where the package is built.

See https://search.maven.org/artifact/name.neykov/extract-tls-secrets/4.0.0/jar:

<systemPath>/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/../lib/tools.jar</systemPath>

can't see the TLS response of request

Hi, thanks for providing this tool, after following the steps in README, I successfully configured the connection between Wireshark and my java-httpclient demo, and now Wireshark captured the TLS request sent in httpClient(the test https link is https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie), but I can't see the TLS response of this request, Is this because of my Wireshark filter setting, or need extra configuration?
The Wirshark screenshot:

No BCJSSE support?

Hi!

Is there no way to add BCJSSE support?

Cannot decrypt TLS 1.2 with TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite selected

the TLS version is 1.2 and the selected cipher suite is TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
The secrets log is like below:
RSA Session-ID: Master-Key:888E59D7EDF19AB2CB6DD704D924E0BAF703B5D998041147D4569D91C0BB5356CD3C694AFCDB2B8ED5CB5CEE1C116D59
RSA Session-ID: Master-Key:E93F8E6EF8A255A6D5BA400233F51C6BAE705CE74E86BBC993180A7DDC9B946FE9125D4F44026A16643264BAD97E7582
RSA Session-ID: Master-Key:BBD7BFBFA0C5000A2CCA2728123188CC3E620911D32DD9896AD068A0E5171D317A270FE860CCF500B524F621DA1CC086
RSA Session-ID: Master-Key:78388503F830251EAFB24BFA17036E99A5A0154E7C0CADB0F22FFC36F7800CB2C609B6BF67F72AF6984360BC8302AD54
RSA Session-ID: Master-Key:E7F59A73F83A76F227DC3581DB58D655BD58223449C4C8B299EC4FBA120C23AC0A6959FB6F5F33510B9A73EA3FABC586
RSA Session-ID: Master-Key:C74452C852514786E15EDD2D9818E3C6804916C0B95929E397A637318F15FE803A1D1F2AB06396C9AA8EF09F1098B6F0

But the wireshark cannot decrypt the TLS packet.

Cannot attach to the process

When I try to run the command:
java -jar extract-tls-secrets-4.0.0.jar <pid> /tmp/secrets.log
I get the error:
Failed to attach to java process <pid>. Cause: Agent JAR loaded but agent failed to initialize

No secrets log being generated (but appears to be attaching)

So I'm a little puzzled. I have a Java jar file that I want to debug. I am launching it with the following command:

java -javaagent:"extract-tls-secrets-4.0.0.jar=secrets.log" -jar MyApp.jar

I have put the extract-tls-secrets jar file in the same directory as my jar file, and am launching it from that directory as well. I see the jar launching as follows:

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Aug 05, 2022 7:56:45 PM name.neykov.secrets.AgentMain main
INFO: Successfully attached agent C:\Users\MyUsername\Documents\Programming\MyAppDir\target\extract-tls-secrets-4.0.0.jar. Logging to C:\Users\MyUsername\Documents\Programming\MyAppDir\target\secrets.log
...
More logs from the jar I'm trying to analyze

However, there is no secrets.log file getting generated in that directory. I've also tried sending the secrets.log file a few other places just to see if it made a difference but have seen none.

I know for a fact this jar is communicating with TLS because I can see the encrypted traffic in Wireshark.

Just to see if there was any additional information getting logged, I also tried launching as follows:

java -D"java.util.logging.config.file=logging.properties" -javaagent:"extract-tls-secrets-4.0.0.jar=secrets.log" -jar MyApp.jar

I added a logging.properties file with this content:

handlers=java.util.logging.FileHandler
java.util.logging.FileHandler.pattern=debug.log
java.util.logging.FileHandler.limit=50000
java.util.logging.FileHandler.count=1
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.FileHandler.level=FINEST

This produces the following debug.log file in the launch directory:

Aug 05, 2022 7:54:49 PM name.neykov.secrets.AgentMain main
INFO: Successfully attached agent C:\Users\MyUsername\Documents\Programming\MyAppDir\target\extract-tls-secrets-4.0.0.jar. Logging to C:\Users\MyUsername\Documents\Programming\MyAppDir\target\secrets.log.

And that's all that ever gets written to the log file. My jar file is threaded, but I wouldn't at all expect that to be an issue since I've used this before to debug a Tomcat app. Any ideas on where I can go next?

Provider WindowsAttachProvider could not be instantiated

JAVA_HOME C:\Program Files\Java\jdk1.8.0_281
I get this error when trying to list available process IDs

java -jar extract-tls-secrets-4.0.0.jar list
java.util.ServiceConfigurationError: com.sun.tools.attach.spi.AttachProvider: Provider sun.tools.attach.WindowsAttachProvider could not be instantiated

attach problem

(LINUX) Can someone please guide me how and where to attach the file.. coz am about throw this laptop out the window...
In terminal It tells me number that i need attach to and when i type the number it says theres no java at this process then gives me new number... and again and again...
I would be gratefull if someone could save me from a headache

Unable to log down information

I am trying to use the extract-tls-secrets-4.0.0.jar on OpenJDK13, however, I am not able to find any information generated although my java process can be started. Can you advise?

From the log, it looks like the extract-tls-secrets-4.0.0.jar is loaded.

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Apr 22, 2020 7:29:54 AM name.neykov.secrets.AgentMain main
INFO: Successfully attached agent /data1/config/extract-tls-secrets-4.0.0.jar. Logging to /data1/logs/secrets.log.
WARNING: An illegal reflective access operation has occurred

EXPORTER_SECRET not work

EXPORTER_SECRET not logged, thanks

Attaching to running process with missing tools.jar

I'm currently trying to attach extract-ssl-secrets to a running Java process. I'm using OpenJDK 12.0.2 and OpenJDK 13.0.2
It is missing the tools.jar, therefore the extract-ssl-secrets does not work.

Invalid JAVA_HOME environment variable 'C:\jdk-12.0.2'.
Must point to a local JDK installation containing a 'lib/tools.jar' file.

Is there a workaround or a fix for this?

No CLIENT_RANDOM output with Java 8, 11 and later

In my tests, the callback function onCalculateKeys only seems to be called with Java 9 and 10. I could not get CLIENT_RANDOM with Java 8, 11 and later.

With Java 9 and 10, I can get both entries in the key log, e.g.,

RSA Session-ID: _xxx_ Master-Key: _yyy_
CLIENT_RANDOM _zzz_ _yyy_

But with Java 8, 11 and later, I'm only getting RSA Session-ID in the key log without CLIENT_RANDOM.

Not sure is anyone else experiencing this problem?

Can't see TLS response

Similar to #13 I can't see the TLS response to my request.

Unlike that issue, it is not a simple issue of my filters. I have the entire stream showing, but the HTTP response data is not decrypted, even though the request is.

The elided contents of my secrets log:

SERVER_HANDSHAKE_TRAFFIC_SECRET a b
SERVER_HANDSHAKE_TRAFFIC_SECRET a b
CLIENT_HANDSHAKE_TRAFFIC_SECRET a c
CLIENT_HANDSHAKE_TRAFFIC_SECRET a c
SERVER_TRAFFIC_SECRET_0 a d
SERVER_TRAFFIC_SECRET_0 a d
CLIENT_TRAFFIC_SECRET_0 a e
CLIENT_TRAFFIC_SECRET_0 a e

Any ideas of what to check?

Doesn't Work with Tomcat 9 (Tested with Tomcat 9.0.12)

Attaching to IBM java

Hi guys,
I've tried attaching to IBM java process but not able to get any log outputs.
IBM J9 VM (build 2.9, JRE 1.8.0 )
Thanks.

Error: Could not find or load main class name.neykov.secrets.AgentAttach

Hi, nice tool, by the way!
I am trying to attach to a Java process on a CentOS machine that is currently being executed by user runner.
When logging with my user tomas, java -jar extract-tls-secrets-4.0.0.jar list runs fine and shows:

[tomas@myhost tls-decrypt]
$ java -jar extract-tls-secrets-4.0.0.jar list
  3259833 extract-tls-secrets-4.0.0.jar list

Then, after becoming root, it also let me do a list, which shows the process I want to attach to (the one with pid=3109):

[tomas@myhost tls-decrypt]
$ sudo su
[root@myhost tls-decrypt]
# java -jar extract-tls-secrets-4.0.0.jar list
  3109 com.example.containers.pretty.Pretty -mybatis.environment rac -configuration /etc/core/configuration.xml -connections 1
  3259908 extract-tls-secrets-4.0.0.jar list
  2551 com.example.workers.demux.Demux -mybatis.environment rac -configuration /etc/core/configuration.xml
  2935 com.example.containers.restzly.Restzly -mybatis.environment rac -configuration /etc/core/configuration.xml -connections 2

If I try to attach to it, it fails with:

[root@myhost tls-decrypt]
# java -jar extract-tls-secrets-4.0.0.jar 3109 /tmp/secrets.log
Failed to attach to java process 3109. Cause: Unable to open socket file: target process not responding or HotSpot VM not loaded

So I try to attach sudoing to user runner:

[root@myhost tls-decrypt]
# su runner

[runner@myhost tls-decrypt]
$ java -jar extract-tls-secrets-4.0.0.jar list
Error: Could not find or load main class name.neykov.secrets.AgentAttach

which returns this basic error of not finding or loading main class...
I don't really get why is returning that... it must be something basic I am not aware of...

Could you please suggest what else can I try?
Thanks in advance!

No output file

I was using your tool to capture keys during a gradle publish task run, tried both attach on startup or attach to the running process, but can't get the output file,

I was using this gradle publish example project
https://github.com/jfrog/project-examples/tree/master/gradle-examples/gradle-example-publish

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/

[project-examples/gradle-examples/gradle-example-publish]$ /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64//bin/java -Xmx64m -Xms64m -javaagent: ~/project-examples/extract-tls-secrets-4.0.0.jar=/tmp/secrets.log -Dorg.gradle.appname=gradlew -classpath ~/project-examples/gradle-examples/gradle-example-publish/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain clean build publish
Apr 20, 2021 6:36:08 AM name.neykov.secrets.AgentMain main
INFO: Successfully attached agent ~/project-examples/extract-tls-secrets-4.0.0.jar. Logging to /tmp/secrets.log.

BUILD SUCCESSFUL in 2s
12 actionable tasks: 12 executed

ls: cannot access /tmp/secrets.log: No such file or directory

Also tried running gradle task first(./gradlew clean build publish) then attach to it, same result...
java -jar extract-tls-secrets-4.0.0.jar $(java -jar extract-tls-secrets-4.0.0.jar list | grep publish | awk '{print $1}') /tmp/logkey.txt

Seemed I'm stuck here, any advice? Thanks! @neykov