IR Data? about nintendo_switch_reverse_engineering HOT 7 OPEN

Well I had some progress REing the MCU image transfer protocol thanks to packets I got from testing with libnx and REing the Switch services.

I'll do a write up when I figure out more things.

from nintendo_switch_reverse_engineering.

I'm trying to dig into this. Based on the documentation in this repo I can get a right Joy-Con to push IR input reports, but they have 313 bytes of all 0xff.
@CTCaer Do the wireshark scripts in this repository support a Bluetooth man in the middle? I have a copy of 1-2-Switch handy and can try to see what kind of packets need to get sent to enable and setup the IR camera. I assume that my current code is not turning on the IR LEDs, or that I need to set a noise floor or contrast ratio before meaningful data comes out of the sensor. Thanks!

from nintendo_switch_reverse_engineering.

https://www.youtube.com/watch?v=xiVSB5s3oU4

You can see IR Camera sends the image data to switch in this video. It works just like a camera.
Cannot tell if it also processes the image and send IR markers like wiimote or not.

from nintendo_switch_reverse_engineering.

@mslmanni
It's not as simple as that. Everything is encoded to currently unknown formats.
I tried once to make sense of the main configuration sent to the IR camera, and after 11 hours straight + 10 more, I was fed up and stopped.
But yes, you can actually choose, if it will send an image, objects, figures, etc.

@wormyrocks
Nope.
Try BtleJuice. I don't know if it works though, because it says BLE mitm. And the joy-con communicate on the normal standard bt hid profile, not any BLE related profile.
That's also why Joy-Con work even with a BT2 adapter.

There's also btproxy. Another software solution that manipulates 2 BT devices.
After sep 2017 BT exploits, there is a surge on new mitm tools.

Your current code is not turning the IR Camera/Leds.
The commands are documented, but it's very difficult to decode the packets.
I tried once and felt sick, ahhahah. Also, the packets that you send and receive, use additional checksums btw.

Forget what you know about joy-con normal communication though. The IR camera communication style is way different. You speak to MCU, MCU speaks back, you configure it, reset it, mplah mplah, do more configuring and then you can start using manually polling (depends on the mode you choose) to make joy-con send the big packets that have the IR data.

I will try to get some meaningful packets for you.

from nintendo_switch_reverse_engineering.

Thanks.
Yeah I fired up 1-2-Switch and realized that the IR leds turning on is visible + obvious.
So you have found the subcommand to turn on the LEDs? Can it be added to the big repository? What method did you use to read the configuration - do you have a method to sniff packets or did you find a way to build a joy-con spoof w/ embedded system? Sorry I've been out of the loop for the last few months. I haven't found any discussion of earnest attempts to RE the camera data lately. :( Let me know if there are any threads I should read through in non-obvious places.
In my own attempt, all I'm going from is the info in this repository, i.e. I go to input mode 0x31 and then send 0x11 ... 0x03 0x00 and see that I get long packets full of 0xff.
Perhaps the developers reused wiimote IR camera data pipeline, which is not terribly encouraging but gives us something to go on.
Will definitely take a look at btproxy.
Thanks for your patience and help. This has been my first real experience with reverse engineering anything and so I am happy to learn all I can.

from nintendo_switch_reverse_engineering.

I looked at the Wiimote protocol.
If it was that easy.... :/

A configuration command is actually 2 x21 subcmd packets.
The arguments for the x21 subcmds look like this:
23040901 30900131 24013200 00100001 2e80012f 00000e07 0143c800 14000000 00000000 e2
23040900 15000016 3f0017ef 00180100 2b28002c 28002a00 00205000 07010000 00000000 87
And they configure the MCU for image processing. Similar commands are used to configure it for NFC functions.

The packets sent from the controllers have already-processed data. For example, if you choose the RunClusteringProcessor cmd, it will send objects. If you choose RunImageTransferProcessor, it sends still images (like a camera), with which you can make a video. And so on. So everything is done inside the controller HW. You just get the result.

In my tests, I also found out, that some arguments used encoding (welcome to hd rumble style fun all over again).

The packets were taken with an UART sniffer, on a Switch v1.0.0. I was able to fine tune the sent configurations by using the available nn::irsensor services.

from nintendo_switch_reverse_engineering.

Have you try to get the raw IR data on the Joy-Con right PCB? I found some test points are related to the IR motion camera. But these test points are seen to be the CLK of the IR camera, neither of them is data pin.

from nintendo_switch_reverse_engineering.