Boot loop after restoring about watoi HOT 59 CLOSED

Success! The only thing that needs to be changed in the plist file is the file size. I'll write up the complete procedure tomorrow. A lot of chats seem to have ended up archived, will have to move those out by hand.

from watoi.

Steps:

1. I Used WhatsApp-Key-DB-Extractor-4.7-E1.0 to extract an unencrypted msg store.db from android.
2. Downloaded WhatsApp_v2.19.71_1.6r-105.ipa
3. Used watoi to convert the database, fixed the code for the null error as described in issue #31
4. Installed whatsapp on new iPhone, activated it so that it creates the database
5. connect iPhone to Mac, create backup. Copy that backup to safety before you start the next steps.
6. Upload converted ChatStorage.sqlite to another app in iTunes (old)/Finder (new) by dragging it onto the app under the files tab.
7. Backup the iPhone again.
8. Find out the id of the app in step 6 by using:
   DB=Manifest.db; test -f $DB && sqlite3 $DB -header "select * from Files where relativePath like '%ChatStorage%';"
   Run this in your /Users/username/Library/Application Support/MobileSync/Backup/backupnumber folder.
9. extract the plist blob from whatsapp:
   sqlite3 manifest.db "select hex(file) from Files where fileID == '7c7fba66680ef796b916b067077cc246adacf01d'" | xxd -r -p > whatsapp.plist
10. extract the plist from the other app in step 6:
    sqlite3 manifest.db "select hex(file) from Files where fileID == 'IDFROMSTEP8'" | xxd -r -p > otherapp.plist
11. Copy the file size parameter from the otherapp plist and put it in the whatsapp.plist
12. Do the file swap; I only copied the new database over the small whatsapp file (use the 7c and other app id's to find the correct folder and file).
13. Copy the manifest.db from your backup somewhere else and only open it there with a db viewer. I opened it with DB browser for sqlite (https://sqlitebrowser.org/dl/) . Update the whatsapp blob through the interface by replacing it with the one from step 11 and save the database:

• Click "browse data"
• Filter the fileID column on 7c7fba66680ef796b916b067077cc246adacf01d
• Click on the blob field (last column). Binary contents appear to the right.
• Click import, import the edited whatsapp plist from step 11.
• Click apply
• Click write changes at the top of the editor.

14. Write out the new hex string from that database using
    DB=Manifest.db; test -f $DB && sqlite3 $DB 'select HEX(file) from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";'
15. Copy the hex string part.
16. go back to the backup dir with the actual manifest.db. Do not open it with an editor in the actual backup location!
17. Update the hex string from step 15:
    DB=Manifest.db; test -f $DB && sqlite3 $DB "UPDATE Files SET file = X'YOURHEXSTRINGHERE' where fileID == '7c7fba66680ef796b916b067077cc246adacf01d';"
18. Restore the backup to your iPhone.

from watoi.

@cool00geek yes, you're right - no need for step 6, command is correct.

from watoi.

I am on iOS 14.2, I don't use iTunes because I'm on OSX Big Sur which has the sync app built into finder.

from watoi.

@73567876543783 fixed. Please re-download the script from the same link, I've pushed a new revision.

PlistBuddy (one of the system utils that can edit plists) silently converted binary to xml form, which went unnoticed by me. I haven't found an option to preserve it, so had to use the second tool, plutil, to convert back. When writing the initial script I've tried plutil first, but was unable to persuade it to make required changes. What a mess.

It boots, and all Whatsapp messages show up! Great! Thank you so much.

from watoi.

@73567876543783 great news! Thank you for helping out with the testing!

from watoi.

I just ran the script and can confirm that it worked perfectly for me.
iPhone 8 running iOS 14.3 beta and MacOS Catalina

from watoi.

Update to my previous comment.
I was able to restore the chat history after the second attempt. (I did no extra steps but the ones mentioned in the README)
Downloaded a different ipa from the same link mentioned above. Also, I have made the fix mentioned in #31 even though the script was not crashing in my case.

I have one issues now:

• Starred messages do not come up as starred after transfer. Most of the messages seem to be there but the ones marked as starred before are not showing up like that anymore.

from watoi.

Continuing the discussion started in #37.

This time I used the latest IPA from here and the version was matching the version of WhatsApp installed on my iPhone.

The restore process completed without issues. The only thing that confused me was that when I opened WhatsApp there was no restored chats. But then it turned out they all are "archived". To see archived chats one needs to pull the chat list down. Probably it needs to be added to the readme (or fixed).

Thank you for help!

from watoi.

Hi, @AnkithaMahajan.

This never happened to me when doing initial development, even though I've uploaded corrupted whatsapp DB a few times.

Does the restore of the original (unmodified) backup works as expected?

It's a good practice to provide more details of your setup when reporting bugs. In this case it's iOS and iTunes versions. Please specify file size of original and modified whatsapp DB, and a path (relative to backup root) to the DB file.

from watoi.

Hello @residentsummer,

I’d like to add to this as I experienced the same issue, which hopefully we’ll be able to solve. I’m trying to move my WhatsApp messages from an android to an iPhone using iOS version 14.2 and iTunes version 12.8.2.3.

My original WhatsApp DB file, the one found in my iTunes backup named after the blob, was 610KB. My modified ChatStorage.sqlite and subsequent DB file, which resulted from your script, was 102,9MB. My unencrypted msgstore.db file from Android was 225MB.

My path to the DB was 7c/7cxxxxxxxxxxxxxxx. I can give you the full blob but I’m not sure if this is sensitive information. The file looked to contain all my messages from WhatsApp whilst looking at in in textedit.

After trying to restore my iPhone with the the modified DB in place, the Apple logo kept flashing without ever going away. I went into DFU and restored my iTunes backup, after replacing the modified database with the original one again, which was possible without any issues.

Some things which might be the cause:
I used a WhatsApp .ipa of version 2.20.110 (https://iosninja.io/ipa-library/download-whatsapp-watusi-ipa-ios) whilst my current WhatsApp install is of version 2.20.121. I couldn’t find an up to date one.

The last step of your instructions led to a file in the backup with a “?” appended to it’s name. It was the same size as the ChatStorage.sqlite so the copy seemed to have gone right. Still it was a bit strange. Before that the terminal only returned an > or said that the directory was not available. But in the end the final file looked alright.

It feels as if just one small thing is left out which somehow makes WhatsApp say, my files are not right, making the OS block the full restore as a safety measure. As far as I can say as a lay-ish person. I suspect that the difference between the .ipa and installation causes this. If you know where to find or how to extract an up-to-date .ipa let me know, so I can check if this is the cause. I'll search around a bit more in the meantime.

Thanks in advance!

If you need more information let me know.

from watoi.

@73567876543783 Thanks for the detailed information!

My path to the DB was 7c/7cxxxxxxxxxxxxxxx. I can give you the full blob but I’m not sure if this is sensitive information. The file looked to contain all my messages from WhatsApp whilst looking at in in textedit.

Seems like this blob name is the same for all iphones and iOS versions. Mine is 7c7fba66680ef796b916b067077cc246adacf01d.

The last step of your instructions led to a file in the backup with a “?” appended to it’s name. It was the same size as the ChatStorage.sqlite so the copy seemed to have gone right. Still it was a bit strange. Before that the terminal only returned an > or said that the directory was not available. But in the end the final file looked alright.

Not sure I understood that part... Which file had a question mark in its name?

I suspect that the difference between the .ipa and installation causes this. If you know where to find or how to extract an up-to-date .ipa let me know, so I can check if this is the cause.

It's unlikely the cause of failed restore - the difference in version may lead to issues starting from app launch and onward.

So for now I have few hypotheses:

1. Backup structure has changed since iOS 14.
2. The file size began to matter for blobs inside the backup.
3. Difference in size (610 KB vs 100+ MB) lead to the situation where new file require more filesystem extents for storage. Speculating on the backup/restore process of the iOS, this may lead to corruption if the procedure employs low-level FS manipulations.

To test (1) I'll need to update my phone to iOS 14 and tinker with the backup. For (2) and (3) we might take a practical approach - copy modified DB to running iPhone with iTunes, make another backup and perform a "rename" inside Manifest.db.

from watoi.

Hi @residentsummer,

Thanks for responding, and your work on the script of course!

Mine is 7c7fba66680ef796b916b067077cc246adacf01d

Same for me, the file was placed in 7c/7c7fba66680ef796b916b067077cc246adacf01d.

Not sure I understood that part... Which file had a question mark in its name?

The modified 7c7fba66680ef796b916b067077cc246adacf01d was called 7c7fba66680ef796b916b067077cc246adacf01d?. This was a error on my part made during the cp which I just figured out. I blindly copied cp ChatStorage.sqlite "~/Library/Application Support/MobileSync/Backup/<backup>/ab/abcdef01234567890" which should not have the " before and after the path right?

To test (1) I'll need to update my phone to iOS 14 and tinker with the backup. For (2) and (3) we might take a practical approach - copy modified DB to running iPhone with iTunes, make another backup and perform a "rename" inside Manifest.db.

Okay, I can test (2) and (3) for you if possible. To check if my procedure is correct:

1. Backup iPhone with unmodified DB
2. Move modified DB to iPhone (through an App within the File Sharing option in iTunes?)
3. Backup iPhone with unmodified & modified DB on iPhone
4. Remove unmodified DB and replace with modified DB within backup
5. Restore backup with modified DB in place

I'm not sure about (2) and (4) as you mention that I have to rename within the Manifest.db. I don't know how I'd make the Manifest.db point towards the modified DB, if that's what you mean here.

Thanks in advance!

from watoi.

I'm having an identical issue. I'm on iOS 14.3 on an iPhone 8, and using MacOS 10.15.6.

The sqlite DB from the itunes backup was 385KB, the db from Android was 56.7 MB and the patched db was 7.1MB. Upon injecting the modified database into the itunes backup and restoring it, my iPhone also bootloops and I had to restore it.

I'm guessing there's a field in the manifest which verifies the hash or file size.

from watoi.

@73567876543783

The modified 7c7fba66680ef796b916b067077cc246adacf01d was called 7c7fba66680ef796b916b067077cc246adacf01d?. This was a error on my part made during the cp which I just figured out. I blindly copied cp ChatStorage.sqlite "~/Library/Application Support/MobileSync/Backup//ab/abcdef01234567890" which should not have the " before and after the path right?

In this case double quotes are one way to protect whitespace in Application Support (or cp command will think that we've provided 3 arguments, not 2). I'm not sure how question mark ended up in the name, but let's skip this - you've already figured out how to place modified blob properly :)

To check if my procedure is correct:

1. Backup iPhone with unmodified DB
2. Move modified DB to iPhone (through an App within the File Sharing option in iTunes?)
3. Backup iPhone with unmodified & modified DB on iPhone
4. Remove unmodified DB and replace with modified DB within backup
5. Restore backup with modified DB in place

This plan is perfectly fine. You're right about the (2) as well. A quick reminder of how to do it:

Open iTunes, choose your iPhone and select "File Sharing" tab. There will be a list of apps that support uploading documents via iTunes. Choose any (e.g. Pages) and press "Add" button at the bottom. Seems that it's possible to upload files of any type, even if they're not supported by the app.

We'll need to do some figuring out to make (4). When uploading modified DB through iTunes, please use filename ChatStorage.sqlite. After making a backup (3), go to it's root directory (where Manifest.db is stored), run the following command and post its output here:

DB=Manifest.db; test -f $DB && sqlite3 $DB -header "select * from Files where relativePath like '%ChatStorage%';"

In my case the output looks as follows:

> DB=Manifest.db; test -f $DB && sqlite3 $DB -header "select * from Files where relativePath like '%ChatStorage%';"
fileID|domain|relativePath|flags|file
7c7fba66680ef796b916b067077cc246adacf01d|AppDomainGroup-group.net.whatsapp.WhatsApp.shared|ChatStorage.sqlite|1|bplist00'(X$versionX$objectsY$archiverT$top

Hopefully we'll get 2 entries after (2) and (3) :)

@cool00geek you can try this as well if you're willing to help.

from watoi.

I moved the File through the app Strava as you can see:

DB=Manifest.db; test -f $DB && sqlite3 $DB -header "select * from Files where relativePath like '%ChatStorage%';"
fileID|domain|relativePath|flags|file
719c3655b722fa17d4d148ef15036312ea947fa0|AppDomain-com.strava.stravaride|Documents/ChatStorage.sqlite|1|bplist00?
X$versionY$archiverT$topX$objects
eb59c2823d09cc43f3794f6b9b132cf9b770ab77|HomeDomain|Library/Mobile Documents/57T9237FN3~net~whatsapp~WhatsApp/Accounts/31621484036/backup/.ChatStorage.sqlite.icloud|1|bplist00?
X$versionY$archiverT$topX$objects
7c7fba66680ef796b916b067077cc246adacf01d|AppDomainGroup-group.net.whatsapp.WhatsApp.shared|ChatStorage.sqlite|1|bplist00?
X$versionY$archiverT$topX$objects

Seems like this is what you were expecting. Apparently there is also an .icloud version of the DB on my device.

from watoi.

@73567876543783 that's great.

To swap old and new database files in the backup, do the following (for other readers: these instructions are specific to @73567876543783 case):

1. Make copies of all files you're going to touch (Manifest.db, 719c3655b722fa17d4d148ef15036312ea947fa0, 7c7fba66680ef796b916b067077cc246adacf01d) or copy of the whole backup.
2. Change to the root dir of the backup you're going to modify.
3. Paste this into a file named swap.sql:

select * from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";
select * from Files where fileID == "719c3655b722fa17d4d148ef15036312ea947fa0";
update Files set fileID = "719c3655b722fa17d4d148ef15036312ea947fa0.tmp", relativePath = "Documents/ChatStorage.sqlite", domain = "AppDomain-com.strava.stravaride" where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";
update Files set fileID = "7c7fba66680ef796b916b067077cc246adacf01d", relativePath = "ChatStorage.sqlite", domain = "AppDomainGroup-group.net.whatsapp.WhatsApp.shared" where fileID == "719c3655b722fa17d4d148ef15036312ea947fa0";
update Files set fileID = "719c3655b722fa17d4d148ef15036312ea947fa0" where fileID == "719c3655b722fa17d4d148ef15036312ea947fa0.tmp";
select * from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";
select * from Files where fileID == "719c3655b722fa17d4d148ef15036312ea947fa0";

3. Run the script DB=Manifest.db; test -f $DB && sqlite3 $DB '.read swap.sql' || echo "No database file"
4. Swap the blobs themselves:

cp 7c7fba66680ef796b916b067077cc246adacf01d 719c3655b722fa17d4d148ef15036312ea947fa0.tmp
cp 719c3655b722fa17d4d148ef15036312ea947fa0 7c7fba66680ef796b916b067077cc246adacf01d
cp 719c3655b722fa17d4d148ef15036312ea947fa0.tmp 719c3655b722fa17d4d148ef15036312ea947fa0

Cross your fingers and try to restore the modified backup :)

from watoi.

Great work guys, I am anxiously awaiting this final fix for the restore procedure. I managed to piece together the steps needed to get the database from Android 10 without root by piecing together various bits from the xda-developers forum.. I can help in documenting those steps and testing the replaced database restore procedure.

from watoi.

Alright, I tried it again.

I changed step 4 to:

cp 7c/7c7fba66680ef796b916b067077cc246adacf01d 71/719c3655b722fa17d4d148ef15036312ea947fa0.tmp
cp 71/719c3655b722fa17d4d148ef15036312ea947fa0 7c/7c7fba66680ef796b916b067077cc246adacf01d
cp 71/719c3655b722fa17d4d148ef15036312ea947fa0.tmp 71/719c3655b722fa17d4d148ef15036312ea947fa0

Because the files are not in the root of the backup folder. Without the subfolders added it would return "No such file or directory". Should this also be done in the swap.sql?

For some reason the restore takes a very long time and I haven't been able to complete the restore after two attempts. The time remaining often reduces before it increases again, making no real progression.

from watoi.

@73567876543783 yeah, sorry, I forgot about the folders...

Does the original backup still restore at normal speed? I've found numerous reports about ridiculous restore times under normal conditions (== unmodified backup), even some articles on 3rd party resources on how to fix that. If the original backup restore time is unaffected, though, we'll need to think again of what is wrong with new approach. :/

from watoi.

Instead of doing the complete restore it might be useful to use a tool like Reincubate iPhone Backup Extractor? The free version shows me the structure of my backup, so I assume once the Manifest database has been changed correctly, the chatstorage should show up correctly in the extractor under expert mode.

from watoi.

I replaced the ChatStorage.sqlite file using backup extractor and still ran into the blinking iPhone logo. So it's not really about the location but about the file itself..

from watoi.

I wonder if the time stamp of the file would be causing issues, since the file would have been modified later than the backup. I don't have access to a Mac at the moment but maybe changing the time the file was modified in the backup to the time the backup was created might allow it to work?

from watoi.

@cool00geek with new approach we do not modify files inside the backup, just changing paths. So in theory timestamps should be ok.

from watoi.

@73567876543783 guess we have to compare the file info in binary fields of Manifest.db. Could you please post outputs of the following commands against unmodified backup with both files (e.g. when you've uploaded ChatStorage.sqlite with iTunes)?

DB=Manifest.db; test -f $DB && sqlite3 $DB 'select HEX(file) from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";'
DB=Manifest.db; test -f $DB && sqlite3 $DB 'select HEX(file) from Files where fileID == "719c3655b722fa17d4d148ef15036312ea947fa0";'

from watoi.

I installed whatsapp on my new iPhone and activated it. It gives me an empty database with only the chat groups. That is file 7c7fba66680ef796b916b067077cc246adacf01d: hex output is:

62706C6973743030D4010203040506070A582476657273696F6E592461726368697665725424746F7058246F626A6563747312000186A05F100F4E534B657965644172636869766572D1080954726F6F748001A40B0C242555246E756C6CDC0D0E0F101112131415161718191A1B1C1D1E1F2021221B235C4C6173744D6F64696669656455466C6167735747726F757049445624636C6173735F10104C6173745374617475734368616E67655C52656C6174697665506174685542697274685453697A655B496E6F64654E756D626572544D6F6465565573657249445F100F50726F74656374696F6E436C617373125FB3F5DF10001101F58003125FB3D80B8002125FB3C189120005E000120001ABC81181A410045F10124368617453746F726167652E73716C697465D2262728295A24636C6173736E616D655824636C6173736573564D4246696C65A2282A584E534F626A65637400080011001A00240029003200370049004C005100530058005E00770084008A0092009900AC00B900BF00C400D000D500DC00EE00F300F500F800FA00FF01010106010B011001130115012A012F013A0143014A014D0000000000000201000000000000002B00000000000000000000000000000156

I then uploaded a watoi converted database into my goodreads sandbox, id eb004981f01eb150a44ed3680d80726c8a74934e. Hex output is:

62706C6973743030D4010203040506070A582476657273696F6E592461726368697665725424746F7058246F626A6563747312000186A05F100F4E534B657965644172636869766572D1080954726F6F748001A40B0C232455246E756C6CDC0D0E0F101112131415161718191A1B1C1D1E191F20211B225C4C6173744D6F64696669656455466C6167735747726F757049445624636C6173735F10104C6173745374617475734368616E67655C52656C6174697665506174685542697274685453697A655B496E6F64654E756D626572544D6F6465565573657249445F100F50726F74656374696F6E436C617373125FB3F04310001101F58003125FB50D8980021205022000120007C4C01181A410045F101C446F63756D656E74732F4368617453746F726167652E73716C697465D2252627285A24636C6173736E616D655824636C6173736573564D4246696C65A22729584E534F626A65637400080011001A00240029003200370049004C005100530058005E00770084008A0092009900AC00B900BF00C400D000D500DC00EE00F300F500F800FA00FF01010106010B010E0110012F0134013F0148014F01520000000000000201000000000000002A0000000000000000000000000000015B

When I just copy the empty database from the backup and upload it into the goodreads sandbox, the file id is eb004981f01eb150a44ed3680d80726c8a74934e. Hex output is:

62706C6973743030D4010203040506070A582476657273696F6E592461726368697665725424746F7058246F626A6563747312000186A05F100F4E534B657965644172636869766572D1080954726F6F748001A40B0C232455246E756C6CDC0D0E0F101112131415161718191A1B1C1D1E191F20211B225C4C6173744D6F64696669656455466C6167735747726F757049445624636C6173735F10104C6173745374617475734368616E67655C52656C6174697665506174685542697274685453697A655B496E6F64654E756D626572544D6F6465565573657249445F100F50726F74656374696F6E436C617373125FB3F04310001101F58003125FB50D8980021205022000120007C4C01181A410045F101C446F63756D656E74732F4368617453746F726167652E73716C697465D2252627285A24636C6173736E616D655824636C6173736573564D4246696C65A22729584E534F626A65637400080011001A00240029003200370049004C005100530058005E00770084008A0092009900AC00B900BF00C400D000D500DC00EE00F300F500F800FA00FF01010106010B010E0110012F0134013F0148014F01520000000000000201000000000000002A0000000000000000000000000000015B

The hex file is a plist. I exported both (the original empty db currently in whatsapp, as well as the uploaded converted database) as plist files and compared them:

https://tweakers.net/fotoalbum/image/cxlgddsW7wBTU88aId1yK8q8.png

It seems we need to change the folder in the blob as well, as well as perhaps modified dates and status change. Interestingly the size does not matter, when I swapped out the converted db of 40MB (which displays as 84.025.344 size?) with the empty one at 385K the size parameter still showed the bigger size. I suspect it comes down to the folder difference, the restore procedure might be moving items into place based on the plist and errors on the incorrect directory.

from watoi.

@CopyCat73 could you please post the output of the following command for me to prepare a sql script to swap files?

DB=Manifest.db; test -f $DB && sqlite3 $DB -header "select * from Files where relativePath like '%ChatStorage%';"

Or, if you're able to do it yourself, I've prepared updated hexes (changed relative paths).

After swap, update hexes to this:
For 7c7fba66680ef796b916b067077cc246adacf01d:

62706c6973743030d4010203040506070a582476657273696f6e592461726368697665725424746f7058246f626a6563747312000186a05f100f4e534b657965644172636869766572d1080954726f6f748001a40b0c232455246e756c6cdc0d0e0f101112131415161718191a1b1c1d191e1f2021221b5c4c6173744d6f64696669656455466c6167735747726f757049445624636c6173735f10104c6173745374617475734368616e67655542697274685c52656c6174697665506174685453697a655b496e6f64654e756d626572544d6f64655f100f50726f74656374696f6e436c61737356557365724944125fb3f04310001101f58003125fb50d8980021205022000120007c4c01181a410045f10124368617453746f726167652e73716c697465d2252627285a24636c6173736e616d655824636c6173736573564d4246696c65a2292a564d4246696c65584e534f626a65637400080011001a00240029003200370049004c005100530058005e00770084008a0092009900ac00b200bf00c400d000d500e700ee00f300f500f800fa00ff01010106010b010e01100125012a0135013e01450148014f0000000000000201000000000000002b00000000000000000000000000000158

For eb004981f01eb150a44ed3680d80726c8a74934e:

62706c6973743030d4010203040506070a582476657273696f6e592461726368697665725424746f7058246f626a6563747312000186a05f100f4e534b657965644172636869766572d1080954726f6f748001a40b0c242555246e756c6cdc0d0e0f101112131415161718191a1b1c1d1e1f202122231b5c4c6173744d6f64696669656455466c6167735747726f757049445624636c6173735f10104c6173745374617475734368616e67655542697274685c52656c6174697665506174685453697a655b496e6f64654e756d626572544d6f64655f100f50726f74656374696f6e436c61737356557365724944125fb3f5df10001101f58003125fb3d80b125fb3c1898002120005e000120001abc81181a410045f101c446f63756d656e74732f4368617453746f726167652e73716c697465d2262728295a24636c6173736e616d655824636c6173736573564d4246696c65a22a2b564d4246696c65584e534f626a65637400080011001a00240029003200370049004c005100530058005e00770084008a0092009900ac00b200bf00c400d000d500e700ee00f300f500f800fa00ff01040106010b011001130115013401390144014d01540157015e0000000000000201000000000000002c00000000000000000000000000000167

How I did it:

# decode hex to binary form (from clipboard)
pbpaste | xxd -r -p > dump.bin
# convert plist to xml form
plutil -convert xml1 -o dump.{xml,bin}
# edit it
cp dump{,_fixed}.xml
vi dump_fixed.xml
# convert back
plutil -convert binary1 -o dump_fixed.{bin,xml}
# put updated hex into clipboard
cat dump_fixed.bin | xxd -p -c 9999 | pbcopy

from watoi.

I swapped the files, edited the plist and put the edited version back in the WhatsApp file blob. But then restoring the backup keeps failing saying that the iPhone has disconnected. So there's something else wrong now that forcefully quits even the file copy.. Edit: iPhone disconnecting is because I used DB browser for sqlite to edit and save manifest.db. Apparently that messes up the database. Insert of the plist has to be done via command line then it works:

DB=Manifest.db; test -f $DB && sqlite3 $DB "UPDATE Files SET file = X'YOURHEXSTRINGHERE' where fileID == '7c7fba66680ef796b916b067077cc246adacf01d';"

from watoi.

@CopyCat73 that's great to hear!

from watoi.

We'll need a volunteer to check if it's possible to replace the database with just one backup (== without copying to the third party app). I'll prepare (hopefully) easy to use script.

from watoi.

If we can get the exact size of the new database, it is not needed to upload it to another app. But it needs to be the exact size as defined in the plist, otherwise you get the blinking apple logo again.

from watoi.

@CopyCat73 no problem here - it's a normal file size in bytes, just like displayed by regular tools:

> ls -l 7c/7c7fba66680ef796b916b067077cc246adacf01d
-rw-r--r--@ 1 user  staff  53026816 Jul 19  2019 7c/7c7fba66680ef796b916b067077cc246adacf01d
> cat chat.xml
--- cut ---
			<key>Size</key>
			<integer>53026816</integer>
--- cut ---

from watoi.

@residentsummer sorry for my late reply, but I see that @CopyCat73 was a way better responder! I'm currently trying the method proposed by @CopyCat73 but I could wait a bit so I could volunteer for your new script. Let me know. Great work!

from watoi.

@73567876543783 nice. I hope it took me not too long.

Here is the script: https://gist.github.com/residentsummer/ab437dea72346dce85e04fa5b5ff382f

Put it somewhere and make executable (chmod +x bedit.sh).

It has 3 commands (run without args to get usage hint):

• list-backups
• get-blob
• replace-blob

If you already have modified ChatStorage.sqlite, you'll only need the last command (it will both put the blob into backup and fix its size in the Manifest.db):

./bedit.sh replace-blob <backup id> 7c7fba66680ef796b916b067077cc246adacf01d <path to modified ChatStorage>

E.g. in my case it was

./bedit.sh replace-blob 3105fe2b1e4688d54920d5b7eff3a06a71fd5957 7c7fba66680ef796b916b067077cc246adacf01d ChatStorage.sqlite

from watoi.

Great, I’ll give it a try tomorrow. I could’ve done it manually but I hope that trying out the script helps you or other people.

The last command would suffice but I can try out the other too. Thanks!

from watoi.

I'll be testing the script either tonight or tomorrow as well! Just to clarify, this script doesn't need me to put the modified file into an app first right (step 6 in #29 (comment))? So my backup ID is 8811492706ad587a94bab7ae7c7727ca5ac2c270 so the command should just be:

./bedit.sh replace-blob 8811492706ad587a94bab7ae7c7727ca5ac2c270 7c7fba66680ef796b916b067077cc246adacf01d ~/Desktop/modifiedChatStorage.sqlite

Is this correct?

from watoi.

Just to clarify - with the script we're testing here, the only deviation from original instruction (README.md) is the second to last step (copying the modified blob). Instead of doing plain copy with cp, we'll be using replace-blob command from the new script.

from watoi.

All right, I just finished the restore using the new script. The files copied fine and the restore went at normal speed this time, no increasing time remaining or anything. Unfortunately, after the restore finished I again was greeted with the flashing apple logo upon startup.

After the recovery I'll try the manual method with the file transfer in iTunes etc.

If this works, can we conclude that the backup should remain the same size for any alterations to work?

from watoi.

@73567876543783 That's unexpected... Could you please post the output of commands bellow?

# In the root of the backup that was modified
ls -l 7c/7c7fba66680ef796b916b067077cc246adacf01d
DB=Manifest.db; test -f $DB && sqlite3 $DB 'select HEX(file) from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";'

from watoi.

@cool00geek did you tried to restore?

from watoi.

@CopyCat73 what is the iOS version on the device you've restored to? Well, could you please post iTunes version too, just to get the full picture.

As I see from the steps you've kindly provided, the only change that you've made to the hex of 7c7fba66680ef796b916b067077cc246adacf01d was the correction of file size, right? Could you please post the hex from the backup you've managed to successfully restore?

from watoi.

@73567876543783 That's unexpected... Could you please post the output of commands bellow?

# In the root of the backup that was modified
ls -l 7c/7c7fba66680ef796b916b067077cc246adacf01d
DB=Manifest.db; test -f $DB && sqlite3 $DB 'select HEX(file) from Files where fileID == "7c7fba66680ef796b916b067077cc246adacf01d";'

First command:
-rw-r--r--@ 1 username staff 102682624 Nov 20 19:23 7c/7c7fba66680ef796b916b067077cc246adacf01d

Second command:
3C3F786D6C2076657273696F6E3D22312E302220656E636F64696E673D225554462D38223F3E0A3C21444F435459504520706C697374205055424C494320222D2F2F4170706C652F2F44544420504C49535420312E302F2F454E222022687474703A2F2F7777772E6170706C652E636F6D2F445444732F50726F70657274794C6973742D312E302E647464223E0A3C706C6973742076657273696F6E3D22312E30223E0A3C646963743E0A093C6B65793E2461726368697665723C2F6B65793E0A093C737472696E673E4E534B6579656441726368697665723C2F737472696E673E0A093C6B65793E246F626A656374733C2F6B65793E0A093C61727261793E0A09093C737472696E673E246E756C6C3C2F737472696E673E0A09093C646963743E0A0909093C6B65793E24636C6173733C2F6B65793E0A0909093C646963743E0A090909093C6B65793E4346245549443C2F6B65793E0A090909093C696E74656765723E333C2F696E74656765723E0A0909093C2F646963743E0A0909093C6B65793E42697274683C2F6B65793E0A0909093C696E74656765723E313630353838373035383C2F696E74656765723E0A0909093C6B65793E466C6167733C2F6B65793E0A0909093C696E74656765723E303C2F696E74656765723E0A0909093C6B65793E47726F757049443C2F6B65793E0A0909093C696E74656765723E3530313C2F696E74656765723E0A0909093C6B65793E496E6F64654E756D6265723C2F6B65793E0A0909093C696E74656765723E3130353738343C2F696E74656765723E0A0909093C6B65793E4C6173744D6F6469666965643C2F6B65793E0A0909093C696E74656765723E313630353839353738383C2F696E74656765723E0A0909093C6B65793E4C6173745374617475734368616E67653C2F6B65793E0A0909093C696E74656765723E313630353839353738383C2F696E74656765723E0A0909093C6B65793E4D6F64653C2F6B65793E0A0909093C696E74656765723E33333138383C2F696E74656765723E0A0909093C6B65793E50726F74656374696F6E436C6173733C2F6B65793E0A0909093C696E74656765723E343C2F696E74656765723E0A0909093C6B65793E52656C6174697665506174683C2F6B65793E0A0909093C646963743E0A090909093C6B65793E4346245549443C2F6B65793E0A090909093C696E74656765723E323C2F696E74656765723E0A0909093C2F646963743E0A0909093C6B65793E53697A653C2F6B65793E0A0909093C696E74656765723E3130323638323632343C2F696E74656765723E0A0909093C6B65793E5573657249443C2F6B65793E0A0909093C696E74656765723E3530313C2F696E74656765723E0A09093C2F646963743E0A09093C737472696E673E4368617453746F726167652E73716C6974653C2F737472696E673E0A09093C646963743E0A0909093C6B65793E24636C61737365733C2F6B65793E0A0909093C61727261793E0A090909093C737472696E673E4D4246696C653C2F737472696E673E0A090909093C737472696E673E4E534F626A6563743C2F737472696E673E0A0909093C2F61727261793E0A0909093C6B65793E24636C6173736E616D653C2F6B65793E0A0909093C737472696E673E4D4246696C653C2F737472696E673E0A09093C2F646963743E0A093C2F61727261793E0A093C6B65793E24746F703C2F6B65793E0A093C646963743E0A09093C6B65793E726F6F743C2F6B65793E0A09093C646963743E0A0909093C6B65793E4346245549443C2F6B65793E0A0909093C696E74656765723E313C2F696E74656765723E0A09093C2F646963743E0A093C2F646963743E0A093C6B65793E2476657273696F6E3C2F6B65793E0A093C696E74656765723E3130303030303C2F696E74656765723E0A3C2F646963743E0A3C2F706C6973743E0A

from watoi.

@73567876543783 hmm, that blob is substantially longer than ones I've seen before. The data, as printed by plutil, seems correct though.

from watoi.

Ah, I see now. The hex that is produced by my util is wrong (it's in xml format, rather than binary). Please stand by for a fix.

from watoi.

@73567876543783 fixed. Please re-download the script from the same link, I've pushed a new revision.

PlistBuddy (one of the system utils that can edit plists) silently converted binary to xml form, which went unnoticed by me. I haven't found an option to preserve it, so had to use the second tool, plutil, to convert back. When writing the initial script I've tried plutil first, but was unable to persuade it to make required changes. What a mess.

from watoi.

Does this discovery about plist files bring #1 further?

from watoi.

I used bedit.sh replace-blob instead of step 6 of the Readme.md and still got the flashing apple logo on a new iPhone 12 mini running 14.2.1. So it is not a fool-proof process just yet.

from watoi.

@hugo-lyppens I've updated README to reflect the latest successful migration path from this thread (note that bedit.sh has changed, please use one from the repo). Could you please try those steps?

from watoi.

@HimbeersaftLP not much, just the path became a little bit clearer. Someone still have to implement this.

from watoi.

@hugo-lyppens did you had a chance to complete the process from README?

from watoi.

I have followed the steps in the README for an iPhone 12 pro with iOS 14.2.1 with Big Sur latest. I am still getting the flashing logo after restore. This is my first attempt.

I am using the Whatsapp IPA from https://www.iphonecake.com/app_310633997_.html with version 2.20.131

from watoi.

I want to try again. But I have a concern - if I get apple logo flash and I have to restore , will my ESIM profile be deleted?

from watoi.

can someone guide me with steps on ios 14.3 and MacOS Big Sur 11.1 ?

from watoi.

@stephunk Could you please share the ipa you managed to download? I can't download it from iPhoneCake, I just end up in a loop with ads. My WhatsApp version is 2.20.132.

from watoi.

had the same issue as @stephunk, and their steps worked for me as well. running ios 14.3 and whatsapp 2.20.132.

@juhogalle the ipa from Cracker/Uploader: telefonbuch worked for me, while the one from Cracker/Uploader: user_hidden-ICPDA caused a bootloop

Update to my previous comment.
I was able to restore the chat history after the second attempt. (I did no extra steps but the ones mentioned in the README)
Downloaded a different ipa from the same link mentioned above. Also, I have made the fix mentioned in #31 even though the script was not crashing in my case.

I have one issues now:

* Starred messages do not come up as starred after transfer. Most of the messages seem to be there but the ones marked as starred before are not showing up like that anymore.

from watoi.

Lots of thanks @RahulSriram! I managed to do the transfer with the correct ipa (telefonbuch's version of WhatsApp 2.20.132). I didn't make the fix that @stephunk mentioned.

Only issue that I have is that all restored chats were archived, although that's minor. I would've hoped that placeholders for images () were preview thumbnails. I tried a software called Backuptrans (trial version) and it places thumbnails from msgstore.db instead of just placeholder.

from watoi.

After moving through all steps successfully.
I did last step of restore backup from iTunes. I see message Restore in progress ans in few seconds, phone went off and Apple logo is flashing. I had to go recovery mode and restore iphone :/ .

How did you restore it? I am stuck in the bootloop and the phone is not recognized by my PC and wont enter recovery mode

from watoi.

After moving through all steps successfully.
I did last step of restore backup from iTunes. I see message Restore in progress ans in few seconds, phone went off and Apple logo is flashing. I had to go recovery mode and restore iphone :/ .

How did you restore it? I am stuck in the bootloop and the phone is not recognized by my PC and wont enter recovery mode

I went into DFU mode to restore. Did you try that already? That should be possible.

from watoi.

Cleanup

from watoi.