Files outputting with no extension and 0 kb size about snapchatmemoriescaptionadder HOT 9 CLOSED

Was able jump away and to my laptop for a bit. Good news is I think I solved the issue of outputted files being NULL files. The script still hangs up with videos, but that's less of a concern from now (I've still posted the logger info below when running only one MP4 through the memories_history.json file, if it's helpful.

I merged the following changes into my fork:

1. Added rename_files.py. Took the lazy way of getting rid of the new prefixed date format by just renaming all of the files without the prefix instead of changing adder.py
2. The reason the files outputted were all NULL is because line 65 of adder.py includes a colon in the outputted file name (for HH:MM). Not sure if colons are acceptable on MacOS, but this was causing an issue on Windows
3. With those two changes, I was able to at least get an overlayed output for one test picture. I'm ging to kick off the process for all of my pictures soon

As far as videos, here is the logger output when it simply hangs up. Maybe it's a timing thing and I need to just let it sit for a while? Not sure. That said, it does output an MP4 file in my output folder. This file is just corrupted though and does not open.

DEBUG:root:Args(memories_history=WindowsPath('input/memories_history.json'), memories_folder=WindowsPath('input/memories'), output_folder=WindowsPath('output'))
Handling images...
0it [00:00, ?it/s]
Handling videos... (this will be slower than the pictures and will have hitches!)
  0%|                                                                                                                | 0/1 [00:00<?, ?it/s]DEBUG:root:base image found: input\memories\0C01B92D-46B8-4B74-B169-64E8E1C23530-main.mp4
DEBUG:root:Overlay: input\memories\0C01B92D-46B8-4B74-B169-64E8E1C23530-overlay.png
DEBUG:root:output file: output\2020-11-070C01B92D-46B8-4B74-B169-64E8E1C23530.mp4
100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s] 
Waiting for final videos...
  0%|

from snapchatmemoriescaptionadder.

The issue wasn't fixed... I the new file name didn't work for me... i needed to make a little fix in the code adder.py:
71 overlay := root.with_name(base.name[:-9] + "-overlay").with_suffix(".png")
i fixed it this way...

The problem was, I had the captions but they were not recognized and I had a Caption None

from snapchatmemoriescaptionadder.

Ah yep, if they've changed the file names that would explain your first issue. If you look in the test folder in this repo, you can see how my memories download looked - no dates in the file name, just that big blob as a name. The script assumes it can always use that blob to get an image's name, so I'd hope that your issue in 2 is that somewhere the script is still assuming it can find the image with the blob name, actually grabs nothing, tries to attach a nothing caption to a nothing image, and then returns... nothing 🤷

Couple things we can do:

• if you open main.py and uncomment line 17 (# logging.basicConfig(level=logging.DEBUG)), you'll get more logs. That may help you in seeing what the script is doing, would definitely help me. I'd recommend only running this with like one image's data in the memories_history.json file, or else you're going to get a lot of output.

• I can make another memories request to see the new export format by Snapchat, but that may take a little bit to get back. To save some time, could you pick a random file and share its file name (in my old data an example is 062c8942-3124-a480-71fc-3c4833e3e569-main.jpg), the file name for its caption (in my old data it's 62c8942-3124-a480-71fc-3c4833e3e569-overlay.png), and the corresponding data for it in memories.json (by running grep "04a5f170-2cf4-9196-f85d-3e3479af674f" -C 5 memories_history.json I'm able to get that snippet, or just open in a text editor and ctrl-f)? That way I can see all the changes to the format just to double check everything and adapt accordingly.

• Could you push your changes to your fork? I have some ideas on how to tackle this (probably by changing add_metadata in SnapchatMemoriesMetadataAdder/adder.py to change the base and output names) but it'd be nice to avoid needlessly doing work you've already done :)

I'd be optimistic that once the script is adapted to the new file format it should work.

from snapchatmemoriescaptionadder.

Ah, you are absolutely amazing. I’m away from my laptop for a bit due to the holidays, so I’ll get to this as soon as I can - also, no rush at all on your end once I get back to you :)

Do you happen to have a support page by chance? Gotta support folks like you publishing your code

from snapchatmemoriescaptionadder.

Good to know that : causes issues on Windows and good job on finding/fixing it! I've only ran this on Linux so thanks for discovering this, would've totally eluded me.

Not confident on why the videos are hanging - it wasn't taking all that long on my laptop (a few seconds per video?), so it's probably something I'm doing incorrectly with ffmpeg. Easiest way to analyze this is to get ffmpeg's logs - in SnapchatMemoriesMetadataAdder/_ffmpeg.py change line 31 to have ...run_async() instead of ...run_async(quiet=True). Then when you run the script, ffmpeg will output what it's doing which should let us know if it got stuck or had an error or it's just taking forever. (Will definitely be a lot of output, so stick to running it with one video!)

Happy holidays! Same to you about no rush to respond :) I don't have a support page but it's no need, just happy this script is actually being useful to someone else :)

from snapchatmemoriescaptionadder.

This is great, the fork does indeed work but seems to hang on videos.
It seems that some of the files from the Snapchat export are corrupted, making ffmpeg hang on these files with this error.

[mov,mp4,m4a,3gp,3g2,mj2 @ 00000193fda47740] moov atom not found [in#0 @ 00000193fda4a1c0] Error opening input: Invalid data found when processing input drop=2 speed=0.38x Error opening input file input\memories\cd99332c-c184-4fd3-bb56-fe8afd14d394-main.mp4. Error opening input files: Invalid data found when processing input 0.1kbits/s speed=1.26x

Removing these video files does resolve the issue, I do not have the coding capacity to make ffmpeg ignore these corrupt files. I did check that it's the export and not something any of the scripts do, as these files are also corrupted in the raw backup I made.

from snapchatmemoriescaptionadder.

Happy New Year! I implemented fixes for creating valid file names on windows and for finding the new type of Snapchat export names, so the next person to use this shouldn't run into these issues. Thanks for reporting, explaining, and pushing your fork to help out with this!

Really sucks about your corrupted videos - maybe a new export from Snapchat would have non-corrupted files?

Thank you again!

from snapchatmemoriescaptionadder.

Getting back late here, but thanks for taking care of those issues. I made the change on my end for the video logger to output to my terminal, and there was never any error message on my end - once I’d fixed the : naming convention, all of my media processed properly, but the script just never finished executing (which is what was causing the hanging earlier, combined with the naming convention which is why I just couldn’t see any of my output)

Bottom line, everything works as expected on my end, but I just needed to confirm all of my media was processed properly.

One suggestion, I’d found another program that also added location metadata to media. Unfortunately, it didn’t convert UTC to local time nor did it handle captions, and it was an executable so not something I could edit. I just used the best of both your script and their program, and now I can finally delete Snapchat. Thanks for everything! Happy New Year :)

from snapchatmemoriescaptionadder.

@Enricone27 Thank you for testing and fixing this! I totally forgot to handle overlays with the new file name :) I just added a commit to fix this bug, thank you!

from snapchatmemoriescaptionadder.