Comments (17)
I expected already i miss something or that it is needed also to cater to py2.
I wrote magic/picklemagic as a lib that supported both py2/3. To prevent them from diverging and making my life more painful it's easier to just keep them in sync.
The other fix is just doing the right thing regarding the protocol. Py2 only goes up to pickle protocol 2, but in py3 a bunch of things have been added. And ren'py might at one point just up the protocol so it's good to just keep up to date.
I think we can close this then in a few...
Yeah I'll close it after actually committing the code.
from unrpyc.
That'd indeed be a nice way of handling it. Regarding TODO's before releases, for stuff that doesn't break any previous behaviour I wouldn't worry too much. I've in fact been working to just a new release today to get the new compatibility rules out of the door. With the new testing framework preparing releases seems to be even easier as there's a lot less risk of un.rpyc bugs (I still tested it with actual ren'py for the record).
I'll just do that now. No worries about it, when dev's finished on other action items we can just make another one.
from unrpyc.
With run 1) we get the translation file with a short string, but its binary/unknown encoding gibberish.
You get a file just containing a pickle, so I'm not particularly surprised by that.
I think how the feature works is that it allows you to decompile a game that already has translations, in a different language.
When you first run it with -T
it'll gather the contents of any TranslateString
and Translate
nodes in the specified language and save it in the specified file.
Then when you run it again with -t, it'll decompile the thing, using the stored translations to change the unrpyc output to be in the wanted language.
from unrpyc.
Ok. I figured this basically.
with -T it'll gather the contents of any
TranslateString
andTranslate
nodes in the specified language and save it in the specified file.
But as can be seen in the attached files, from use of option -T, there are not much in there in terms of strings. I printed the return values of unrpyc.py#L172 out and they where basically empty.
terminal output
olli@tty2*blue $ python unrpyc.py -T /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game//tl/tl.txt -l german /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/ -c
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/neus/neus_room_quest.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/level3.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/milf_rooms/milf_rooms_events.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/two_bodies_rooms/two_bodies_mc_room_quest.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/mc/mc_room_quest.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/level1.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/bathroom/bath_room_quest.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/extra.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/image.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/two_bodies_rooms/two_bodies_rooms.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/nym_rooms/nym_rooms.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/gui.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/kitchen/kitchen_spell.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/audio.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/gamescreens.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/neus/neus_spell.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/tl/None/common.rpymc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/living/living_spell.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/script.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/attic/attic_room.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/tree_stats_quests/tree_skill_stats_quests_screens.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/photo.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/mini_game/fifteen_game.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/init_object.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/mini_game/spanking.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/mini_game/spanking_image.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/style.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/mini_game/puzzle.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/rooms.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/level0.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/kinetic_text_tags.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/bathroom/bath_hallway.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/variables.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/options.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/neus/neus_hallway.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/introduction.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/fx.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/save_compatibility.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/tree_stats_quests/skill.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/milf_rooms/milf_rooms.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/gallery/gallery.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/icons.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/glitch_ren.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/inventory/inventory.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/tree_stats_quests/quests.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/bathroom/bath_spell.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/ctc.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/util.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/DynamicAnimation.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/tree.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/room.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/glitch_tag.rpyc...
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/character.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/inventory.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/class/quest.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/ui_game/notify_personalized.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/endings.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/story/level2.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/living/living_room_quest.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/screens.rpyc...
extract_translations: .dialogue: {} .strings: {}
Extracting translations from /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/scripts/room/kitchen/kitchen_room_quest.rpyc...
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
extract_translations: .dialogue: {} .strings: {}
Writing translations to /home/olli/.xlib/RPG/_test/MWNeus-0.9-pc/game/tl/tl.txt...
Decompilation of 61 script files successful
(i put in there just
print(f"extract_translations: .dialogue: {translator.dialogue} .strings: {translator.strings}")
)
from unrpyc.
Huh. Does this game have translate nodes in it?
from unrpyc.
So, your question made me think in a different direction about this whole translation feature and i realized i misunderstood it.
I thought you need to give it with opt -l language
a name which isn't already in .../tl/
present to get all the strings from the files which you can translate then. I did this every time in my tests with 7 different games.
However it's reverse. You must choose a language which is already present in tl
. In the second run it puts these languages strings in the rpy files. So if i am right this time, this feature just switches the basic language of the game with another. But why do such thing? I can just run the game and switch the l10n in the menu. Makes no sense to me.
To my excuse i oriented myself in my expectation how it works on Ren'Py and his tl feature. Also, this here comes without a manual. 😁
Anyway with one game in py3 i have this error:
olli@tty2*blue $ python /home/olli/Code/rpy_tools/unrpyc/unrpyc-2.0.0/unrpyc.py -T /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/game/tl/en_tl.txt /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/game/ -c
...
Error while decompiling /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/game/tl/english/scripts/00parts/01arrival/01script/script.rpyc:
Traceback (most recent call last):
File "/home/olli/Code/rpy_tools/unrpyc/unrpyc-2.0.0/unrpyc.py", line 253, in worker
return extract_translations(filename, args.language)
File "/home/olli/Code/rpy_tools/unrpyc/unrpyc-2.0.0/unrpyc.py", line 215, in extract_translations
return magic.safe_dumps(translator.dialogue), translator.strings
File "/home/olli/Code/rpy_tools/unrpyc/unrpyc-2.0.0/decompiler/magic.py", line 612, in safe_dumps
SafePickler(file, protocol).dump(obj)
File "/usr/lib/python3.10/pickle.py", line 487, in dump
self.save(obj)
File "/usr/lib/python3.10/pickle.py", line 560, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python3.10/pickle.py", line 972, in save_dict
self._batch_setitems(obj.items())
File "/usr/lib/python3.10/pickle.py", line 998, in _batch_setitems
save(v)
File "/usr/lib/python3.10/pickle.py", line 560, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python3.10/pickle.py", line 932, in save_list
self._batch_appends(obj)
File "/usr/lib/python3.10/pickle.py", line 959, in _batch_appends
save(tmp[0])
File "/usr/lib/python3.10/pickle.py", line 603, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib/python3.10/pickle.py", line 687, in save_reduce
save(cls)
File "/usr/lib/python3.10/pickle.py", line 572, in save
self.save_global(obj)
File "/home/olli/Code/rpy_tools/unrpyc/unrpyc-2.0.0/decompiler/magic.py", line 525, in save_global
self.write(pickle.GLOBAL + obj.__module__ + '\n' + obj.__name__ + '\n')
TypeError: can't concat str to bytes
Writing translations to /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/game/tl/en_tl.txt...
Decompilation of 468 files successful, but decompilation of 26 files failed
I think this should be a py2->3 residue.
script.zip
from unrpyc.
in the rpy files. So if i am right this time, this feature just switches the basic language of the game with another. But why do such thing? I can just run the game and switch the l10n in the menu. Makes no sense to me.
Yep, that's what it does, that's what I meant in the last post. I believe Jackmcbarn's intention was so you can read through the decompiled code in a different language, preferably the one you can read.
Ah, seems like magic.py messes up generating the save_global code. pickle.GLOBAL is a bytes object, and everything there should be getting properly encoded. Easy fix.
from unrpyc.
Yeah. I still needed to find pickle.GLOBAL
to look what the hell this is. Not in the py docs... how nice.
self.write(pickle.GLOBAL + obj.module + '\n' + obj.name + '\n'
needs to be changed to
self.write(pickle.GLOBAL
+ bytes(obj.__module__, 'utf-8') + b'\n'
+ bytes(obj.__name__, 'utf-8') + b'\n')
and it works from the first look. If in the outfile everything right is do i not know.
from unrpyc.
Yeah. I still needed to find pickle.GLOBAL to look what the hell this is. Not in the py docs... how nice.
The internal format of the pickle protocol is only documented in the source code ;). Luckily the pickle docs page links to that at the top. I'm very familiar with it due to all the shenanigans I've done with it over the years.
Yup. I came up with the complete:
def save_global(self, obj, name=None, pack=struct.pack):
if isinstance(obj, FakeClassType):
if PY2:
self.write(pickle.GLOBAL + obj.__module__ + '\n' + obj.__name__ + '\n')
elif self.proto >= 4:
self.save(obj.__module__)
self.save(obj.__name__)
self.write(STACK_GLOBAL)
else:
self.write(pickle.GLOBAL + (
obj.__module__ + '\n' + obj.__name__ + '\n').decode("utf-8")
)
self.memoize(obj)
return
(magic.py is part of the picklemagic library used by un.rpyc, or at least, I wrote it at first as a separate library. So I'll make the change there as well).
from unrpyc.
(laughs)
I expected already i miss something or that it is needed also to cater to py2.
so you can read through the decompiled code in a different language, preferably the one you can read.
Yeah, i misunderstood this really. Seems not often used by people, but at least it makes sense now. If nothing else at least i found another bug to fix for you. 😛
I think we can close this then in a few...
from unrpyc.
it's easier to just keep them in sync.
This came to mind as i noticed you let the py2 code in it even in py3 branch. Makes sense from this POV.
from unrpyc.
Not this important imho, for a bad weather day, but problems should be always noted. I add this here instead of making another issue and adjusted for it the title.
There are multiple problems with the cli args for the TL feat in itself and in combination with other decompiling cli args:
-l language
arg without the needed-T
arg (or any other!) is silently without effect--try-harder
in combination with TL args is part-silently without effect. Only by failed decompile output noticeable.-T tranlation-file
and-t tranlation-file
can be given at the same time and-t
is silently without effect.
Not this hard to fix with some condition checks after the parser and/or argparse itself has some very useful features for stuff like this, like subparser, mutualy-exclusive.
What i don't understand is, why J.McBarn made a second run with -t if its anyway mandatory? Why not run this automatically after the TL-file from -T is written?
from unrpyc.
What i don't understand is, why J.McBarn made a second run with -t if its anyway mandatory? Why not run this automatically after the TL-file from -T is written?
I'd advice not to look for logic where there likely is none. It probably made it easier to implement as the tool is currently structured for a single pass. Doing a double pass for translations (and not having to save them to a file) would indeed be significantly cleaner. Then it could also only use one command line flag.
from unrpyc.
I'd advice not to look for logic where there likely is none.
Yeah, that's right. I tend to search in everything the sense and logic.
Then it could also only use one command line flag.
Exactly my thought! I believe whatever we ever write in a possible doc for this, will still confuse people.
Also, to hijack this issue just because its related to translation was a fault. There are other collisions in the cli args... well, something for another time.
from unrpyc.
Just a update:
I looked at this a bit and think the TL feat can be relatively less invasive changed. This will then not write the TL file out after step 1 and goes direct to step 2 to switch the main language.
Means, it uses then just one argparse option instead of three and some guards for bad argparse arg combinations would be unneeded.
--translate {language}
This would be the new arg without a default language(english is mostly the None lang), so users need to look the available up in the TL directory instead running this with default and wondering why nothing happens like now.
So far are my view. As said, this feature is IMO not often used and not really broken, so its at the end of my TODO(read: what-i-like-to-get-done-before-next-release).
from unrpyc.
Related Issues (20)
- Legacy-backports HOT 4
- Dev_py3: Possible missing colons in decompiled v8.2 files HOT 5
- Check for manipulated header fails sometimes HOT 7
- AttributeError: 'RawBlock' object has no attribute 'statements' HOT 12
- AttributeError: 'module' object has no attribute '_loads' with Ren'Py 6.99.12.4.2187 HOT 5
- Instaliation issues HOT 6
- Sugar syntax error in the SL2 decompiler HOT 7
- Interleaved output when multiprocessing is used. HOT 2
- Python string RegEx does not match multiline strings HOT 3
- Failed decompile script 'options.rpyс' from renpy 6.11.2 HOT 11
- How to unrpyc a single file during runtime in a rpy file HOT 13
- Missing guards after argparse for bad arg combinations HOT 1
- Safepickler in Py3 possibly broken HOT 10
- Uncaught exception while running a game with un.rpy HOT 10
- Sometimes "elif True" if generated HOT 3
- ModuleNotFoundError: No module named 'renpy' HOT 4
- After decompiled .rpy files, the game crashed. HOT 10
- Module Not Found Error HOT 4
- Cannot import as a library; fails at importing 'renpy' HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unrpyc.