gandag / pyfomod Goto Github PK

View Code? Open in Web Editor NEW

7.0 3.0 6.0 356 KB

A high-level fomod library written in Python.

Home Page: https://pyfomod.rtfd.io/

License: Apache License 2.0

Python 100.00%

fomod installer parser

pyfomod's People

Contributors

Stargazers

Watchers

Forkers

beermotorwb lahvuun leontristain krateng holt59 bbergeron0

pyfomod's Issues

CDATA in xml tags are not parsed

Not a blocker for me, just thought the issue ought to be logged. Thanks!

Steps to repro:

Download the NMM version of Skyrim Special Edition mod "Insects Begone" (https://www.nexusmods.com/skyrimspecialedition/mods/1190?tab=files). This version comes with a FOMOD installer.
Extract to a folder (let's say it's accessible at src/insects-begone)
Observe the following behavior in python interpreter (using python 3.7.0, not sure if relevant):

>>> import pyfomod
>>> pyfomod.__version__
'0.8.1'
>>> root = pyfomod.parse('src/insects-begone')
>>> root.pages[0][0][0].name
'A reminder'
>>> root.pages[0][0][0].description
''
>>>

Check the FOMOD xml to see this section:

<plugin name="A reminder">
        <description>
        <![CDATA[Read the descriptions for each of the files, they explain completely what each one does so you know exactly what you are getting. Don't rush through this installation if you want it to work completely.]]>
        </description>
        <files>
        </files>
        <typeDescriptor><type name="Optional"/></typeDescriptor>
</plugin>

Allow for case-sensitive systems to parse from a folder

Hi! Stumbled upon this package when i was scouting for some inspiration on parsers for fomod.

When attempting to parse a package with the first option, pyfomod assumes that the filename is moduleconfig.xml, which works fine in a case-insensitive file-system. However, when doing the same in a case-sensitive file-system (such as a file-system often used in linux), this fails.

Example

>>> x = pyfomod.parse(".")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/{REDACTED}/.local/lib/python3.10/site-packages/pyfomod/parser.py", line 272, in parse
    raise FileNotFoundError(
FileNotFoundError: [Errno 2] No such file or directory: 'moduleconfig.xml'
>>> x = pyfomod.parse(("./fomod/info.xml", "./fomod/ModuleConfig.xml"))
>>> x
<pyfomod.fomod.Root object at 0x7f12de46f880>
>>>

Improve testing

Imrpove testing by changing the old tests to use mock/proper setups instead of downoading data from fomod-lang repo.

Validate Error when no file is given in plugin section

<plugin name="None"> <description> Uses vanilla ragdolls </description> <image path="fomod\1.jpg"/> <typeDescriptor> <type name="Optional"/> </typeDescriptor> </plugin>
This is invalid because after the description, it expects either a sequence with 'files'+'conditionFlags' or a sequence with 'conditionFlags'+ 'files'. I don't even know if you're still active but if you see this let me know and maybe post an idea. My work around: creating a new tree and inserting the files section with no content.

Installer cannot return multiple directory paths with same destination

This happens on multiple mods, but just as an example, I have the fomod Bijin Wives. The ModuleConfig.xml file is properly parsed to install multiple paths for one option:

mod = pyfomod.parse((None,"./ModuleConfig.xml"))

page1 = mod.pages[0]
group1 = page1[0]
option1 = group1[0]
print(option1.name)
# All IN ONE
print(option1.files)
# <pyfomod.fomod.Files object at 0x7f2d77c68b80>
print(list(option1.files))
# ['02 ESPs\\AIO/', '00 Common/', '01 Characters\\Camilla/', '01 Characters\\Grelka/', '01 Characters\\Morwen/', '01 Characters\\Muiri/', '01 Characters\\Senna/', '01 Characters\\Sylgja/', '01 Characters\\Taarie/', '01 Characters\\Temba/', '01 Characters\\Ysolda/']

However, when I actually select that option with the installer, it only remembers the last path:

mod = pyfomod.parse((None,"./ModuleConfig.xml"))
installer = pyfomod.Installer(mod)

page1 = installer.next()
group1 = page1[0]
option1 = group1[0]
print(option1.name)
# All IN ONE
installer.next([option1])
print(installer.files())
# {'01 Characters\\Ysolda': '.'}

From multiple test cases, this seems to be always the last file added - and not just per page or group, but over the entire installer.

Reinforce parser recovery

The parser should be able to recover from all but the most egregious of syntax errors. Currently it fails on some schema errors (missing required attributes, invalid attribute values, see GandaG/fomod-validator#8).

Cannot install mods due to default value mismatch for flag conditions that have never been set

Hi GandaG!,

I believe I found a bug that prevents the installation of some mods due to different expectations on default value matches.

Please see the latest version of The Men of Winter for Skyrim SE, the first of two main files, labeled "The Men of Winter SSE", version V5.3, uploaded date 08 May 2020 8:58PM. (nexus file id 138832, size 128788 in kb).

In its ModuleConfig.xml, there are options that would set the flag MiraakOption to "On". These are not required options and the user does not have to choose them. These are the only places in the xml that attempt to set this flag to anything.

The final page of the installer is implemented with the last two <installStep>s, each conditionalized on MiraakOption being either "On" or "". If the user did not choose the optional option that results in MiraakOption being "On", the xml seems to expect that MiraakOption == "" would resolve to True.

With the current pyfomod, a flag that has never been explicitly set via an option would simply be missing in the .flags() list, and a missing flag will cause a flag condition to resolve to False even when compared to "". This means if the user never chose the MiraakOption = "On" option, the final page (which contains a lot of default files) will simply be skipped, resulting in an empty install.

In contrast, the latest release of MO2 will install this correctly. I don't know what's the true correct behavior for fomods, but I feel like anything MO2 supports that pyfomod differ would not be desirable.

Thanks,
leontristain

Implement element reordering

Add methods to FomodElement allowing same tag nodes to reorder.

Incorrect Exception on type mismatch

Currently type argument mismatches (user providing an integer where a string is expected) raises a ValueError - this should be a TypeError.

TypeError when passing a string to path parameter on Installer

Installer.files() returns a dictionary of of destination/source key-pairs, this makes it impossible to have multiple options copy files to the same directory seeing as there can only be one key-pair per destination.

A fix could be using a list of tuples instead.

Expose requiredInstallFiles

Reading in CBBE the info.xml clearly indicates a requiredInstallFiles folder

But the final files() call to installer doesn't return "00 Required (Slim)" as one of the to-be-installed folders.

Interestingly it is in installer.files() before any options are chosen -- is this how one knows it is in "requiredInstallFiles"? Perhaps the root object / installer should expose required_files too?

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://qconsulting.ca/fo3/ModConfig5.0.xsd">
        <moduleName>Caliente&apos;s Beautiful Bodies Enhancer -CBBE-</moduleName>
        <moduleImage path="FOMod\CBBE.png"/>
        <requiredInstallFiles>
                <folder source="00 Required (Slim)" destination="" />
        </requiredInstallFiles>
        <installSteps order="Explicit">
                <installStep name="Base Options">
                        <optionalFileGroups order="Explicit">
                                <group name="Body Shape" type="SelectExactlyOne">
                                        <plugins order="Explicit">
                                                <plugin name="Slim">
                                                        <description>
                                                                <![CDATA[Choose this option if you want to use the slim body shape. Affects the other options as well.]]>
                                                        </description>
                                                        <image path="FOMod\Slim.jpg"/>
                                                        <conditionFlags>
                                                                <flag name="bSlim">On</flag>
                                                        </conditionFlags>
                                                        <files>
                                                                <folder source="== Installer ==" destination="" priority="0"/>
                                                        </files>
                                                        <typeDescriptor>
                                                                <type name="Optional"/>
                                                        </typeDescriptor>
                                                </plugin>
                                                <plugin name="Curvy">
                                                        <description>
                                                                <![CDATA[Choose this option if you want to use the curvy default shape. Affects the other options as well.]]>
                                                        </description>
                                                        <image path="FOMod\Curvy.jpg"/>
                                                        <conditionFlags>
                                                                <flag name="bCurvy">On</flag>
                                                        </conditionFlags>
                                                        <files>
                                                                <folder source="01 Curvy" destination="" priority="0"/>
                                                        </files>
                                                        <typeDescriptor>
                                                                <type name="Optional"/>
                                                        </typeDescriptor>
                                                </plugin>
                                                <plugin name="Vanilla Shape">
                                                        <description>
                                                                <![CDATA[The CBBE body, shaped to conform closely to the proportions of the vanilla female body - with a few adjustments. Affects the other options as well.]]>
                                                        </description>
                                                        <image path="FOMod\Vanilla.jpg"/>
                                                        <conditionFlags>
                                                                <flag name="bVanilla">On</flag>
                                                        </conditionFlags>
                                                        <files>
                                                                <folder source="02 Vanilla" destination="" priority="0"/>
                                                        </files>
                                                        <typeDescriptor>
                                                                <type name="Optional"/>
                                                        </typeDescriptor>
                                                </plugin>
                                        </plugins>
                                </group>
[...]

Source www.nexusmods.com/skyrim/mods/2666?tab=files&file_id=1000170527

Empty destination paths are not preserved in `to_string()` for Files objects

Steps to repro:

Download the NMM version of Skyrim Special Edition mod "Insects Begone" (https://www.nexusmods.com/skyrimspecialedition/mods/1190?tab=files). This version comes with a FOMOD installer.
Extract to a folder (let's say it's accessible at src/insects-begone)
Observe the following behavior in python interpreter (using python 3.7.0, probably irrelevant):

>>> import pyfomod
>>> pyfomod.__version__
'0.8.1'
>>> root = pyfomod.parse('src/insects-begone')
>>> print(root.pages[1][1][0].files.to_string())
<files>
  <folder source="Mesh Replacers\Webs" priority="2"/>
</files>
>>>

The actual tag in the xml says:

<folder source="Mesh Replacers\Webs" destination="" priority="2"/>

I believe this is relevant as a matter of correctness because an empty string for destination implies the root of the folder, while if destination is not provided it means it should be the same value as the source. Let me know if my understanding is incorrect.

BTW, I fully understand that to_string() according to the docs is not generally recommended and that API isn't stable, I just encountered this because I'm using it anyway in order to parse out the priority information which is currently not supported by the Files object. There's no pressure, just thought to mention it and keep the issue logged. 😃