Saccharose.wiki is a web app that provides various tools to auto-generate wikitext for use by the editors of the Genshin Impact Fandom Wiki
- Git
- Node.js
- Python 3
- PIP package:
pycld2
-
On Linux, run:
pip install pycld2
-
On Windows, pip installing does not work!
Instead, clone the repo,
cd
into it, and run./setup.py install
as admin.
-
This is only necessary to be installed if you plan to use or develop the "media-search" functionality.
- PostgreSQL (must be installed in a Linux system)
- On Windows, install in WSL: https://learn.microsoft.com/en-us/windows/wsl/install-manual
- Install PostgreSQL https://www.postgresql.org/download/linux/ubuntu/
- Modify
postgresql.conf
and setlisten_addresses = '*'
- In WSL, install apt packages:
make
,build-essential
(Linux should already have these)
- Other PIP packages:
imagehash
,psycopg2
,python-dotenv
- On Linux, having PostgreSQL installed is required to install
psycopg2
- On Linux, you'll need to apt-get install
libpq-dev
andpython3-dev
first to install psycopg2. - On Windows, you can install these in Windows, not WSL
- On Linux, having PostgreSQL installed is required to install
- Install
pg-spgist_hamming
on the system with PostgreSQL installed (https://github.com/fake-name/pg-spgist_hamming)- First run:
sudo apt install postgresql-server-dev-XX
- Replace
XX
with your installed version of PostgreSQL
- Replace
- Follow instructions:
cd bktree make sudo make install
- First run:
-
Clone the repo from GitHub (
git clone [email protected]:kwwxis/Saccharose.git
) -
Run
npm install
in the repo -
If on Windows, run this command to configure Bash as the script shell:
- 32bit:
npm config set script-shell "C:\\Program Files (x86)\\git\\bin\\bash.exe"
- 64bit:
npm config set script-shell "C:\\Program Files\\git\\bin\\bash.exe"
- 32bit:
-
Copy
.env.example
to new file.env
and configure it according to the comments and this readme.
Bash is required for this program to run in all operating systems, specifically it
uses the grep
command.
So you must specify a SHELL_PATH
and SHELL_EXEC
in the .env
file.
If you're on Linux/Unix, you should already have Bash. But on Windows, it's recommended that you install Git Bash.
After installing Git Bash, you can configure the follow:
- On Windows:
SHELL_PATH='/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin' SHELL_EXEC='C:/Program Files/Git/usr/bin/bash.exe'
- On Linux:
SHELL_PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin SHELL_EXEC=/bin/bash
This project can be run both with and without SSL. For local development, it doesn't really matter much, so if you're having trouble getting SSL working or don't want to bother, then you can just skip it.
-
Create a file called
openssl.<WEB_DOMAIN>.cnf
with these contents.Replace
<WEB_DOMAIN>
in the file name and the file contents with theWEB_DOMAIN
in your.env
file.authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment subjectAltName=DNS:<WEB_DOMAIN>
-
Run
openssl genrsa -des3 -out rootSSL.key 2048
. It'll ask you to create a password, only you're using this password so it doesn't need to be a very good password but make sure you remember it as you'll be asked to re-enter this password in later commands. -
Run
openssl req -x509 -new -nodes -key rootSSL.key -sha256 -days 1024 -out rootSSL.pem
. It'll give you a questionnaire, fill it out appropriately. For the "Common Name" question, give it a decent name so you know what it's for (e.g. "Self-Signed Local Certificate").The rootSLL.pem will expire after 1024 days, so you'll need to repeat this process when it expires. Alternatively, you can set
-days 1024
to a higher number when you run the command. -
Run this command. It's multi-line so if your terminal asks if you're sure you want to paste it, then say yes.
Remember to replace
<WEB_DOMAIN>
openssl req \ -new -sha256 -nodes \ -out <WEB_DOMAIN>.csr \ -newkey rsa:2048 -keyout <WEB_DOMAIN>.key \ -subj "//C=<2LetterCountryCode>\ST=<StateFullName>\L=<CityFullName>\O=<OrganizationName>\OU=<OrganizationUnitName>\CN=<WEB_DOMAIN>\emailAddress=<EmailAddress>"
Example for the last line:
-subj "//C=US\ST=Washington\L=Seattle\O=kwwxis\OU=kwwxis\CN=<WEB_DOMAIN>\[email protected]"
-
Run this command.
Remember to replace
<WEB_DOMAIN>
openssl x509 \ -req \ -in <WEB_DOMAIN>.csr \ -CA rootSSL.pem -CAkey rootSSL.key -CAcreateserial \ -out <WEB_DOMAIN>.crt \ -days 500 \ -sha256 \ -extfile openssl.<WEB_DOMAIN>.cnf
-
Edit the
.env
file and set the SSL_KEY and SSL_CERT properties.SSL_KEY=C:/<wherever-you-put-the-files>/<WEB_DOMAIN>.key SSL_CERT=C:/<wherever-you-put-the-files>/<WEB_DOMAIN>.crt SSL_CA=C:/<wherever-you-put-the-files>/rootSSL.pem
-
You'll need to register the
rootSSL.pem
file you created with your Operating System. You can find instructions on how to do that here in step 3 "Trust the Certificate Authority CA on your local development machine".
If you don't want to use SSL locally, you can use these settings:
- No SSL on localhost:
Application would be accessed at http://localhost:3002/
VHOST=localhost VHOSTED=0 HTTP_PORT=3002 SSL_ENABLED=false
- No SSL on custom domain name:
Application would be accessed at http://saccharose.localhost/
VHOST=saccharose.localhost VHOSTED=1 HTTP_PORT=80 SSL_ENABLED=false
You'll need to repeat this step after every new Genshin Impact version.
It's recommended you install ts-node
globally with npm install -g ts-node
-
Obtain data folders for GI/HSR/ZZZ
-
Obtain the GI Data folder and specify the location to it in the
GENSHIN_DATA_ROOT
property of.env
- The Genshin Data folder should contain these folders:
ExcelBinOutput
,Readable
,Subtitle
,TextMap
.- The
Readable
folder should contain sub-folders where each sub-folder name is<LangCode>
for each language code. - The
Subtitle
folder should contain sub-folders where each sub-folder name is<LangCode>
for each language code. Within each language code folder there should be SRT files with the file extension.txt
or.srt
- The
TextMap
folder should contain JSON files in the format ofTextMap<LangCode>.json
where<LangCode>
is one of these:'CHS', 'CHT', 'DE', 'EN', 'ES', 'FR', 'ID', 'IT', 'JP', 'KR', 'PT', 'RU', 'TH', 'TR', 'VI'
. For exampleTextMapCHS.json
.
- The
- Also needs the
BinOutput
folder
- The Genshin Data folder should contain these folders:
-
Obtain the HSR Data folder and specify the location to it in the
HSR_DATA_ROOT
property of.env
-
Obtain the ZZZ Data folder and specify the location to it in the
ZENLESS_DATA_ROOT
property of.env
-
-
Import files (normalize)
- Run with:
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --normalize
- Run with:
ts-node ./src/backend/importer/hsr/import_hsr_files.ts --normalize
- Run with:
ts-node ./src/backend/importer/zenless/import_zenless_files.ts --normalize
- Run with:
-
Import files (plaintext)
This will create a new folder called Plain at
{DATA_ROOT}/TextMap/Plain
and will fill this folder with files calledPlainTextMap<LangCode>_Hash.dat
andPlainTextMap<LangCode>_Text.dat
for each language code.- Run with:
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --plaintext
. - Run with:
ts-node ./src/backend/importer/hsr/import_hsr_files.ts --plaintext
- Run with:
ts-node ./src/backend/importer/zenless/import_zenless_files.ts --plaintext
- Run with:
-
Import files (voice) [genshin only]
This will create or overwrite a file called
VoiceItems.json
in yourGENSHIN_DATA_ROOT
folder.- Run with:
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --voice-items
.
- Run with:
-
Run import_db for each game
- Run with:
ts-node ./src/backend/importer/import_db.ts
.- Use the
--help
flag to see all the options - Use the
--game
flag with one of these values:genshin
,hsr
,zenless
- You can use
--run-all
to first time you run it - Other options such as
--run-only
can regenerate specific tables on the existing database
- Use the
- Run with:
-
Import files (index)
There are various other files to import after importing the database.
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --index
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --gcg-skill
ts-node ./src/backend/importer/genshin/import_genshin_files.ts --voice-overs
- Run
npm run build:dev
thennpm run start
to build and run the application.- For production build, use
npm run build:prod
instead.
- For production build, use
- Note that
build:dev/prod
builds both the frontend and backend.- To build just the backend, run
npm run backend:build
- To build just the frontend, run
npm run webpack:dev
ornpm run webpack:prod
- To build just the backend, run
While developing, you'll want to use live reloading. This will watch for file changes and
automatically reload the code as you're developing. This is much faster than running
npm run build:dev
and npm run start
every time you make a code change.
- Run
npm run ts-serve:dev
to start the backend with live-reloading. - Run
npm run webpack:dev-watch
to start the frontend with live-reloading.
It doesn't matter which order you run those two commands in.
/dist
- build output for backend (gitignored folder)/public/dist
- build output for frontend (gitignored folder)/src/backend
- backend code/src/frontend
- frontend code/src/shared
- shared code used by both frontend and backend code. The frontend and backend folders should not share code between each other. Any shared code should go in./src/shared
You'll want to create the public/images/genshin
folder and add the images matching these conditions.
All files in Texture2D
starting with:
UI_
Eff_
Skill_
MonsterSkill
All files in Sprite
starting with:
UI_Gcg_Dice
UI_Gcg_Buff
UI_Gcg_Tag
UI_Buff
UI_HomeWorldTabIcon
Having these images isn't necessary for the application to run, but you'll have a bunch of broken images without them in certain areas of the UI. If you don't know where to get these images, you can ask for them in the "Saccharose.wiki" Discord editing forums post.
Commands:
-
Print Texture2D:
find ./Texture2D/ -type f -regextype posix-extended -iregex '.*/(UI_|MonsterSkill|Eff_UI_Talent|.*Tutorial).*'
-
Copy Texture2D:
find ./Texture2D/ -type f -regextype posix-extended -iregex '.*/(UI_|MonsterSkill|Eff_UI_Talent|.*Tutorial).*' -exec cp '{}' dist ';'
-
Copy Sprite:
find ./Sprite/ -type f -regextype posix-extended -iregex '.*/(UI_Buff|UI_Gcg_Dice|UI_Gcg_Buff|UI_Gcg_Tag|UI_HomeWorldTabIcon).*' -exec cp '{}' dist ';'
-
Transfer::
rsync -avP ./dist hostname:/dest/path