We are planning to maintain the list of active projects and their details in the attached google sheet. We also have a page on the wiki which hosts almost simliar information.

It would be good if we can add SheetMW extension to our wiki and embed the sheet in the wiki page instead of having to update both at separate instances.

https://docs.google.com/spreadsheets/d/1fNXrs0fmByG-Fji27vtvjW2zuJ6e5DeSH2Q3U08xXTE/edit?usp=sharing

Currently, we are using Node 0.10.0 on Debian's Jessie (2 stable releases
ago) for the Parsoid container.

It's possible that this is the recommended version for Parsoid itself. So, one
would have to first investigate whether that's the case. And if not, upgrade the
Node version.

parsoid/Dockerfile
1:FROM node:10-jessie

If a page is in "Popular", then it's better to deduplicate it from "Trending" so that we can showcase as many articles as possible. This involves changing the pywikibot script to fetch 20 trending items, dedup, and post the top 10.

Recently, we started facing problems with this extension. In particular, on the Feedback page, users are able to see the feedback that was submitted by administrators are unable to mark the feedback as Resolved, Useful, etc.

I had to restart all the containers on our server because the memory usage was as high as 85% and this triggered a Newrelic alert for Error rate > 10%.

One of the errors I saw on the website was unable to fork() - cannot allocate memory

Slack thread

$wgStyleVersion was removed completely since Mediawiki 1.32. This is used inside the Maps extension here. We use Maps v4.4.0.

We don't seem to be having any problems with our current setup, but if something does come up related to the Maps extension, we might have to update the version of the extension we use.

It is a script to send Facebook posts to metakgp-announce. See https://github.com/metakgp/FaceMail

Current source is near to its death. Use pip install pywikibot and make other necessary changes to accommodate this change.

Source

• I want to see logs from the X container (X can be nginx, backup, or any running container)
• I want to run the wiki locally

The restore-from-backup.sh script only restores the database backup. Add the following:

1. A condition to check if the backup includes an images/ folder (it may not)
2. If the files exist copy them inside the mediawiki container's /srv/static/
3. Execute chown -LR www-data:www-data /srv/static/images inside the container

We will need to add more documentation in case of backup, especially the steps needed to be taken in case of a migration. The following steps are to be highlighted:

• Backup of static-volume (Currently, only the peqp/ folder.)
• DROP DATABASE metakgp_wiki_db; followed by CREATE_DATABASE metakgp_wiki_db in the mysql shell inside the mysql container; followed by ./scripts/restore-from-backup.sh. This ensures that any existing database is empty before restoring the latest backup.
• Steps to put the old wiki in Read Only mode
• Steps to be taken in the overall process

The extension we are using, RecentChanges doesn't seem to be supported any longer.

Whereas the SimpleChanges extension looks promising:

The SimpleChanges extension creates a Special page that displays a barebones Recent Changes list. It is ideal for transclusion.
-- SimpleChanges

• The UA-XXXXXX-XX google analytics id is deprecated and hence the googleAnalytics extension no longer works.
• Need to find a way to use the new "head" tags of Google Analytics and adapt the jobs script accordingly.

Currently, the MySQL container is not writing any query logs at all. Erroneous query logs are written to stderr.

# Last lines of the mysql error log
icyflame@metakgp-blr:~$ docker exec -it metakgp-wiki_mysql_1 tail /var/log/mysql/error.log
2018-02-26T23:11:59.997694Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 180226 23:11:59
2018-02-26T23:12:01.707688Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2551387
2018-02-26T23:12:01.709698Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-02-26T23:12:01.709711Z 0 [Note] Shutting down plugin 'MEMORY'
2018-02-26T23:12:01.709716Z 0 [Note] Shutting down plugin 'CSV'
2018-02-26T23:12:01.709720Z 0 [Note] Shutting down plugin 'sha256_password'
2018-02-26T23:12:01.709723Z 0 [Note] Shutting down plugin 'mysql_native_password'
2018-02-26T23:12:01.709892Z 0 [Note] Shutting down plugin 'binlog'
2018-02-26T23:12:01.711473Z 0 [Note] mysqld: Shutdown complete

These are the last few lines in the Error log and they were written back in February 2018. I checked the Global variables that enable logging, and they are not set properly.

mysql> SELECT @@global.general_log;
+----------------------+
| @@global.general_log |
+----------------------+
|                    0 |
+----------------------+
1 row in set (0.00 sec)mysql> SELECT @@global.general_log_file;
+---------------------------------+
| @@global.general_log_file       |
+---------------------------------+
| /var/lib/mysql/4efb1f830bf6.log |
+---------------------------------+
1 row in set (0.00 sec)mysql> SELECT @@global.log_output;
+---------------------+
| @@global.log_output |
+---------------------+
| FILE                |
+---------------------+
1 row in set (0.00 sec)

We have to set general_log = 1 in the MySQL configuration file. We are not currently using one in this repository. We have to start using it when we build the mysql Docker image.

The configuration file has to be written with some basic options and then inserted into the correct path. More documentation on this can be found on the webpage for the official mysql docker image under the Using a custom MySQL configuration file section.

Possible tests:

• Main page should render without errors on desktop and mobile browsers
• Create user (should fail because of captcha)
• Upload image (check that uploads are configured correctly)
• View image (check that thumbnailing works)
• Try to access all the images, or wiki config files (check that nginx disallows file access)

Look into using something like PhantomJS or Selenium.

Bots take 10 sec for every edited page as per the current limiting rate. When updating multiple pages via bots can be time-consuming.

For example:
In the case of blackjack bot, which updates the courses with new grade distribution, the no of courses 'updated' is approx 1000, so 10 sec gap extends the process to nearly 3 hrs.

It would be good to have a CI workflow that builds the wiki containers and runs various tests. (on every commit and pr)

• #118
• Tests:
  • Make sure container builds
  • Make sure wiki starts and runs
  • #14
  • #63

The failure is because of the lack of the python module oauth2client: https://github.com/googleapis/oauth2client

It looks like this module has been deprecated recently. The usage of google-auth is recommended on the repository page. This is the particular module inside google-auth that we would be using: https://google-auth.readthedocs.io/en/latest/reference/google.oauth2.service_account.html#module-google.oauth2.service_account

Failure log

icyflame@metakgp-blr:~$ docker exec metakgp-wiki_jobs_1 /root/update_top_trending.sh
Updating Top and Trending Pages
+ echo 'Updating Top and Trending Pages'
+ cd /root/pywikibot
+ export METAKGP_BOT_NAME=batman
+ METAKGP_BOT_NAME=batman
+ timeout 10s python pwb.py login
Logging in to metakgp:en as Batman@update-statistics
Logged in on metakgp:en as Batman.
+ timeout 30s python pwb.py updatestatistics
Traceback (most recent call last):
  File "pwb.py", line 264, in <module>
    if not main():
  File "pwb.py", line 257, in main
    run_python_file(filename, [filename] + args, argvu, file_package)
  File "pwb.py", line 120, in run_python_file
    main_mod.__dict__)
  File "./scripts/updatestatistics.py", line 7, in <module>
    from oauth2client.service_account import ServiceAccountCredentials
ImportError: No module named oauth2client.service_account
CRITICAL: Closing network session.
<type 'exceptions.ImportError'>

Use HitCounters extension instead of the google analytics workflow to update the trending and popular pages on the main page.

#78 (comment)

Scribunto is shipped with mediawiki-core starting Mediawiki 1.34. We have some configuration on our side (specifically, two lines that set some variables and making the Lua executable executable using chmod a+x while installing other extensions)

Investigate whether these lines are required. If they aren't required anymore, remove them. 👍

Steps:

• Set up redis container
• Set up a batman container with Python
• Add environment variables for Slack access
• Modify docker-compose.yml to link everything together, and pass in env
• Run batman inside its container as a daemon, perhaps by using some kind of init system

[some hash or something]Caught exception of type MWException

This is the error displayed when tried to submit a feedback.

See #109

A refresh of the wiki is being developed in the metawiki-v2 branch.

Changes

• Upgrade to Mediawiki v1.40.0
• Upgrade extension versions, update LocalSettings.php
• #77
• #76
• #73
• Bugs
  • #88
  • #80
  • #74
• Combine mediawiki and php containers.
• Split nginx from the compose. (See https://github.com/metakgp/metaploy)
• Update compose files.
• Documentation
  • #120
  • #39
  • #32
• #111
• #112

The following documentation needs to be added/updated in the README:

• Getting started (#129)
• How to set up secondary services (#129)
• All environment variables (#121)
• Production deployment instructions (#129)
• An overview of the wiki's architecture, written on a piece of paper or drawn on a blackboard

The following documentation needs to be updated in the RUNBOOK:

• Backup restoration with the new script (#122)

Migrate the wiki database to Postgres under the database of babel.

The backup container currently uses the Docker image python:2-jessie. Python 2
reached End-of-Life recently, so we should stop using it.

------------------------ >8 ------------------------ Do not modify or remove

the line above. Everything below it will be ignored.

Creating an issue for metakgp/metakgp-wiki

Write a message for this issue. The first block of
text is the title and the rest is the description.

The jobs container is used for running recurrent jobs like updating the trending pages and the spam IP list. Backup is also a recurrent job, so there is no need to make a separate container for it.

TODO

• Fix the backup script.
• Make the backup script delete backups older than one month (excluding the last backup that has images).
• Merge the backup script into the jobs container.

Discussion thread for ideas on how automatic deployment (CD) could be set up.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• Update dependency dropbox to v9.5.0
• Update php Docker tag to v7.4
• Update dependency dropbox to v11
• Update mysql Docker tag to v8
• Update php Docker tag to v8
• Update python Docker tag to v3

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

There are several PHP warnings.

Probable cause: we are still using Mediawiki 1.28 and PHP 7. ref

These are some of the warnings:

because it is the place it should reside.

The Nginx Dockerfile doesn't copy the peqp dump to /srv/static by default. This has to done by hand.
Recently the server had to be restarted which led to the deletion of the static files.
Maybe we should add some code so that this addition automatically happens with new builds.

@icyflame @amrav @thealphadollar

Users are unable to create account in the wiki due their IP getting blocked by the DNS Blacklist.
Temporary fix is to disable the DNSBL option.(PR#130)

The blacklist currently used is sbl.spamhaus.org. We should use another blacklist or find an alternative solution.

Visual editor probably (anecdotal) increases the number of edits by the long tail of infrequent contributors by making it very simple to make small changes/updates. This helps reduce the entry barrier to becoming a more frequent editor, and is also essential for keeping the wiki up to date.

Add a parsoid container for running parsoid (and optionally restbase), and add visual editor to the list of installed extensions.

Add the DismissableSiteNotice extension.

• Build docker containers using a password other than the current password, and then restore. This should work properly
• If not, where is the current password stored?