User:Blinkingline/Site Admin stuff

From Archon Arcana - The KeyForge Wiki

General Architecture Notes

All of AA resides on a Digital Ocean Droplet.

Outside of the main MW directory, we use block storage to store all of the files and images for the wiki. Block storage is mounted at /var/www/html/pool and then symlinked to the images and file directories within the MediaWiki installation.

Give a user CLI access

Generate Site Map

  • To generate a sitemap manually:
php maintenance/generateSitemap.php --memory-limit=50M --fspath=/var/www/html/aa-en/sitemap/ --identifier=archonarcana --urlpath=/sitemap/ --server=https://archonarcana.com --compress=yes

Reboot the box

  • Rebooting the box in X minutes
shutdown -r X

Rebuilding the search index

    • in /etc/elasticsearch/ get the list of indices:
curl -X GET "localhost:9200/_cat/indices/*?v&s=index&pretty"
    • Delete the all of the Mediawiki indices:
curl -X DELETE "localhost:9200/mediadb?pretty"
    • In /var/www/html/aa-en/extensions/CirrusSearch/maintenance:
      • Re-initialize CirrusSearch (include the php version until php8 is supported):
php7.4 updateSearchIndexConfig.php --wiki=wikiid --startOver
      • Then force a reindex.
php7.4 forceSearchIndex.php --wiki=wikiid

Note that after a major upgrade if you upgrade Elasticsearch, you'll need to reenable it as a service 

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Pywikibot

  • PyWikiBot is installed on geeb.us in /etc/pywikibot
  • When a big update is we may need to do a bunch of null edits to force the pages to update. The easiest way to do this is through pywikibot's touch command. Below will login via Pywikibot, then do a touch of all articles in the "Card" category
cd /etc/pywikibot/core
python3 pwb.py login
nohup python3 pwb.py touch -cat:Card -start:! &

Last command puts the output to a file called nohup.out. The job should continue to run even if the terminal session gets disconnected.

You MAY need to then run runJobs.php on the AA box, just to clean anything else up.

Jobrunner Service

Jobs seemed to be piling up in the queue so there's a service now that runs runJobs.php at regular intervals, rather than waiting on Web requests to trigger it. When we upgrade, we'll need to adjust the path of the jobrunner.

  • /usr/local/bin/mwjobrunner
#!/bin/bash
# Put the MediaWiki installation path on the line below
MW_INSTALL_PATH=/var/www/html/mediawiki-1.34.0
RUNJOBS=$MW_INSTALL_PATH/maintenance/runJobs.php
echo Starting job service...
# Wait a minute after the server starts up to give other processes time to get started
sleep 60
echo Started.
while true; do
       # Job types that need to be run ASAP mo matter how many of them are in the queue
       # Those jobs should be very "cheap" to run
       php $RUNJOBS --type="enotifNotify"
       # Everything else, limit the number of jobs on each batch
       # The --wait parameter will pause the execution here until new jobs are added,
       # to avoid running the loop without anything to do
       php $RUNJOBS --wait --maxjobs=20
       # Wait some seconds to let the CPU do other things, like handling web requests, etc
       echo Waiting for 10 seconds...
       sleep 10
done
  • /etc/systemd/system/mw-jobqueue.service
[Unit]
Description=MediaWiki Job runner

[Service]
ExecStart=/usr/local/bin/mwjobrunner
Nice=10
ProtectSystem=full
User=www-data
OOMScoreAdjust=200
StandardOutput=journal

[Install]
WantedBy=multi-user.target
  • Enabling, starting, and checking the service
sudo systemctl enable mw-jobqueue
sudo systemctl start mw-jobqueue
sudo systemctl status mw-jobqueue

Upgrading MediaWiki versions

  1. Create make sure the domain new.archonarcana.com is available for testing.
  2. Create aa-new directory in the /html folder
  3. Download the new version of mediawiki to aa-new and extract it.
  4. Copy LocalSetttings.php to the new install. Note you will need to install extensions and skins noted below.
    1. Modify the $wgServer = "https://new.archonarcana.com"; until testing is over.
  5. Make symlinks for files and images: ln -s /var/www/html/pool/images images (may have to delete the existing images directory)
  6. Backup the database using the command below
mysqldump -u root -p current_database > backup.sql
  1. Create a new database for the new version using (new_database should follow appropriate name)
CREATE DATABASE new_database;
  1. Copy the database to the new database using the command below (Replace "new_database" with the appropriate name)
mysql -u root -p new_database < backup.sql
  1. Grant the appropriate privileges to the database user.
  2. Edit LocalSettings.php to reflect the new database name
  3. run php update.php from the maintenance folder to update the db schema.
  4. Use https://new.archonarcana.com/mw-update to complete the update process, if necessary.

Once upgrade is complete and moved to stable, copy robots.txt to the new directory and generate a new site map.

Skins and Extensions needed as of MW 1.38

These are the skins and extensions used outside of core:

Retrieved from "https://archonarcana.com/index.php?title=User:Blinkingline/Site_Admin_stuff&oldid=126196"