Ticket #706 (closed maintenance: fixed)
Upgrade Mediawiki to 1.22.5 and install the new VisualEditor
Reported by: | chris | Owned by: | chris |
---|---|---|---|
Priority: | major | Milestone: | Maintenance |
Component: | Mediawiki | Keywords: | |
Cc: | ed | Estimated Number of Hours: | 0.0 |
Add Hours to Ticket: | 0 | Billable?: | yes |
Total Hours: | 3.1 |
Description
From MediaWiki-announce:
this is a notice that on Thursday, March 27th between 17:00-18:00 UTC (Thursday, March 27th, 9:00-10:00am PST) we will release security and maintenance updates for current and supported branches of the MediaWiki software. Downloads and patches will be available at that time.
Attachments
Change History
comment:2 in reply to: ↑ 1 Changed 3 years ago by chris
Replying to ed:
I am going to start asking for a wiki package with a much more significant WYSIWYG in the next few months. fyi.
We could update to 1.22.5 this week and switch to the new VisualEditor at the same time if you would like.
comment:3 Changed 3 years ago by chris
More on the new WYSIWYG editor here https://www.mediawiki.org/wiki/VisualEditor
Test it here: https://www.mediawiki.org/wiki/VisualEditor:Test
comment:4 follow-up: ↓ 5 Changed 3 years ago by ed
Whatever is available and more friendly for management people like me, and will encourage better documentation from techs, we need it.
Documentation will be in the terms and conditions for future contracts and we will need to offer a *nice* solution. *Nice* from my perspective :)
comment:5 in reply to: ↑ 4 Changed 3 years ago by chris
Replying to ed:
Whatever is available and more friendly for management people like me
Can you test this to see if it is acceptable?
comment:6 follow-up: ↓ 7 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 0.0 to 0.25
The VisualEditor project aims to create a reliable rich-text editor for MediaWiki.
This is clearly the one to use and it looks stable enough for us:
Use at your own risk; it's not ready for production deployment except for experts!
Parsoid will be needed:
- https://www.mediawiki.org/wiki/Parsoid#Getting_started
- https://www.mediawiki.org/wiki/Parsoid/Setup
This will take some time to set up and document (a few hours I guess) but the result would be that we would using the new default WYSIWYG Wikipedia editor and this is clearly going to be the main one for the next few years, all these others will be abandoned in time:
Should I change the title of this ticket to "Upgrade Mediawiki to 1.22.5 and install the new VisualEditor"?
comment:7 in reply to: ↑ 6 Changed 3 years ago by chris
I don't think I'll time tonight to do the 1.19.14 upgrade as it's not yet out.
Tomorrow, if I have an answer to this:
Replying to chris:
Should I change the title of this ticket to "Upgrade Mediawiki to 1.22.5 and install the new VisualEditor"?
I could either upgrade to 1.19.14 or 1.22.4 with the new WYSIWYG editor --Ed which would you like me to do?
comment:9 Changed 3 years ago by chris
- Summary changed from Mediawiki 1.19.14 to Upgrade Mediawiki to 1.22.5 and install the new VisualEditor
comment:10 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.75
- Total Hours changed from 0.25 to 2.0
The existing wiki site is running Mediawiki 1.19.13 and has the following installed extensions:
- SyntaxHighlight (Version 1.0.8.10) https://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi "This extension is bundled with MediaWiki 1.21 and above. Thus you do not have to download it again."
- Piwik Integration (Version 2.0.0) https://www.mediawiki.org/wiki/Extension:Piwik_Integration
- WikiEditor (Version 0.3.1) https://www.mediawiki.org/wiki/Extension:WikiEditor "This extension is bundled with MediaWiki 1.18 and above. Thus you do not have to download it again."
So the only extension that will need manually downloading is Piwik Integration.
Following the upgrade notes, trac:wiki/MediaWiki#Updates
sudo -i cd /web/wiki.transitionnetwork.org export MW="1.22.5" wget http://releases.wikimedia.org/mediawiki/1.22/mediawiki-$MW.tar.gz -O mediawiki-$MW.tar.gz wget http://releases.wikimedia.org/mediawiki/1.22/mediawiki-$MW.tar.gz.sig -O mediawiki-$MW.tar.gz.sig gpg --verify mediawiki-$MW.tar.gz.sig gpg: Signature made Fri Mar 28 00:21:11 2014 GMT using RSA key ID 7F901A30 gpg: Good signature from "Mark A. Hershberger <mah@everybody.org>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 3CEF 8262 806D 3F0B 6BA1 DBDD 7956 EE47 7F90 1A30 tar -zxvf mediawiki-$MW.tar.gz rsync -av mediawiki-$MW/ www/ chown root:root -R www/ chown -R www-data:www-data www/cache/ chown -R www-data:www-data www/images/ cd www/maintenance/ php update.php cd - wget https://github.com/DaSchTour/piwik-mediawiki-extension/archive/master.zip unzip master.zip rsync -av piwik-mediawiki-extension-master/ www/extensions/Piwik/ rm mediawiki-$MW.tar.gz mediawiki-$MW.tar.gz.sig rm -rf mediawiki-$MW rm -rf piwik-mediawiki-extension-master/ rm master.zip
And now on the https://wiki.transitionnetwork.org/Special:Version page we have:
- MediaWiki 1.22.5
- SyntaxHighlight (Version 1.0.8.11)
- Piwik Integration (Version 2.4.0)
- WikiEditor (Version 0.3.1)
Now for the https://www.mediawiki.org/wiki/Extension:VisualEditor install, following https://www.mediawiki.org/wiki/Parsoid/Setup
cd /web/wiki.transitionnetwork.org git clone https://gerrit.wikimedia.org/r/p/mediawiki/services/parsoid aptitude install nodejs npm Couldn't find any package whose name or description matched "nodejs" Couldn't find any package matching "npm".
So, following https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager#backports /etc/apt/sources.list.d/backports.list was created containing:
deb http://ftp.debian.org/debian wheezy-backports main
And following https://github.com/joyent/node/wiki/backports.debian.org
aptitude update aptitude install nodejs-legacy curl curl --insecure https://www.npmjs.org/install.sh | bash
Then:
cd /web/wiki.transitionnetwork.org/parsoid npm install
Then create /web/wiki.transitionnetwork.org/parsoid/api/localsettings.js containing:
/* * This is a sample configuration file. * * Copy this file to localsettings.js and edit that file to fit your needs. * * Also see the file ParserService.js for more information. */ exports.setup = function( parsoidConfig ) { // The URL of your MediaWiki API endpoint // // We pre-define wikipedias as 'enwiki', 'dewiki' etc. Similarly for // other projects: 'enwiktionary', 'enwikiquote', 'enwikibooks', // 'enwikivoyage' etc. // // Optionally, you can also pass in a proxy specific to this prefix // (overrides defaultAPIProxyURI), or null to disable proxying for // this end point. //parsoidConfig.setInterwiki( 'localhost', 'http://localhost/w/api.php' ); parsoidConfig.setInterwiki( 'localhost', 'http://wiki.transitionnetwork.org/api.php' ); // A default proxy to connect to the API endpoints. Default: undefined // (no proxying). Overridden by per-wiki proxy config in setInterwiki. // parsoidConfig.defaultAPIProxyURI = 'http://proxy.example.org:8080'; // Enable debug mode (prints extra debugging messages) // parsoidConfig.debug = true; // Use the PHP preprocessor to expand templates via the MW API (default true) //parsoidConfig.usePHPPreProcessor = false; // Use selective serialization (default false) parsoidConfig.useSelser = true; // Allow cross-domain requests to the API (default '*') // Sets Access-Control-Allow-Origin header // disable: //parsoidConfig.allowCORS = false; // restrict: parsoidConfig.allowCORS = 'wiki.transitionnetwork.org'; };
The start the server:
cd /web/wiki.transitionnetwork.org/parsoid export INTERFACE=127.0.0.1 node api/server.js
So it looks like it's working, create a /etc/init.d/pslparsoid file following the example at https://www.mediawiki.org/wiki/Talk:Parsoid
#! /bin/sh # /etc/init.d/pslparsoid ### BEGIN INIT INFO # Provides: pslparsoid # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Should-Start: $local_fs # Should-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start Parsoid daemon # Description: Start up Parsoid, Debian run-level script by PSL GbR (draft) ### END INIT INFO # Some things that run always #env VCAP_APP_PORT="8000" #env NODE_PATH="/web/wiki.transitionnetwork.org/parsoid/node_modules" export VCAP_APP_PORT="8000" export NODE_PATH="/web/wiki.transitionnetwork.org/parsoid/node_modules" # Carry out specific functions when asked to by the system case "$1" in start) echo "Starting Parsoid..." cd "/web/wiki.transitionnetwork.org/parsoid" /usr/bin/node /web/wiki.transitionnetwork.org/parsoid/api/server.js >> nohup.out 2>&1 & ;; restart) echo "Stopping Parsoid..." echo "killall -9 node, could do more here..." killall -9 node ;; 'reload'|'force-reload') echo "Could fail..." cd "/web/wiki.transitionnetwork.org/parsoid" /usr/bin/node /web/wiki.transitionnetwork.org/parsoid/api/server.js >> nohup.out 2>&1 & ;; stop) echo "Stopping Parsoid..." killall -9 node ;; *) echo "Usage: /etc/init.d/pslparsoid {start|stop|restart|reload|force-reload}" exit 1 ;; esac exit 0
Install the
cd /web/wiki.transitionnetwork.org wget https://codeload.github.com/wikimedia/mediawiki-extensions-VisualEditor/legacy.tar.gz/REL1_22 -O VisualEditor.tar.gz tar -zxvf VisualEditor.tar.gz mkdir /web/wiki.transitionnetwork.org/www/extensions/VisualEditor rsync -av wikimedia-mediawiki-extensions-VisualEditor-43e96d7/ www/extensions/VisualEditor/ mv rm -rf VisualEditor.tar.gz wikimedia-mediawiki-extensions-VisualEditor-43e96d7/
Edit LocalSettings?.php and add:
require_once("$IP/extensions/VisualEditor/VisualEditor.php"); // OPTIONAL: Enable VisualEditor in other namespaces // By default, VE is only enabled in NS_MAIN //$wgVisualEditorNamespaces[] = NS_PROJECT; // Enable by default for everybody $wgDefaultUserOptions['visualeditor-enable'] = 1; // Don't allow users to disable it // $wgHiddenPrefs[] = 'visualeditor-enable'; // OPTIONAL: Enable VisualEditor's experimental code features //$wgVisualEditorEnableExperimentalCode = true; // URL to the Parsoid instance // MUST NOT end in a slash due to Parsoid bug $wgVisualEditorParsoidURL = 'http://localhost:8000';
Start the Parsoid server and set it to run on a reboot:
/etc/init.d/pslparsoid start update-rc.d pslparsoid defaults
And we now have a new WYSIWYG editor for the Main namespace (most pages):
TODO:
- Test it
- Document it
comment:11 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Status changed from new to closed
- Resolution set to fixed
- Total Hours changed from 2.0 to 2.25
I have done some testing and the new editor seems to be working fine, I have also updated the wiki:MediaWiki page, the upgrade notes can be updated next time it is upgraded -- this will be a more complex task in the future.
Closing this ticket now.
comment:12 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 2.25 to 2.5
It's worth noting that adding nodejs has increased the memory usage on the server, see:
I have updated the /etc/munin/plugin-conf.d/munin-node files:
[multips_memory] env.names php5-fpm munin-node nginx mysqld tracd python PassengerHelper PassengerLoggin PassengerWatchd nodejs ruby1.8
To keep track of this here:
There are 11 nodejs processes:
ps -lA | grep nodejs | wc -l 11
This is probably way more than we need, it's using over 500MB of RAM, but I can't find where to set this, we might need to allocate more RAM to the server.
comment:13 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 2.5 to 3.0
Looking at /web/wiki.transitionnetwork.org/parsoid/api/server.js it's clear why 11 server are started:
// Start a few more workers than there are cpus visible to the OS, // so that we get some degree of parallelism even on single-core // systems. A single long-running request would otherwise hold up // all concurrent short requests.
We have these options:
cd /web/wiki.transitionnetwork.org/parsoid/api ./server.js -h Usage: node ./server.js [-h|-v] [--param[=val]] Options: -n, --num-workers [default: 11] -c, --config [default: "/web/wiki.transitionnetwork.org/parsoid/api/localsettings.js"] -v, --version [default: false] -h, --help [default: false]
So /etc/init.d/pslparsoid has been edited to:
#! /bin/sh # /etc/init.d/pslparsoid ### BEGIN INIT INFO # Provides: pslparsoid # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Should-Start: $local_fs # Should-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start Parsoid daemon # Description: Start up Parsoid, Debian run-level script by PSL GbR (draft) ### END INIT INFO # Some things that run always #env VCAP_APP_PORT="8000" #env NODE_PATH="/web/wiki.transitionnetwork.org/parsoid/node_modules" export VCAP_APP_PORT="8000" export NODE_PATH="/web/wiki.transitionnetwork.org/parsoid/node_modules" NUM_WORKERS="4" # Carry out specific functions when asked to by the system case "$1" in start) echo "Starting Parsoid with ${NUM_WORKERS} worker processes..." cd "/web/wiki.transitionnetwork.org/parsoid" /usr/bin/node /web/wiki.transitionnetwork.org/parsoid/api/server.js -n ${NUM_WORKERS} >> nohup.out 2>&1 & ;; restart) echo "Stopping Parsoid..." echo "killall -9 node, could do more here..." killall -9 node ;; 'reload'|'force-reload') echo "Could fail..." cd "/web/wiki.transitionnetwork.org/parsoid" /usr/bin/node /web/wiki.transitionnetwork.org/parsoid/api/server.js -n ${NUM_WORKERS} >> nohup.out 2>&1 & ;; stop) echo "Stopping Parsoid..." killall -9 node ;; *) echo "Usage: /etc/init.d/pslparsoid {start|stop|restart|reload|force-reload}" exit 1 ;; esac exit 0
And it was stopped and restarted and the number of processes checked:
/etc/init.d/pslparsoid stop /etc/init.d/pslparsoid start ps -lA | grep nodejs | wc -l 4
We should now see less memory usage overall:
And specifically less used by nodejs:
I have done some testing of the new editor on a new https://wiki.transitionnetwork.org/Sandbox page and it seems good to me.
There are now two edit options, "Edit" and "Edit source" -- the "Edit" option is the new editor and the "Edit source" is the old one.
I have added a wiki:MediaWiki#Parsoid section to the documentation.
comment:14 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 3.0 to 3.1
I have just dropped the number of node.js processes from 4 to 2 -- we don't use the Mediawiki site enough to justify more at the moment, this will half the RAM usage.
comment:15 follow-up: ↓ 16 Changed 3 years ago by ed
nice
comment:16 in reply to: ↑ 15 Changed 3 years ago by chris
Replying to ed:
nice
Indeed, it's by far the best HTML WYSIWYG editor I've ever used, the only issue I found was no table support, but I expect that this will be on the todo list.
Now we need to work out how the Transition Network can make more use of their wiki!
comment:17 follow-up: ↓ 18 Changed 3 years ago by ed
atm everyone is all over google docs. make it as good as that and we've a chance.
comment:18 in reply to: ↑ 17 Changed 3 years ago by chris
Replying to ed:
atm everyone is all over google docs. make it as good as that and we've a chance.
Interesting.
Are people doing real-time multi-user editing?
If they are then we should look at setting up pad.transitionnetwork.org running Etherpad Lite:
It needs node.js and we have just installed this...
Are people also using Google for spreadsheets? If they are then Ethercalc at calc.transitionnetwork.org is the answer:
It also uses node.js:
Shall I open a new ticket for this...?
comment:19 Changed 3 years ago by ed
No new tickets for now. TN have an intranet group working on doing some requirements as we type. Needs range from Open Atrium stuff to google docs and Huddle shared docs and groups stuff. Etherpad has the shared editing which is good - imo the killer app for google docs is the commenting - e.g. https://docs.google.com/document/d/1pINZIzr4NhqBS2KYph_llyuQ4ofSVNDYAE21tKAI-yQ/edit
I am going to start asking for a wiki package with a much more significant WYSIWYG in the next few months. fyi.