Ticket #470 (closed maintenance: fixed)
Penguin install and configuration
| Reported by: | chris | Owned by: | chris |
|---|---|---|---|
| Priority: | major | Milestone: | Maintenance |
| Component: | Live server | Keywords: | |
| Cc: | ed, laura, jim | Estimated Number of Hours: | 25.0 |
| Add Hours to Ticket: | 0 | Billable?: | yes |
| Total Hours: | 31.8 |
Description (last modified by chris) (diff)
penguin.webarch.net is a new 2GB RAM virtual server which will replace NewLiveServer and DevelopmentServer for running all non-Drupal sites and is due to go live in early 2013. Drupal sites from the old servers will be migrated to PuffinServer.
This ticket has been created for tracking time and tasks done during the install.
See https://tech.transitionnetwork.org/trac/wiki/PenguinServer and also the corresponding PuffinServer.
Change History
comment:1 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 0.0 to 1.0
comment:2 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.54
- Total Hours changed from 1.0 to 2.54
Installing nginx, trac and munin, attempting to use the debian versions from backports, following http://backports-master.debian.org/Instructions/
echo "deb http://backports.debian.org/debian-backports squeeze-backports main" > /etc/apt/sources.list.d/backports.list aptitude -t squeeze-backports install "nginx" aptitude -t squeeze-backports install "trac" aptitude -t squeeze-backports install "munin"
And following http://forums.debian.net/viewtopic.php?f=17&t=80679#p438915
echo 'APT::Default-Release "stable";' >> /etc/apt/apt.conf
Mediwiki is in testing but not backports, so I'm not sure if we should use the debian package or not.
For nginx ticket:357 was read, and rather than using the dotdeb repos the wheezy http://packages.debian.org/wheezy/php5-fpm was considered as dotdeb did cause some problems on kiwi with mysql upgrades. Using dotdeb would also result in their nigix rather than the backports one, so the suggested way of doing it from here seems like the best bet: http://serverfault.com/questions/448246/php-fpm-for-nginx-on-debian
comment:3 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.7
- Total Hours changed from 2.54 to 4.24
These are the packages installed from backports:
dpkg -l | awk '/^ii/ && $3 ~ /bpo[456]0/ {print $2}'
gawk
geoip-database
libcairo2
libfreetype6
libgeoip1
liblog-dispatch-perl
libnet-server-perl
libpixman-1-0
liburi-perl
libxfont1
munin
munin-common
munin-doc
munin-node
munin-plugins-core
munin-plugins-extra
nginx
nginx-common
nginx-full
python-babel
trac
x11-common
Tidying up the apt changes so that everything is in a .d directory:
cd /etc/apt mv apt.conf apt.conf.d/02stable
Created /etc/apt/preferences.d/security.pref with the following:
Pin: release l=Debian-Security Pin-Priority: 1000 Package: *
Created /etc/apt/preferences.d/stable.pref with the following:
Pin: release a=stable Pin-Priority: 990 Package: *
Created /etc/apt/preferences.d/dotdeb.pref:
Package: * Pin: release o=packages.dotdeb.org Pin-Priority: 100 Package: php-pear php5-cli php5-common php5-fpm php5-mysql Pin: release o=packages.dotdeb.org Pin-Priority: 991
Created /etc/apt/preferences.d/backports.pref:
Package: gawk geoip-database libcairo2 libfreetype6 libgeoip1 liblog-dispatch-perl libnet-server-perl libpixman-1-0 liburi-perl libxfont1 munin munin-common munin-doc munin-node munin-plugins-core munin-plugins-extra nginx nginx-common nginx-full python-babel trac x11-common Pin: release o=backports Pin-Priority: 992
Then create /etc/apt/sources.list.d/dotdeb.list with the following in it:
deb http://packages.dotdeb.org squeeze all deb-src http://packages.dotdeb.org squeeze all
Install the gpg key:
cd wget http://www.dotdeb.org/dotdeb.gpg cat dotdeb.gpg | sudo apt-key add -
Update:
apt-get update apt-get install -t stable php-pear php5-cli php5-common php5-fpm php5-mysql
However this isn't working right as an upgrade would now result in mysql coming from dotdeb:
aptitude upgrade
The following packages will be upgraded:
mysql-common
apt-show-versions -u
mysql-common/squeeze upgradeable from 5.1.66-0+squeeze1 to 5.5.28-1~dotdeb.0
This was fixed by editing /etc/apt/preferences.d/stable.pref and changing "stable" to "squeeze":
Package: * Pin: release a=squeeze Pin-Priority: 990 Package: mysql-common Pin: release a=squeeze Pin-Priority: 995
Mysql packages are installed:
apt-show-versions | grep mysql libdbd-mysql-perl/squeeze uptodate 4.016-1 libmysqlclient16/squeeze uptodate 5.1.66-0+squeeze1 mysql-client-5.1/squeeze uptodate 5.1.66-0+squeeze1 mysql-common/squeeze uptodate 5.1.66-0+squeeze1 mysql-server-5.1/squeeze uptodate 5.1.66-0+squeeze1 mysql-server-core-5.1/squeeze uptodate 5.1.66-0+squeeze1 php5-mysql/squeeze uptodate 5.3.19-1~dotdeb.0
comment:4 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.27
- Total Hours changed from 4.24 to 4.51
A test install of media wiki from wheezy was done by enabling wheezy repos by creating /etc/apt/sources.list.d/wheezy.list with the following in it:
deb http://ftp.uk.debian.org/debian/ wheezy main deb-src http://ftp.uk.debian.org/debian/ wheezy main
And /etc/apt/preferences.d/wheezy.pref with this in it:
Package: * Pin: release a=wheezy Pin-Priority: 99 Package: mediawiki Pin: release a=wheezy Pin-Priority: 993
Then:
aptitude install -t wheezy mediawiki
But this pulls in mediawiki from squeeze and not wheezy, so I think the apt settings have a problem, this is one for another night!
comment:5 Changed 4 years ago by chris
At the meeting on 17th Dec, ticket:471, Jim suggested we use http://collectd.org/ rather than munin
comment:6 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 4.51 to 5.51
I have installed a couple scripts that I wrote for other servers to update packages and update the /root/Changelog file, see AptitudeUpdateScript
This has shown up a potential issue, apt-show-versions -b -u lists some available updates:
geoip-database/wheezy libfreetype6/squeeze-backports nginx/squeeze-backports nginx-common/squeeze-backports nginx-full/squeeze-backports
But these don't get updated with apt-get upgrade or aptitude upgrade or aptitude full-upgrade as they are only manually upgradeable:
apt-show-versions -u geoip-database/wheezy *manually* upgradeable from 20121207-1~bpo60+1 to 20130108-1~bpo60+1 libfreetype6/squeeze-backports *manually* upgradeable from 2.4.9-1~bpo60+1 to 2.4.9-1.1 nginx/squeeze-backports *manually* upgradeable from 1.2.1-2~bpo60+1 to 1.2.6-1~dotdeb.0 nginx-common/squeeze-backports *manually* upgradeable from 1.2.1-2~bpo60+1 to 1.2.6-1~dotdeb.0 nginx-full/squeeze-backports *manually* upgradeable from 1.2.1-2~bpo60+1 to 1.2.6-1~dotdeb.0
I don't think we don't want to switch from the squeeze-backports version of nginx to the dotdeb version so those updates can be ignored, for the geoip update:
dpkg -r geoip-database apt-get install geoip-database=20130108-1~bpo60+1
For libfreetype6 there doesn't seem to be a problem changing back to the squeeze version, so this was done:
dpkg -r --force-depends libfreetype6 aptitude install multiarch-support libfreetype6
The notes here were helpful.
Notes on this have been added to PenguinServer#Updates
comment:7 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 5.51 to 6.01
I have made a start on migrating the static archives:
Copy the files:
mkdir -p /web/2011.archive.transitionnetwork.org/www rsync -av kiwi:/web/2011.archive.transitionnetwork.org/www/ /web/2011.archive.transitionnetwork.org/www/ mkdir -p /web/2010.archive.transitionnetwork.org/www rsync -av kiwi:/web/transitiontowns.org.webarch.net/www/ /web/2010.archive.transitionnetwork.org/www/
Copy nginx config across:
cd /etc/nginx/sites-available rsync -av kiwi:/etc/nginx/sites-available/archive . cd /etc/nginx rsync -av kiwi:/etc/nginx/archive-shared .
These files contain, archive:
# virtual server
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name 2011.archive.transitionnetwork.org;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/archive.access.log;
error_log /var/log/nginx/archive.error.log debug;
# document root
# http://nginx.org/en/docs/http/ngx_http_core_module.html#root
root "/web/2011.archive.transitionnetwork.org/www";
# include the config that both archive sites share
include archive-shared;
}
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name 2010.archive.transitionnetwork.org;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/archive.access.log;
error_log /var/log/nginx/archive.error.log crit;
# document root
# http://nginx.org/en/docs/http/ngx_http_core_module.html#root
root "/web/2010.archive.transitionnetwork.org/www";
# include the config that both archive sites share
include archive-shared;
}
archive-shared:
# Shared archive config
# http://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex
autoindex on;
# document index
# http://nginx.org/en/docs/http/ngx_http_index_module.html#index
index archive.html index.php pmwiki.php;
# location match
# http://nginx.org/en/docs/http/ngx_http_core_module.html#location
# Prevent access to any files starting with a dot, like .htaccess
# or text editor temp files
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
# Prevent access to tmp files created by vim
location ~ .\~$ {
return 403;
}
# /uploads is the upload directory where users can upload files witrh extensions
# listed in LocalSettings.php so we want to make sure that html / php documents
# are served as plain text
location ^~ /uploads/ {
default_type application/octet-stream;
types {
text/plain htm html shtml php php5;
# whitelist here of all file types allowed to be uploaded,
image/gif gif;
image/jpeg jpeg jpg jpe;
image/png png;
text/plain txt;
application/msword doc dot;
application/rtf rtf;
application/pdf pdf;
application/vnd.oasis.opendocument.text odt;
application/vnd.ms-excel xls xlb xlt;
application/vnd.ms-powerpoint ppt pps;
}
}
# deny access to these directories and files
location ^~ /webprojectblog/ { return 403; }
location ^~ /scripts/ { return 403; }
location ^~ /cookbook/ { return 403; }
location ^~ /local/ { return 403; }
location ^~ /forum/bb-admin/ { return 403; }
#location ^~ /forum/bb-templates/ { return 403; }
location ^~ /forum/my-plugins/ { return 403; }
location = /forum/bb-config.php { return 403; }
location = /forum/bb-config-sample.php { return 403; }
location = /forum/bb-cron.php { return 403; }
location = /forum/bb-edit.php { return 403; }
location = /forum/bb-login.php { return 403; }
location = /forum/bb-post.php { return 403; }
location = /forum/bb-reset-password.php { return 403; }
location = /forum/bb-settings.php { return 403; }
location = /forum/edit.php { return 403; }
location = /forum/register.php { return 403; }
# no external access needed
#location ^~ /forum/bb-includes/ { internal; }
# Converted apache rules
# via http://www.anilcetin.com/convert-apache-htaccess-to-nginx/
location ~ /forum {
rewrite ^/forum/page/([0-9]+)/?$ /forum/index.php?page=$1 last;
rewrite ^/forum/forum/([^/]+)/page/([0-9]+)/?$ /forum/forum.php?id=$1&page=$2 last;
rewrite ^/forum/forum/([^/]+)/?$ /forum/forum.php?id=$1 last;
rewrite ^/forum/forum/?$ /forum/ redirect;
rewrite ^/forum/topic/([^/]+)/page/([0-9]+)/?$ /forum/topic.php?id=$1&page=$2 last;
rewrite ^/forum/topic/([^/]+)/?$ /forum/topic.php?id=$1 last;
rewrite ^/forum/topic/?$ /forum/ redirect;
rewrite ^/forum/tags/([^/]+)/page/([0-9]+)/?$ /forum/tags.php?tag=$1&page=$2 last;
rewrite ^/forum/tags/([^/]+)?$ /forum/tags.php?tag=$1 last;
rewrite ^/forum/profile/([^/]+)/page/([0-9]+)/?$ /forum/profile.php?id=$1&page=$2 last;
rewrite ^/forum/profile/([^/]+)/([^/]+)/?$ /forum/profile.php?id=$1&tab=$2 last;
rewrite ^/forum/profile/([^/]+)/([^/]+)/page/([0-9]+)/?$ /forum/profile.php?id=$1&tab=$2&page=$3 last;
rewrite ^/forum/profile/([^/]+)/?$ /forum/profile.php?id=$1 last;
rewrite ^/forum/profile/?$ /forum/profile.php last;
rewrite ^/forum/view/([^/]+)/page/([0-9]+)/?$ /forum/view.php?view=$1&page=$2 last;
rewrite ^/forum/view/([^/]+)/?$ /forum/view.php?view=$1 last;
rewrite ^/forum/rss/topic/([^/]+)?$ /forum/rss.php?topics=$1 last;
rewrite ^/forum/rss/tags/([^/]+)/?$ /forum/rss.php?tag=$1 last;
rewrite ^/forum/rss/tags/([^/]+)/topics/?$ /forum/rss.php?tag=$1&topics=1 last;
rewrite ^/forum/rss/profile/([^/]+)/?$ /forum/rss.php?profile=$1 last;
rewrite ^/forum/rss/view/([^/]+)/?$ /forum/rss.php?view=$1 last;
rewrite ^/forum/rss/?$ /forum/rss.php last;
#rewrite ^/forum/.*$ /forum/index.php last;
# Pass php requests to php5-fpm
location ~ \.php?$ {
try_files $uri =404;
include fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
}
}
# Prevent editing and uploading
# http://wiki.nginx.org/IfIsEvil - but not sure how else to do this
if ($args ~ "^(.*)action=edit"){
return 403;
}
if ($args ~ "^(.*)action=upload"){
return 403;
}
location /Totnes {
rewrite ^/Totnes/([^/a-z].*) /Totnes/index.php?n=$1;
}
location /Kinsale {
rewrite ^/Kinsale/([^/a-z].*) /Kinsale/index.php?n=$1;
}
location /Skibb {
rewrite ^/Skibb/([^/a-z].*) /Skibb/index.php?n=$1;
}
location / {
try_files $uri $uri/ @rewrite;
}
location @rewrite {
rewrite ^/([A-Z0-9xa0-xff].*)$ /pmwiki.php?n=$1;
}
# Pass php requests to php5-fpm
location ~ \.php?$ {
try_files $uri =404;
include fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
}
Copy the database across:
cd /tmp/ rsync -av kiwi:/var/backups/mysql/sqldump/bbforum.sql .
Create the database and user:
mysql mysql> create database bbforum; mysql> grant all on bbforum.* to 'bbforum'@'bbforum' identified by 'XXX'; mysql> flush privileges;
Insert the database:
cat /tmp/bbforum.sql | mysql bbforum
comment:8 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.5
- Total Hours changed from 6.01 to 7.51
The work last week resulted in static HTML working:
But not PHP:
This is because the socket specified in /etc/nginx/archive-shared:
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
Doesn't exist, so following ticket:357#comment:16
mkdir /var/run/php5-fpm chown www-data:www-data /var/run/php5-fpm
Change these things in /etc/php5/fpm/pool.d/www.conf:
listen = /var/run/php5-fpm/phpfpm.sock listen.owner = www-data listen.group = www-data listen.mode = 0666 listen.allowed_clients = 127.0.0.1,81.95.52.111,penguin.transitionnetwork.org,penguin.webarch.net
And restart apps:
/etc/init.d/php5-fpm restart /etc/init.d/nginx restart
Now PHP is working:
But MySQL is needed for the old forum:
And that wasn't working because, although the database had been imported a user hadn't been created correctly see ticket:470#comment:7 so this was done to fix it:
mysql mysql mysql> DROP USER 'bbforum'@'bbforum'; mysql> CREATE USER 'bbforum'@'localhost' IDENTIFIED BY 'xxx'; mysql> GRANT ALL ON bbforum.* TO 'bbforum'@'localhost'; mysql> FLUSH PRIVILEGES;
The Gandi DNS has been updated, we did have a *.archive entry but there isn't really a need for a wild card entry so it's been removed and replaced with:
2010.archive 3600 IN A 81.95.52.111 2011.archive 3600 IN A 81.95.52.111
And these site should soon be running on penguin:
comment:9 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 7.51 to 8.51
So, there are some issues with the archive, specifically images are not working on the 2010 site:
http://2010.archive.transitionnetwork.org/index.php
But they are working here:
http://2011.archive.transitionnetwork.org/index.php
I'm at a loss for the reason for this and having spent an hour on it I think I should move on, so I have created a seperate ticket for this, ticket:474
comment:10 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.3
- Total Hours changed from 8.51 to 9.81
Setting up Mediawiki, copy the nginx config from kiwi:
cd /etc/nginx/ scp kiwi:/etc/nginx/wiki-shared . cd /etc/nginx/sites-available/ scp kiwi:/etc/nginx/sites-available/wiki .
The /etc/nginx/wiki-shared file contains:
# Everything below is the same for http and https
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name wiki.transitionnetwork.org wiki.penguin.webarch.net;
# document root
# http://nginx.org/en/docs/http/ngx_http_core_module.html#root
root "/web/wiki.transitionnetwork.org/www";
# document index
# http://nginx.org/en/docs/http/ngx_http_index_module.html#index
index index.php;
# http://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex
autoindex off;
# location match
# http://nginx.org/en/docs/http/ngx_http_core_module.html#location
# Prevent access to any files starting with a dot, like .htaccess
# or text editor temp files
location ~ /\. {
return 403;
}
# Prevent access to tmp files created by vim
location ~ .\~$ {
return 403;
}
# Block access to these directories and files
location = /LocalSettings.php { return 403; }
location ^~ /maintenance/ { return 403; }
location ^~ /bin/ { return 403; }
location ^~ /mw-config/ { return 403; }
location ^~ /cache/ { return 403; }
location ^~ /tests/ { return 403; }
location ^~ /includes/ { return 403; }
location ^~ /maths/ { return 403; }
location ^~ /serialized/ { return 403; }
location ^~ /languages/ { return 403; }
# /images is the upload directory where users can upload files witrh extensions
# listed in LocalSettings.php so we want to make sure that html / php documents
# are served as plain text
location ^~ /images/ {
default_type text/plain;
types {
text/plain htm html shtml php php5;
# whitelist here of all file types allowed to be uploaded,
image/gif gif;
image/jpeg jpeg jpg jpe;
image/png png;
text/plain txt;
video/x-msvideo avi;
audio/mpeg mpga mpega mp2 mp3 m4a;
audio/x-pn-realaudio ra rm ram;
video/mp4 mp4;
image/svg+xml svg svgz;
application/pdf pdf;
application/rtf rtf;
application/msword doc dot;
application/vnd.ms-powerpoint ppt pps;
application/vnd.oasis.opendocument.chart odc;
application/vnd.oasis.opendocument.database odb;
application/vnd.oasis.opendocument.formula odf;
application/vnd.oasis.opendocument.graphics odg;
application/vnd.oasis.opendocument.graphics-template otg;
application/vnd.oasis.opendocument.image odi;
application/vnd.oasis.opendocument.presentation odp;
application/vnd.oasis.opendocument.presentation-template otp;
application/vnd.oasis.opendocument.spreadsheet ods;
application/vnd.oasis.opendocument.spreadsheet-template ots;
application/vnd.oasis.opendocument.text odt;
application/vnd.oasis.opendocument.text-master odm;
application/vnd.oasis.opendocument.text-template ott;
application/vnd.oasis.opendocument.text-web oth;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx;
application/vnd.openxmlformats-officedocument.spreadsheetml.template xltx;
application/vnd.openxmlformats-officedocument.presentationml.presentation pptx;
application/vnd.openxmlformats-officedocument.presentationml.slideshow ppsx;
application/vnd.openxmlformats-officedocument.presentationml.template potx;
application/vnd.openxmlformats-officedocument.wordprocessingml.document docx;
application/vnd.openxmlformats-officedocument.wordprocessingml.template dotx;
image/x-photoshop psd;
application/postscript ps ai eps epsi epsf eps2 eps3;
image/tiff tiff tif;
image/x-icon ico;
application/vnd.ms-excel xls xlb xlt;
}
# If the files can't be found serve up the front page
try_files $uri /index.php;
}
# This is the main rewrite to get nice URL's
# http://wiki.nginx.org/MediaWiki
location / {
try_files $uri @rewrite;
}
location @rewrite {
# http://blog.bigdinosaur.org/mediawiki-on-nginx/
rewrite ^/([^?]*)(?:\?(.*))? /index.php?title=$1&$2 last;
}
And the /etc/nginx/sites-available/wiki contains:
virtual server
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server
# HTTP Server
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/wiki.access.log;
error_log /var/log/nginx/wiki.error.log crit;
# Redirect logins to https
location ~ .*Special:UserLogin.* {
#rewrite ^/Special:UserLogin(.*) https://$server_name:4430/Special:UserLogin$1? redirect;
rewrite ^/Special:UserLogin(.*) https://$server_name/Special:UserLogin$1? redirect;
}
if ($args ~ "^(.*)title=Special:UserLogin"){
#rewrite ^/(.*) https://$server_name:4430/index.php?$args? redirect;
rewrite ^/(.*) https://$server_name/index.php?$args? redirect;
}
include wiki-shared;
# Pass php requests to php5-fpm
location ~ /(api|index|opensearch_desc|redirect|trackback|img_auth|load|thumb)\.php?$ {
try_files $uri =404;
include fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
}
}
# HTTPS Server
server {
#listen 4430;
listen 443;
access_log /var/log/nginx/wiki.ssl_access.log;
error_log /var/log/nginx/wiki.ssl_error.log crit;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4-SHA:HIGH:!ADH:!SSLv2:!aNULL;
ssl_prefer_server_ciphers on;
include wiki-shared;
# Pass php requests to php5-fpm
location ~ /(api|index|opensearch_desc|redirect|trackback|img_auth|load|thumb)\.php?$ {
try_files $uri =404;
include fastcgi_params;
fastcgi_param HTTPS on;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
}
}
Edited these files and changed wiki.dev to wiki.
Copy the live wiki off quince, and delete the things we don't need like old versions of mediawiki:
mkdir /web/wiki.transitionnetwork.org/ rsync -av quince:/web/wiki.transitionnetwork.org/ /web/wiki.transitionnetwork.org/ cd /web/wiki.transitionnetwork.org/ rm -rf mediawiki-1* rm -rf www/extensions/FCKeditor/
The last change on the wiki, https://wiki.transitionnetwork.org/Special:RecentChanges was Jim yesterday, so it's safe to use last nights backup:
cd /tmp/ rsync -av quince:/var/backups/mysql/sqldump/transwiki.sql . mysql mysql mysql> CREATE DATABASE transwiki; mysql> GRANT ALL ON transwiki.* TO 'mediawiki'@'localhost' IDENTIFIED BY 'xxxx'; mysql> FLUSH PRIVILEGES; mysql> QUIT; cat transwiki.sql | mysql transwiki cd /etc/nginx/sites-enabled ln -s ../sites-available/wiki 10-wiki cd /etc/ssl/transitionnetwork.org scp kiwi:/etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem . /etc/init.d/nginx restart
The nginx transitionnetwork.org.chained.pem was created like this:
cat transitionnetwork.org.crt > transitionnetwork.org.chained.pem cat GandiStandardSSLCA.pem >> transitionnetwork.org.chained.pem cat UTNAddTrustServer_CA.pem >> transitionnetwork.org.chained.pem cat AddTrustExternalCARoot.pem >> transitionnetwork.org.chained.pem
Following a nginx restart the following error is displayed at http://wiki.penguin.webarch.net/
CACHE_ACCEL requested but no suitable object cache is present. You may want to install APC.
So:
aptitude install php5-apc
That didn't fix it, and it turns out that an old DB dump had been imported, so importing a new one:
cd /tmp/ scp quince:/var/backups/mysql/bak.mysql.2013-01-21_02.00.16_.tar.gz . tar -zxvf bak.mysql.2013-01-21_02.00.16_.tar.gz cd bak.mysql.2013-01-21_02.00.16/ cat *transwiki* | mysql transwiki
Also as APC isn't working these settings were commented out in /web/wiki.transitionnetwork.org/www/LocalSettings.php to get the site working:
#$wgMainCacheType = CACHE_ACCEL; #$wgMemCachedServers = array();
The wiki is now running on the new server but there appears to be a problem with cookies and authentication:
So I haven't updated the DNS.
comment:11 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.3
- Total Hours changed from 9.81 to 10.11
Upgrading mediawiki:
cd /web/wiki.transitionnetwork.org/ wget http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.3.tar.gz wget http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.3.tar.gz.sig wget https://www.mediawiki.org/keys/keys.txt gpg --import keys.txt gpg --verify mediawiki-1.19.3.tar.gz.sig tar -zxvf mediawiki-1.19.3.tar.gz rsync -av mediawiki-1.19.3/ www/ cd www/maintenance/ php update.php cd .. chown root:root -R www/ cd www chown -R www-data:www-data cache chown -R www-data:www-data images
The cookie problem was fixed by editing LocalSettings.php:
#$wgCookieDomain = "wiki.transitionnetwork.org"; $wgCookieDomain = "wiki.penguin.webarch.net";
So now it's OK to switch the DNS -- please don't edit https://wiki.transitionnetwork.org/ for the next 30 mins or so!
comment:12 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 10.11 to 10.21
OK, DNS for wiki.transitionnetwork.org has been updated, you will know you have the site off penguin when this page:
https://wiki.transitionnetwork.org/User:Chris
Matches this page:
https://wiki.penguin.webarch.net/User:Chris
The documentation still needs updating, wiki:PenguinServer#Websites
comment:13 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.7
- Total Hours changed from 10.21 to 11.91
I've added some notes about Mediawiki, wiki:PenguinServer#wiki.transitionnetwork.org
Installing Piwik, copy the niginx config across and the files and delete some things we don't need:
cd /etc/niginx/sites-available scp kiwi:/etc/nginx/sites-available/stats . mkdir /web/stats.transitionnetwork.org cd /web/ rsync -av quince:/web/stats.transitionnetwork.org/ stats.transitionnetwork.org/ cd stats.transitionnetwork.org rm -rf piwik.1* rm -rf piwik.bak
Copy the database and install it, on quince, backup the databases, then copy across:
ssh quince /usr/local/bin/mysql_backup cd /tmp/ scp quince:/var/backups/mysql/bak.mysql.2013-01-21_19.41.42_.tar.gz . tar -zxvf bak.mysql.2013-01-21_19.41.42_.tar.gz cd bak.mysql.2013-01-21_19.41.42/ cat *_stats* | mysql stats
Create a Mysql user and database:
mysql mysql mysql> CREATE DATABASE stats; mysql> GRANT ALL ON stats.* TO 'stats'@'localhost' IDENTIFIED BY 'xxxx'; mysql> FLUSH PRIVILEGES; mysql> QUIT;
Nginx:
cd /etc/nginx scp kiwi:/etc/nginx/fastcgi_cache_zone.conf . mkdir -p /var/cache/nginx/fcgicache chown -R www-data:www-data /var/cache/nginx/fcgicache mv nginx.conf nginx.conf.2013-01021.bak scp kiwi:/etc/nginx/nginx.conf . scp kiwi:/etc/nginx/map_cache_piwik.conf . /etc/init.d/nginx restart
Upgrade Piwik, see http://piwik.org/docs/update/
cd /web/stats.transitionnetwork.org cp piwik/config/config.ini.php . wget http://builds.piwik.org/latest.zip unzip latest.zip cp config.ini.php piwik/config/ cd piwik/ php index.php -- "module=CoreUpdater"
All seems to be working OK at https://stats.penguin.webarch.net/ so the DNS has been updated and penguin should soon be serving https://stats.transitionnetwork.org/
(This comment includes the time spent in a ttech skype meeting -- I was doing the migration during the meeting.)
comment:14 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.3
- Total Hours changed from 11.91 to 12.21
Pwiki is now running on penguin, and it's tracking activity today, but there was no recorded traffic for yesterday, I think because the cron tab wasn't set up, so the crontab was created for www-data:
crontab -e -u www-data
Containing:
# m h dom mon dow command 5 0 * * * /web/stats.transitionnetwork.org/piwik/misc/cron/archive.sh > /dev/null
And the script was run on the command line.
comment:15 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.65
- Total Hours changed from 12.21 to 12.86
We now have stats for yesterday in Piwik and I have added stats.transitionnetwork.org to the wiki page, wiki:PenguinServer#stats.transitionnetwork.org and updated the Piwik page, wiki:PiwikServer
I have writtten a penguin TODO list, wiki:PenguinServer#Penguin which has these tasks listed:
- Sort out backups
- Migrate trac
- Migrate the static archive
- Check the mysql and php settings
- Sort out munin (or collect)
- Sort out the apt pinning - there are still some problems with this
comment:16 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.4
- Total Hours changed from 12.86 to 13.26
Backups have been setup using backupninja, the config is in /etc/backupninja.conf and it has the default settings and task config files are in /etc/backup.d/, /etc/backup.d/20.mysql contains:
hotcopy = no sqldump = yes compress = no backupdir = /var/backups/mysql databases = all configfile = /etc/mysql/debian.cnf
And /etc/backup.d/90.rdiff contains:
[source] type = local keep = 60D include = /var/spool/cron/crontabs include = /var/backups include = /etc include = /root include = /home include = /usr/local/*bin include = /var/lib/dpkg/status* include = /web exclude = /home/*/.gnupg exclude = /home/*/.local/share/Trash exclude = /home/*/.Trash exclude = /home/*/.thumbnails exclude = /home/*/.beagle exclude = /home/*/.aMule exclude = /home/*/gtk-gnutella-downloads exclude = /var/cache/backupninja/duplicity [dest] type = remote directory = /var/lib/backup/penguin/penguin host = backup user = penguin
The ssh host details are set in /root/.ssh/config.
The biggest things that is currently backed up is the Pwiki database, it's 1.6G when dumped as plain text.
A test backup run has been run.
comment:17 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.8
- Total Hours changed from 13.26 to 14.06
Migrating http://static.transitionnetwork.org/ first copy the archive across:
mkdir /web/static.transitionnetwork.org rsync -av quince:/web/static.transitionnetwork.org/ /web/static.transitionnetwork.org/
It is 1.9G in total:
816M ./ttcon2011 201M ./ttcon2010 876M ./tnconf2012
Copy the static nginx config from kiwi:
/etc/nginx/sites-available scp kiwi:/etc/nginx/sites-available/static .
After editing this is what we have:
# static.transitionnetwork.org http virtual server
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name static.transitionnetwork.org static.penguin.webarch.net;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/static.access.log;
error_log /var/log/nginx/static.error.log crit;
# document root
# http://nginx.org/en/docs/http/ngx_http_core_module.html#root
root "/web/static.transitionnetwork.org/www";
# http://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex
autoindex on;
# document index
# http://nginx.org/en/docs/http/ngx_http_index_module.html#index
index index.html;
# location match
# http://nginx.org/en/docs/http/ngx_http_core_module.html#location
# If the most specific prefix location has the "^~" prefix then regular expressions are not checked.
# redirect all these to the https version
# http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite
# Some private data
location ^~ /ttcon2010/private {
#rewrite ^/(.*)$ https://static.transitionnetwork.org:4430/$1? permanent;
rewrite ^/(.*)$ https://static.transitionnetwork.org/$1? permanent;
}
location ^~ /tnconf2012/emails/ {
#rewrite ^/(.*)$ https://static.transitionnetwork.org:4430/$1? permanent;
rewrite ^/(.*)$ https://static.transitionnetwork.org/$1? permanent;
}
# Prevent access to any files starting with a dot, like .htaccess
# or text editor temp files
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
}
# static.transitionnetwork.org https virtual server
server {
#listen 4430;
listen 443;
server_name static.transitionnetwork.org static.penguin.webarch.net;
access_log /var/log/nginx/static.ssl_access.log;
error_log /var/log/nginx/static.ssl_error.log debug;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4-SHA:HIGH:!ADH:!SSLv2:!aNULL;
ssl_prefer_server_ciphers on;
root "/web/static.transitionnetwork.org/www";
autoindex on;
index index.html;
# if a "/" request happens frequently, defining "location = /" will speed up the processing of these
# requests, as search terminates right after the first comparison
location /ttcon2010/private {
satisfy any;
deny all;
# http://nginx.org/en/docs/http/ngx_http_auth_basic_module.html
auth_basic "Private Area";
auth_basic_user_file /web/static.transitionnetwork.org/.htpasswd;
}
location /tnconf2012/emails {
satisfy any;
deny all;
# http://nginx.org/en/docs/http/ngx_http_auth_basic_module.html
auth_basic "Private Area";
auth_basic_user_file /web/static.transitionnetwork.org/.htpasswd;
}
# Prevent access to any files starting with a dot, like .htaccess
# or text editor temp files
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
}
Enable it for testing:
cd /etc/nginx/sites-enabled/ ln -s ../sites-available/static 25-static /etc/init.d/nginx configtest /etc/init.d/nginx restart
After testing the DNS was updated and soon https://static.transitionnetwork.org/ will be running via Nginx.
The Penguin wiki page has been updated wiki:PenguinServer#static.transitionnetwork.org
comment:18 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.6
- Total Hours changed from 14.06 to 14.66
Static splash page for http://penguin.webarch.net/ and http://penguin.transitionnetwork.org/ and http://81.95.52.111/
Copying what we have on kiwi and editing files to suit.
mkdir -p /web/penguin.transitionnetwork.org/www rsync -av kiwi:/web/kiwi.webarch.net/www/ /web/penguin.transitionnetwork.org/www/ cd /etc/nginx/sites-available scp kiwi:/etc/nginx/sites-available/kiwi penguin cd /etc/nginx/sites-enabled ln -s ../sites-available/penguin 05-penguin /etc/init.d/nginx configtest /etc/init.d/nginx restart
Now we have a static splash page for the server: https://penguin.transitionnetwork.org/ but one thing to look at, is how to make the IP address serve the first virtual host rather than the last:
comment:19 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.2
- Total Hours changed from 14.66 to 14.86
To make the http://penguin.transitionnetwork.org/ site the default we need to add the IP address to the server_name and default_server to the listen directive:
listen 80 default_server; server_name penguin.transitionnetwork.org penguin.webarch.net 81.95.52.111;
Now we get the right page at these locations:
But default_server should probably be used for the redirects, see wiki:NewLiveServer#Redirects
comment:20 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.12
- Total Hours changed from 14.86 to 14.98
Install NTP:
aptitude install ntp
logchange "libopts25{a} ntp : installed"
comment:21 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.25
- Total Hours changed from 14.98 to 16.23
Domains which are currently redirected by apache, see wiki:NewLiveServer#Redirects
Rewriting for nginx.
cd /etc/nginx/sites-available vim redirects
all the domains listed on quince were checked, some have not been renewed and these were omitted (the wiki:DomainNames page needs updating), the redirects file contains:
# redirects
server {
listen 80;
server_name transition-japan.org www.transition-japan.org;
rewrite ^ http://www.transition-japan.net/ permanent;
}
server {
listen 80;
server_name transitiontowns.org www.transitiontowns.org;
rewrite ^/Bellingen(.*)$ http://www.transitionnetwork.org/cms/bellingen permanent;
rewrite ^/Lewes(.*)$ http://www.transitiontownlewes.org/ permanent;
rewrite ^/Totnes(.*) http://www.transitiontowntotnes.org/ permanent;
rewrite ^/Brixton(.*) http://www.transitiontownbrixton.org/ permanent;
rewrite ^ http://www.transitionnetwork.org$request_uri? permanent;
}
server {
listen 80;
server_name canbritainfeeditself.org www.canbritainfeeditself.org
energydescentactionplan.org www.energydescentactionplan.org
relocalisationmovement.org www.relocalisationmovement.org
relocalisationparty.org www.relocalisationparty.org
relocalizationmovement.org www.relocalizationmovement.org
relocalizationnetwork.com www.relocalizationnetwork.com
relocalizationnetwork.org www.relocalizationnetwork.org
relocalizationparty.org www.relocalizationparty.org
resiliencebuilders.com www.resiliencebuilders.com
resiliencebuilders.org www.resiliencebuilders.org
transitionamerica.org www.transitionamerica.org
transitionaustralia.org www.transitionaustralia.org
transitionbooks.org www.transitionbooks.org
transitionbusiness.org www.transitionbusiness.org
transitioncommunities.org www.transitioncommunities.org
transitionfood.org www.transitionfood.org
transitionireland.org www.transitionireland.org
transitionjournal.org www.transitionjournal.org
transitionmedia.org www.transitionmedia.org
transitionmentoring.org.uk www.transitionmentoring.org.uk
transitionnetwork.org.uk www.transitionnetwork.org.uk
transitionnetwork.co.uk www.transitionnetwork.co.uk
transitionnetworknews.com www.transitionnetworknews.com
transitionnetworknews.org www.transitionnetworknews.org
transitionparty.org www.transitionparty.org
transitionradio.org www.transitionradio.org
transitiontimesonline.org www.transitiontimesonline.org
transitiontoday.org www.transitiontoday.org
transitiontours.com www.transitiontours.com
transitiontours.org www.transitiontours.org
transitiontown.com www.transitiontown.com
transitiontown.net www.transitiontown.net
transitiontown.org www.transitiontown.org
transitionvideo.org www.transitionvideo.org
transitionwales.org www.transitionwales.org;
rewrite ^ http://www.transitionnetwork.org$request_uri? permanent;
}
The redirects were enabled:
cd /etc/nginx/sites-enabled ln -s ../sites-available/redirects 05-redirects /etc/init.d/nginx configtest Testing nginx configuration: nginx: [emerg] could not build the server_names_hash, you should increase either server_names_hash_max_size: 512 or server_names_hash_bucket_size: 64 nginx: configuration file /etc/nginx/nginx.conf test failed
Due to the large number of server names this needed adding to /etc/nginx/nginx.conf:
server_names_hash_max_size 1024;
See http://nginx.org/en/docs/http/server_names.html
The Gandi "Transition Redirects" redirects zone file was going to be updated, bt it's owned by Ed and I don't have permission to update it, Gandi also has a list of all domains using this file, some of them have just expired and some are due to expire soon:
transitionplaces.org 2013-01-17 localtransition.org 2013-01-17 transitionmentoring.org.uk 2013-01-18 transitionmentoring.org 2013-01-18 relocalizationnetwork.org 2013-02-25 relocalizationmovement.org 2013-02-25 relocalizationnetwork.com 2013-02-25 relocalisationmovement.org 2013-02-25 transitioncommunities.org 2013-03-25 transitiontown.org 2013-04-07 transitiontown.net 2013-04-07 transitiontown.com 2013-04-07 resiliencebuilders.org 2013-06-09 transitionbooks.org 2013-06-09 resiliencebuilders.com 2013-06-09 transitiontimesonline.org 2013-08-06 transitionjournal.org 2013-08-07 transitiontoday.org 2013-08-07 transitionparty.org 2013-08-17 relocalisationparty.org 2013-08-17 relocalizationparty.org 2013-08-17 transitiontours.org 2013-08-18 transitiontours.com 2013-08-18 transition-japan.org 2013-09-01 canbritainfeeditself.org 2013-09-04 energydescentactionplan.org 2013-10-02 transitionnetworknews.com 2013-10-17 transitionnetworknews.org 2013-10-17 transitionmedia.org 2013-10-20 transitionbusiness.org 2013-11-14 transitionfood.org 2013-11-14 transitionvideo.org 2013-11-17 transitionradio.org 2013-11-17 transitionireland.org 2013-11-17 transitionaustralia.org 2013-11-17 transitionwales.org 2013-11-17 transitionamerica.org 2014-01-17 transitionnetwork.org.uk 2014-01-25
This is a list of them in alphabetical order:
canbritainfeeditself.org energydescentactionplan.org localtransition.org relocalisationmovement.org relocalisationparty.org relocalizationmovement.org relocalizationnetwork.com relocalizationnetwork.org relocalizationparty.org resiliencebuilders.com resiliencebuilders.org transitionamerica.org transitionaustralia.org transitionbooks.org transitionbusiness.org transitioncommunities.org transitionfood.org transitionireland.org transition-japan.org transitionjournal.org transitionmedia.org transitionmentoring.org transitionmentoring.org.uk transitionnetworknews.com transitionnetworknews.org transitionnetwork.org.uk transitionparty.org transitionplaces.org transitionradio.org transitiontimesonline.org transitiontoday.org transitiontours.com transitiontours.org transitiontown.com transitiontown.net transitiontown.org transitionvideo.org transitionwales.org
I need to check these are all in the ngnix config and also give Ed the details he need to update the zone file, but first I need to eat.
comment:22 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 16.23 to 16.48
The domains added to the redirect Nginx config:
localtransition.org transitionmentoring.org transitionplaces.org
The only domain in the Ngnix config but not using the Gandi "Transition Redirects" zone file is:
transitionnetwork.co.uk
I'm not a tech contact for this domain and it's pointing to the quince IP address -- Ed it needs to be changed to use the "Transition Redirects" zone file if you have permission to do that (make me tech contact if you want then I should be able to do it).
Ed, can you also copy the "Transition Redirects" zone file, edit it to this:
@ 10800 IN A 81.95.52.111 www 10800 IN A 81.95.52.111
Thanks.
comment:23 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 16.48 to 16.98
Ed, the "Transition Towns" zone file also need editing, from:
@ 10800 IN A 81.95.52.88 totnes 10800 IN CNAME redirects.transitionnetwork.org. www 10800 IN CNAME redirects.transitionnetwork.org.
To:
@ 10800 IN A 81.95.52.111 totnes 10800 IN A 81.95.52.111 www 10800 IN A 81.95.52.111
You should be able to do this here: https://www.gandi.net/admin/domain/zone/470186/2/edit first create a new version, then activate it.
(These seems little point in using a CNAME if we also have a IP address in the file in any case.)
I have created an additional virtual host for totnes.transitiontowns.org:
server {
listen 80;
server_name totnes.transitiontowns.org;
rewrite ^ http://www.transitiontowntotnes.org$request_uri? permanent;
}
Though I wonder if it shouldn't point to the archive of the site rather than the latest version:
comment:24 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 16.98 to 17.48
Looking at how to run trac with nginx have read these pages and searched for other useful resources:
comment:25 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 17.48 to 17.98
More here:
I think it makes sense to first try to setup Trac with Fast CGI:
And if that doesn't work (for example I can't get it to serve trac with the existing URLs) then use tracd, and nginx as a reverse proxy:
See also the notes here:
comment:26 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.45
- Total Hours changed from 17.98 to 18.43
I have copied the "Transition Redirects" zone to "Transition Redirects (2013-01-23)" and made it active, it contains:
@ 10800 IN A 81.95.52.111 www 10800 IN A 81.95.52.111
The "Transition Towns" domain has been copied to "Transition Towns (2013-01-23)" and made active, this contains:
@ 10800 IN A 81.95.52.111 totnes 10800 IN A 81.95.52.111 www 10800 IN A 81.95.52.111
Once the gandi DNS servers have updated all the redirects will be running off penguin.
comment:27 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 18.43 to 19.43
Looking at installing Trac on penguin, http://trac.edgewall.org/wiki/TracOnDebian#AdditionalTracfiles reading the documentation available via zless /usr/share/doc/trac/README.Debian.gz and the documentation of the existing install, wiki:DevelopmentServer#Trac the initial install was done on ticket:1 and the upgrade to 0.12 on ticket:364, there are migration notes here, http://trac.edgewall.org/wiki/TracMigrate#TracMigration
On kiwi, dump the database:
cd /web/tech.transitionnetwork.org/trac/db sqlite3 trac.db ".dump" >> ~/sqlite_trac.sql
On penguin, copy the database across (this will need to be done again after testing):
aptitude install sqlite3 logchange "sqlite3 : installed" mkdir -p /web/tech.transitionnetwork.org/trac/db cd /web/tech.transitionnetwork.org/ scp kiwi:sqlite_trac.sql . cat sqlite_trac.sql | sqlite3 trac.db mv trac.db /web/tech.transitionnetwork.org/trac/db/ chown -R www-data:www-data /web/tech.transitionnetwork.org/trac/db/
Copy attachments across, on penguin (this will need to be done again after testing):
mkdir -p /web/tech.transitionnetwork.org/trac/attachments/ rsync -av kiwi:/web/tech.transitionnetwork.org/trac/attachments/ /web/tech.transitionnetwork.org/trac/attachments/ chown -R www-data:www-data /web/tech.transitionnetwork.org/trac/attachments/
Install a Trac environment:
trac-admin /web/tech.transitionnetwork.org/trac initenv Initenv for '/web/tech.transitionnetwork.org/trac' failed. Directory exists and is not empty.
Clearly this should have been done at the start, so, move it out of the way and try again:
mv /web/tech.transitionnetwork.org/trac/ /web/tech.transitionnetwork.org/trac.bak/
trac-admin /web/tech.transitionnetwork.org/trac initenv
Trac will first ask a few questions about your environment
in order to initialize and prepare the project database.
Please enter the name of your project.
This name will be used in page titles and descriptions.
Project Name [My Project]> Transition Technology
Please specify the connection string for the database to use.
By default, a local SQLite database is created in the environment
directory. It is also possible to use an already existing
PostgreSQL database (check the Trac documentation for the exact
connection string syntax).
Database connection string [sqlite:db/trac.db]>
Creating and Initializing Project
Installing default wiki pages
WikiStart imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiStart
InterWiki imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/InterWiki
TracRevisionLog imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracRevisionLog
WikiFormatting imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiFormatting
SandBox imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/SandBox
TracQuery imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracQuery
TracNotification imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracNotification
TracAccessibility imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracAccessibility
TracUnicode imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracUnicode
InterMapTxt imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/InterMapTxt
TracBackup imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracBackup
PageTemplates imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/PageTemplates
TracModWSGI imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracModWSGI
TracBrowser imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracBrowser
TracSupport imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracSupport
WikiDeletePage imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiDeletePage
TracNavigation imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracNavigation
TracPermissions imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracPermissions
WikiNewPage imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiNewPage
WikiRestructuredText imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiRestructuredText
TracTimeline imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracTimeline
WikiHtml imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiHtml
TracInstall imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracInstall
WikiProcessors imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiProcessors
TracReports imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracReports
TracTicketsCustomFields imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracTicketsCustomFields
TracRoadmap imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracRoadmap
TracGuide imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracGuide
TracWorkflow imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracWorkflow
TracEnvironment imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracEnvironment
TracFastCgi imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracFastCgi
InterTrac imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/InterTrac
TracSearch imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracSearch
TracTickets imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracTickets
TracRss imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracRss
TitleIndex imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TitleIndex
TracWiki imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracWiki
WikiPageNames imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiPageNames
RecentChanges imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/RecentChanges
WikiRestructuredTextLinks imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiRestructuredTextLinks
TracFineGrainedPermissions imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracFineGrainedPermissions
TracInterfaceCustomization imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracInterfaceCustomization
TracLinks imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracLinks
TracLogging imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracLogging
CamelCase imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/CamelCase
TracChangeset imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracChangeset
TracImport imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracImport
WikiMacros imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/WikiMacros
TracCgi imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracCgi
TracPlugins imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracPlugins
TracUpgrade imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracUpgrade
TracModPython imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracModPython
TracRepositoryAdmin imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracRepositoryAdmin
TracStandalone imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracStandalone
TracAdmin imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracAdmin
TracIni imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracIni
TracSyntaxColoring imported from /usr/lib/python2.6/dist-packages/trac/wiki/default-pages/TracSyntaxColoring
---------------------------------------------------------------------
Project environment for 'Transition Technology' created.
You may now configure the environment by editing the file:
/web/tech.transitionnetwork.org/trac/conf/trac.ini
If you'd like to take this new project environment for a test drive,
try running the Trac standalone web server `tracd`:
tracd --port 8000 /web/tech.transitionnetwork.org/trac
Then point your browser to http://localhost:8000/trac.
There you can also browse the documentation for your installed
version of Trac, including information on further setup (such as
deploying Trac to a real web server).
The latest documentation can also always be found on the project
website:
http://trac.edgewall.org/
Congratulations!
The copy across the database and attachments:
cd /web/tech.transitionnetwork.org cp trac.bak/db/trac.db trac/db/ chown www-data:www-data trac/db/trac.db rsync -av trac.bak/attachments/ trac/attachments/ chown -R www-data:www-data trac/attachments/
Test using tracd:
tracd --port 8000 /web/tech.transitionnetwork.org/trac
Everything is looked OK at http://penguin.transitionnetwork.org:8000/trac apart from the plugins and config changes, I'll look at importing them next, also the simplest thing might be to use the tracd server and nginx as a reverse proxy.
comment:28 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 19.43 to 20.43
Sorting out the trac.ini file, copy the kiwi one to penguin and diff it with the new one:
cd /web/tech.transitionnetwork.org mkdir trac.bak/conf/ scp kiwi:/web/tech.transitionnetwork.org/trac/conf/trac.ini trac.bak/conf/ diff trac.bak/conf/trac.ini trac/conf/trac.ini | vim -
Lots of things will need manually working though, but first the wiki:TimingAndEstimationPlugin
cd /web/tech.transitionnetwork.org svn co http://trac-hacks.org/svn/timingandestimationplugin/branches/trac0.12/ mv trac0.12/ timingandestimationpluin_trac0.12/ cd timingandestimationpluin_trac0.12/ python setup.py bdist_egg cp dist/timingandestimationplugin-1.2.8-py2.6.egg ../trac/plugins/
Add the following to /web/tech.transitionnetwork.org/trac/conf/trac.ini
[components] timingandestimationplugin.* = enabled
Enable the logo, previously this was in a site directory outside the trac directory, see http://trac.edgewall.org/wiki/0.12/TracInstall#MappingStaticResources for the reasons why, however if we use tracd this isn't a concern, so:
cd /web/tech.transitionnetwork.org/trac scp kiwi:/web/tech.transitionnetwork.org/www/site/TransitionNetwork-Logo-Web-Small.jpg .
Edit trac.ini changing:
[header_logo] alt = TransitionNetwork.org Technology link = /trac/ src = /trac/TransitionNetwork-Logo-Web-Small.jpg [logging] log_type = syslog log_level = ERROR [notification] admit_domains = transitionnetwork.org, trac.transitionnetwork.org, tech.transitionnetwork.org always_notify_owner = true always_notify_reporter = true mime_encoding = base64 sendmail_path = /usr/sbin/sendmail smtp_default_domain = transitionnetwork.org smtp_enabled = true smtp_from = trac@transitionnetwork.org smtp_from_name = Transiton Technology Trac smtp_replyto = trac@transitionnetwork.org [project] descr = Support and issues tracking for the Transition Network Web Project. [ticket] default_component = Drupal modules & settings [ticket-custom] billable = checkbox billable.label = Billable? billable.order = 3 billable.value = 1 estimatedhours = text estimatedhours.label = Estimated Number of Hours estimatedhours.order = 1 estimatedhours.value = 0 hours = text hours.label = Add Hours to Ticket hours.order = 2 hours.value = 0 totalhours = text totalhours.label = Total Hours totalhours.order = 4 totalhours.value = 0 [trac] base_url = https://tech.transitionnetwork.org/trac
The test the changes:
tracd --port 8000 /web/tech.transitionnetwork.org/trac
The results in the following at http://penguin.transitionnetwork.org:8000/
trac: Error (The Trac Environment needs to be upgraded. Run "trac-admin /web/tech.transitionnetwork.org/trac upgrade")
So:
trac-admin /web/tech.transitionnetwork.org/trac upgrade Timing and Estimation needs an upgrade Upgrading Database Upgrading reports Upgrading usermanual Done Upgrading Upgrade done. You may want to upgrade the Trac documentation now by running: trac-admin /web/tech.transitionnetwork.org/trac wiki upgrade
Things that need sorting out with trac:
- Authentication, see http://penguin.transitionnetwork.org:8000/trac/login
- The logo is a 404
- The Administration section, /trac/admin is missing
- After authentication is sorted the timing plugin needs checking
- Nginx, reverse proxy to tracd
- Tracd init script, see http://www.guyrutenberg.com/2008/06/04/start-trac-on-startup-initd-script-for-tracd/
- A check needs to be done for charset issues, the site was default_charset = iso-8859-15 (which is ISO Latin 1 with a few different characters, see https://en.wikipedia.org/wiki/ISO/IEC_8859-15#Changes_from_ISO-8859-1 ) and now it's default_charset = utf-8
- General testing and checking
comment:29 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.2
- Total Hours changed from 20.43 to 20.63
There are some tracd notes here:
- http://security.osmocom.org/trac/wiki/TracStandalone including notes on http://security.osmocom.org/trac/wiki/TracStandalone#UsingAuthentication and http://security.osmocom.org/trac/wiki/TracStandalone#Servingstaticcontent
Following the notes there to fix the logo issue:
cd /web/tech.transitionnetwork.org/trac mv TransitionNetwork-Logo-Web-Small.jpg htdocs/
Edit conf/trac.ini:
src = /trac/chrome/site/TransitionNetwork-Logo-Web-Small.jpg
And now the image works.
More authentication notes here:
comment:30 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 3.5
- Total Hours changed from 20.63 to 24.13
I'M ABOUT TO MIGRATE TRAC, PLEASE DON'T POST ANY TICKETS, COMMENTS TO TICKETS OR EDIT ANY WIKI PAGES FOR AN HOUR OR SO!
Ta!
Copy nginx reverse proxy config off kiwi and onto penguin:
cd /etc/nginx/sites-available scp kiwi:/etc/nginx/sites-available/tech .
Adding a system user for tracd:
addgroup --system tracd adduser --system --ingroup=tracd --no-create-home --disabled-password tracd
The HOME directory was then set in /etc/passwd to /web/tech.transitionnetwork.org and a directory for the PID was created:
mkdir /var/run/tracd chown tracd:tracd /var/run/tracd chown tracd:tracd /web/tech.transitionnetwork.org/trac -R
A directory for a script to start and a script to stop the server was created, following the suggestions here http://www.guyrutenberg.com/2009/02/17/starting-tracd-without-root-privileges-at-startup/
mkdir /web/tech.transitionnetwork.org/bin
And these scripts were created, tracd-start:
#!/bin/bash
/usr/bin/tracd --daemonize \
--pidfile=/var/run/tracd/tracd.pid \
--port=8080 \
--hostname=127.0.0.1 \
--basic-auth="trac,/web/tech.transitionnetwork.org/.htpasswd,Restricted" \
/web/tech.transitionnetwork.org/trac
And tracd-stop:
#!/bin/bash kill `cat /var/run/tracd/tracd.pid`
And this was added to the tracd crontab:
@reboot /web/tech.transitionnetwork.org/bin/tracd-start
Fixing the favicon.ico, aliased in nginx:
mkdir /web/tech.transitionnetwork.org/www/ scp kiwi:/web/tech.transitionnetwork.org/www/favicon.ico /web/tech.transitionnetwork.org/www/
And in trac.ini:
[project] icon = /favicon.ico
There we a lot of python egg cache errors that were solved (after a lot of faffing) by:
cd /web/tech.transitionnetwork.org ln -s trac/.python-eggs echo "export PYTHON_EGG_CACHE=~/trac/.python-eggs" >> /web/tech.transitionnetwork.org/.bashrc
The nginx config for the proxy:
# tech.transitionnetwork.org
# virtual server
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name trac.transitionnetwork.org tech.transitionnetwork.org tech.penguin.webarch.net;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/tech.access.log;
error_log /var/log/nginx/tech.error.log crit;
# Redirect to HTTPS
location / {
#rewrite ^/(.*)$ https://tech.transitionnetwork.org:4430/$1 permanent;
rewrite ^/(.*)$ https://$server_name/$1 permanent;
}
}
# HTTPS server
#
server {
#listen 4430;
listen 443;
server_name trac.transitionnetwork.org tech.transitionnetwork.org tech.penguin.webarch.net;
access_log /var/log/nginx/tech.ssl_access.log;
error_log /var/log/nginx/tech.ssl_error.log crit;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4-SHA:HIGH:!ADH:!SSLv2:!aNULL;
ssl_prefer_server_ciphers on;
# Redirect requests for / to /trac
location = / {
rewrite ^/$ https://$server_name/trac;
}
# favicon
location = /favicon.ico {
alias /web/tech.transitionnetwork.org/www/favicon.ico;
}
# http://trac.edgewall.org/wiki/TracNginxRecipe
add_header Front-End-Https on;
# http://wiki.nginx.org/LikeApache
#location ~ /trac(?<path_info>/.*|$) {
location /trac {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://localhost:8080/trac;
}
}
Getting the authentication working took some time but it's now all working here:
https://tech.penguin.webarch.net/trac
So after posting this ticket I'm going to do a final sync of the attachments and database and then update the dns and then check for any things that haven't been migrated off kiwi -- I think this is the last thing to move.
comment:31 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.2
- Total Hours changed from 24.13 to 24.33
- Description modified (diff)
OK, the DNS has been updated, tech.transitionnetwork.org and trac.transitionnetwork.org point here, I have also removed the dev.transitionnetwork.org and test.transitionnetwork.org domains so the only domain pointing to kiwi still is kiwi.transitionnetwork.org and I think that can probably be removed.
The documentation has been updated here wiki:PenguinServer#tech.transitionnetwork.org
The gandi DNS server are updating but are not quite there yet:
dig @A.DNS.GANDI.NET tech.transitionnetwork.org ;; ANSWER SECTION: tech.transitionnetwork.org. 3600 IN A 81.95.52.111 dig @B.DNS.GANDI.NET tech.transitionnetwork.org ;; ANSWER SECTION: tech.transitionnetwork.org. 3600 IN A 81.95.52.78
In the meantime you can use this domain: https://tech.penguin.webarch.net/trac
comment:32 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.22
- Total Hours changed from 24.33 to 24.55
I have quickly added Piwik stats to Mediawiki:
cd /web/wiki.transitionnetwork.org/www/extensions git clone git://github.com/DaSchTour/piwik-mediawiki-extension.git mv piwik-mediawiki-extension Piwik
Thje site was added to Piwik and then this was then added to /web/wiki.transitionnetwork.org/www/LocalSettings.php:
require_once( "$IP/extensions/Piwik/Piwik.php" ); $wgGroupPermissions['bureaucrat']['viewpiwik'] = true; $wgPiwikURL = "stats.transitionnetwork.org/"; $wgPiwikIDSite = "3";
It might be also worth adding the stats code to the various archives (and removing the google tracking) and also to this Trac site, but all these things would take longer than this quick fix for Mediawiki.
comment:33 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 24.55 to 24.8
In Piwik there is this warning:
Warning: We've detected the old GeoIP plugin. GeoIP integration is now in Piwik core and this plugin is considered deprecated. New region and city reports will not be shown while this plugin is loaded. Please disable the plugin and configure GeoIP. If you want location data for your old visits, use the script described here then reprocess your reports.
So, following http://piwik.org/faq/how-to/#faq_166
cd /etc/nginx mkdir geoip wget "http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz" wget "http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz" gunzip GeoLiteCity.dat.gz gunzip GeoIP.dat.gz v GeoLiteCity.dat GeoIPCity.dat
Edit /etc/nginx/nginx.conf and add:
geoip_country /etc/nginx/geoip/GeoIP.dat; geoip_city /etc/nginx/geoip/GeoIPCity.dat;
Edit /etc/nginx/fastcgi_params and add:
fastcgi_param GEOIP_ADDR $remote_addr; fastcgi_param GEOIP_COUNTRY_CODE $geoip_country_code; fastcgi_param GEOIP_COUNTRY_NAME $geoip_country_name; fastcgi_param GEOIP_REGION $geoip_region; fastcgi_param GEOIP_REGION_NAME $geoip_region_name; fastcgi_param GEOIP_CITY $geoip_city; fastcgi_param GEOIP_AREA_CODE $geoip_area_code; fastcgi_param GEOIP_LATITUDE $geoip_latitude; fastcgi_param GEOIP_LONGITUDE $geoip_longitude; fastcgi_param GEOIP_POSTAL_CODE $geoip_postal_code;
Restart Nginx:
/etc/init.d/nginx configtest /etc/init.d/nginx restart
Note the warning here, http://dev.maxmind.com/geoip/geolite
Keep in mind that geolocation data will eventually become out of date. It is recommended you update your databases at least once a month.* The free databases are updated once a month on the first Tuesday.
comment:34 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 24.8 to 25.05
The Piwik Nginx config hasn't been documented here and I have just updated the wiki:PiwikServer page.
The /etc/nginx/sites-available/stats file contains:
# piwik.transitionnetwork.org
# virtual server
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server
server {
# listen for ipv4
# http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
#listen 8000;
listen 80;
# server name and server aliases
# http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
server_name stats.transitionnetwork.org stats.penguin.webarch.net;
# logs, error log levels: info | notice | warn | error | crit | alert
# http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
# http://nginx.org/en/docs/ngx_core_module.html#error_log
access_log /var/log/nginx/piwik.access.log;
error_log /var/log/nginx/piwik.error.log crit;
# Redirect to HTTPS
rewrite ^ https://stats.transitionnetwork.org$request_uri? permanent;
}
# HTTPS server
#
server {
#listen 4430;
listen 443;
server_name stats.transitionnetwork.org stats.penguin.webarch.net;
access_log /var/log/nginx/piwik.ssl_access.log;
error_log /var/log/nginx/piwik.ssl_error.log crit;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4-SHA:HIGH:!ADH:!SSLv2:!aNULL;
ssl_prefer_server_ciphers on;
root "/web/stats.transitionnetwork.org/piwik";
index index.php;
# http://wiki.nginx.org/Piwik
# Disable all methods besides HEAD, GET and POST.
if ($request_method !~ ^(GET|HEAD|POST)$ ) {
return 444;
}
## Try all locations and relay to index.php as a fallback.
location / {
try_files $uri /index.php?$query_string;
}
## Relay all index.php requests to fastcgi.
location = /index.php {
include fastcgi_params;
fastcgi_param HTTPS on;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
### Implementation of the microcache concept as presented here:
### http://fennb.com/microcaching-speed-your-app-up-250x-with-no-n
## FastCGI cache.
## cache ui for 5m (set the same interval of your crontab)
## The cache zone referenced.
fastcgi_cache fcgicache;
## The cache key.
fastcgi_cache_key $host$request_uri;
## For 200 and 301 make the cache valid for 5 minutes.
fastcgi_cache_valid 200 301 5m;
## For 302 make it valid for 3 minutes.
fastcgi_cache_valid 302 3m;
## For 404 make it valid 1 minute.
fastcgi_cache_valid 404 1m;
## If there are any upstream errors or the item has expired use
## whatever it is available.
fastcgi_cache_use_stale error timeout invalid_header updating http_500;
## The Cache-Control and Expires headers should be delivered untouched
## from the upstream to the client.
fastcgi_ignore_headers Cache-Control Expires;
## If we have a cookie we should bypass the cache. The same if we have a
fastcgi_cache_bypass $no_cache;
fastcgi_no_cache $no_cache;
## Add a cache miss/hit status header.
add_header X-Piwik-Cache $upstream_cache_status;
## To avoid any interaction with the cache control headers we expire
## everything on this location immediately.
expires epoch;
}
## Relay all piwik.php requests to fastcgi.
location = /piwik.php {
include fastcgi_params;
fastcgi_param HTTPS on;
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
### Implementation of the microcache concept as presented here:
### http://fennb.com/microcaching-speed-your-app-up-250x-with-no-n
## The cache zone referenced.
fastcgi_cache fcgicache;
## The cache key.
fastcgi_cache_key $host$request_uri;
## For 200 and 301 make the cache valid for 2 hours.
fastcgi_cache_valid 200 301 2h;
## For 302 make it valid for 30 minutes.
fastcgi_cache_valid 302 30m;
## For 404 make it valid 10 minutes.
fastcgi_cache_valid 404 10m;
## If there are any upstream errors or the item has expired use
## whatever it is available.
fastcgi_cache_use_stale error timeout invalid_header updating http_500;
## The Cache-Control and Expires headers should be delivered untouched
## from the upstream to the client.
fastcgi_ignore_headers Cache-Control Expires;
## If we have a cookie we should bypass the cache. The same if we have a
fastcgi_cache_bypass $no_cache;
fastcgi_no_cache $no_cache;
## Add a cache miss/hit status header.
add_header X-Piwik-Long-Cache $upstream_cache_status;
## To avoid any interaction with the cache control headers we expire
## everything on this location immediately.
expires epoch;
}
## Any other attempt to access PHP files redirects to the root.
location ~* ^.+\.php$ {
return 302 /;
}
## Redirect to the root if attempting to access a txt file.
location ~* (?:DESIGN|(?:gpl|README|LICENSE)[^.]*|LEGALNOTICE)(?:\.txt)*$ {
return 302 /;
}
## Disallow access to several helper files.
location ~* \.(?:bat|html?|git|ini|sh|svn[^.]*|txt|tpl|xml)$ {
return 404;
}
## No crawling of this site for bots that obey robots.txt.
location = /robots.txt {
return 200 "User-agent: *\nDisallow: /\n";
}
}
And /etc/nginx/fastcgi_cache_zone.conf contains:
fastcgi_cache_path /var/cache/nginx/fcgicache levels=1:2 keys_zone=fcgicache:5M max_size=500M inactive=3h;
The cache currently has 4026 items in it and it's 19M in total, I have added it to the list of things that shouldn't be backed up, in /etc/backup.d/90.rdiff:
exclude = /var/cache/nginx/fcgicache
comment:35 follow-up: ↓ 41 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 25.05 to 25.15
There are some HTTPS settings that need checking and perhaps redirects from addresses like www.wiki.transitionnetwork.org to wiki.transitionnetwork.org need setting up.
HTTPS issues:
This server is vulnerable to the CRIME attack.
Chain issues Contains anchor
Compression Yes INSECURE (more info)
Session resumption No (IDs assigned but not accepted)
https://www.ssllabs.com/ssltest/analyze.html?d=wiki.transitionnetwork.org&s=81.95.52.111
comment:36 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.3
- Total Hours changed from 25.15 to 25.45
I have just checked the logs on kiwi and it's not been doing anything for some days -- all the sites are now on penguin, so I have shut it down and updated some wiki pages.
comment:37 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.3
- Total Hours changed from 25.45 to 25.75
Xen console access has been setup and documented for wiki:PuffinServer and wiki:PenguinServer on the wiki:XenShell page.
comment:38 Changed 4 years ago by chris
Please note that some of the recent emails from Trac, since in was moved to wiki:PenguinServer, contain bug descriptions.
This is caused when a bug, that hasn't had a new comment to it posted since the migration, has a new comment posted to it.
Sorry about this, the "answer" is to scroll to then end of these emails are read the recent comment at the very end of them.
comment:39 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.77
- Total Hours changed from 25.75 to 26.52
Comparing the MySQL settings on wiki:NewLiveServer and wiki:PenguinServer.
| Variable | Puffin | Quince |
|---|---|---|
| key_buffer | 16M | 256M |
| max_allowed_packet | 16M | 64M |
| max_connections | 100 | 20 |
| table_cache | 64 | 1028 |
| query_cache_limit | 1M | 256M |
| query_cache_size | 16M | 64M |
| sort_buffer_size | 1M | |
| read_buffer_size | 1M | |
| read_rnd_buffer_size | 64M | |
| myisam_sort_buffer_size | 32M | |
| table_open_cache | 4096 | |
| join_buffer_size | 24576K | |
| tmp_table_size | 128M | |
| max_heap_table_size | 128M | |
| innodb_buffer_pool_size | 256M | |
| innodb_additional_mem_pool_size | 20M | |
| innodb_data_home_dir | /var/lib/mysql/ibdata | |
| innodb_data_file_path | ibdata1:1G:autoextend:max:12G | |
| innodb_log_file_size | 64M | |
| innodb_log_buffer_size | 8M | |
| innodb_log_group_home_dir | /var/lib/mysql/ibdata | |
| innodb_flush_log_at_trx_commit | 1 | |
| innodb_lock_wait_timeout | 50 |
Check to see what suggestions we can get from http://mysqltuner.com/
cd /usr/local/bin
wget --no-check-certificate https://raw.github.com/rackerhacker/MySQLTuner-perl/master/mysqltuner.pl
perl mysqltuner.pl
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 150)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 65
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 45d 23h 59m 59s (1M q [0.419 qps], 84K conn, TX: 15B, RX: 2B)
[--] Reads / Writes: 85% / 15%
[--] Total buffers: 58.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 463.8M (22% of installed RAM)
[OK] Slow queries: 0% (28/1M)
[OK] Highest usage of available connections: 7% (11/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/290.5M
[OK] Key buffer hit rate: 99.8% (110M cached / 256K reads)
[OK] Query cache efficiency: 25.0% (304K cached / 1M selects)
[!!] Query cache prunes per day: 366
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 71K sorts)
[OK] Temporary tables created on disk: 17% (25K on disk / 145K total)
[OK] Thread cache hit rate: 99% (17 created / 84K connections)
[!!] Table cache hit rate: 0% (64 open / 32K opened)
[OK] Open file limit used: 3% (33/1K)
[OK] Table locks acquired immediately: 99% (1M immediate / 1M locks)
[!!] InnoDB data size / buffer pool: 23.7M/8.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
query_cache_size (> 16M)
table_cache (> 64)
innodb_buffer_pool_size (>= 23M)
So following these suggestions, the database were backed up using ninjahelper and then a bash script to run "OPTIMIZE TABLE tablename" on every table in every database was written and run.
And the following variables were changed as suggested:
#query_cache_size = 16M query_cache_size = 32M #table_cache = 64 table_cache = 128 innodb_buffer_pool_size = 64M
And mysql was restarted, it would be worth running the perl script again tomorrow to see what it suggests.
comment:40 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.45
- Total Hours changed from 26.52 to 26.97
Comparing php settings on wiki:NewLiveServer and wiki:PenguinServer first for the command line options, /etc/php5/cli/php.ini:
| Variable | Puffin | Quince |
|---|---|---|
| serialize_precision | 17 | 100 |
| memory_limit | -1 | 512M |
The following values in /etc/php5/cli/php.ini were changed:
;serialize_precision = 17 serialize_precision = 100 ;memory_limit = -1 memory_limit = 512M
Comparing /etc/php5/fpm/php.ini on penguin and /etc/php5/apache2/php.ini on quince:
| Variable | Penguin | Quince |
|---|---|---|
| expose_php | on | off |
| max_execution_time | 30 | 60 |
| max_input_time | 60 | 120 |
| memory_limit | 128M | 256M |
| error_reporting | E_ALL & ~E_DEPRECATED | E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR |
| post_max_size | 8M | 40M |
| default_charset | "utf-8" | |
| upload_max_filesize | 2M | 24M |
The following variables in /etc/php5/fpm/php.ini were changed:
;max_execution_time = 30 max_execution_time = 60 ;max_input_time = 60 max_input_time = 120 ;memory_limit = 128M memory_limit = 256M ;post_max_size = 8M post_max_size = 125M ;upload_max_filesize = 2M upload_max_filesize = 100M
And /etc/init.d/php5-fpm was restarted.
comment:41 in reply to: ↑ 35 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 3.0
- Total Hours changed from 26.97 to 29.97
Replying to chris:
HTTPS issues:
This server is vulnerable to the CRIME attack.
Chain issues Contains anchor
Compression Yes INSECURE (more info)
Session resumption No (IDs assigned but not accepted)
https://www.ssllabs.com/ssltest/analyze.html?d=wiki.transitionnetwork.org&s=81.95.52.111
Nginx settings from BOA were copied and the following:
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4-SHA:HIGH!ADH:!SSLv2:!aNULL;
Was changed to:
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
In all the files in /etc/nginx/sites-available/, however there was still the issue of the CRIME vulnerability -- this is caused by the combination of gzipped data and ssl so this section of config from /etc/nginx/nginx.conf, which did apply to all sites:
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
# text/html isn't needed in the following list for some reason
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
Was put into a separate file and just included for the non-ssl server sections, this was then tested using ab:
ab -n 1 -v 4 -H "Accept-Encoding: gzip" http://wiki.transitionnetwork.org/Main_Page
But that didn't work as Apache bench uses HTTP 1.0 not 1.1, so it was tested using wget:
wget --server-response --header="Accept-Encoding: gzip" -O /dev/null http://wiki.transitionnetwork.org/Main_Page
But that also didn't work, not sure why, so the Firefox Live HTTP Headers plugin was used and this shows that content is being sent gzipped, this is the config used:
# only include this in http sites, see # https://community.qualys.com/blogs/securitylabs/2012/09/14/crime-information-leakage-attack-against-ssltls # see # http://www.howtoforge.com/how-to-save-traffic-with-nginxs-httpgzipmodule-debian-squeeze gzip on; #gzip_disable "msie6"; gzip_disable "MSIE [1-6]\.(?!.*SV1)"; gzip_vary on; gzip_proxied any; # 1 to 9, 9 is maximum gzip_comp_level 9; gzip_buffers 32 16k; gzip_http_version 1.1; # text/html isn't needed in the following list for some reason gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
However this didn't solve the CRIME issue -- it turns out that the issue is the version on Nginx:
Details on the CRIME attack http://isecpartners.com/blog/2012/9/14/details-on-the-crime-attack.html … SSL comp is disabled in nginx since 1.3.2 and 1.2.2, SPDY comp can be configured/turned off
And we have:
dpkg -p nginx | grep Version Version: 1.2.1-2~bpo60+1
This was solved by fixing the apt pinning problem -- Pin-Priority was set to 1001 in /etc/apt/preferences.d/backports.pref so it now contains:
Package: gawk geoip-database libcairo2 libfreetype6 libgeoip1 liblog-dispatch-perl libnet-server-perl libpixman-1-0 liburi-perl libxfont1 munin munin-common munin-doc munin-node munin-plugins-core munin-plugins-extra nginx nginx-common nginx-full python-babel trac x11-common Pin: release o=backports Pin-Priority: 1001
The following packages were upgraded to newer versions:
munin-common munin-doc munin-plugins-core nginx nginx-common nginx-full
And now we have:
dpkg -p nginx | grep Version Version: 1.2.6-1~dotdeb.0
I don't understand why nginx from dotdeb has been pulled in, we had this for that repo, /etc/apt/preferences.d/dotdeb.pref:
Package: php-pear php5-cli php5-common php5-fpm php5-mysql php5 Pin: release o=packages.dotdeb.org Pin-Priority: 990
And this, /etc/apt/preferences.d/backports.pref for backports:
Package: gawk geoip-database libcairo2 libfreetype6 libgeoip1 liblog-dispatch-perl libnet-server-perl libpixman-1-0 liburi-perl libxfont1 munin munin-common munin-doc munin-node munin-plugins-core munin-plugins-extra nginx nginx-common nginx-full python-babel trac x11-common Pin: release o=backports Pin-Priority: 1001
I need to re-read http://wiki.debian.org/AptPreferences -- I still don't fully understand pinning, but using the nginx from dotdeb is OK so this isn't urgent.
There is the "Chain issues: Contains anchor" warning, but this isn't worth worrying about.
And there was also a "Session resumption No (IDs assigned but not accepted)" warning, this was solved by following this thread and these instructions and adding the following to the main http block:
ssl_session_cache shared:SSL:10m;
So now we have a "A" overall rating for the HTTPS configuration \o/
Jim is keen on using Collectd rather than Munin, so it the server was removed:
dpkg -r munin
However using the node doesn't have a big overhead so that has been set to generate stats here:
https://ecodissident.net/munin/webarch.net/penguin.webarch.net/
The exim plugin was deiabled and mysql and nginx ones were enabled, the nginx ones needed libwww-perl installing.
For php-fpm the plugin here was installed:
cd /usr/share/munin/plugins git clone git://github.com/tjstein/php5-fpm-munin-plugins.git chmod +x php5-fpm-munin-plugins/phpfpm_*
The symlinks were created from /etc/munin/plugins/ and the following needed adding to the default virtual host, /etc/nginx/sites-available/penguin:
location ~ ^/(status|ping)$ {
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
include fastcgi_params;
access_log off;
allow 127.0.0.1;
deny all;
}
The phpfpm_average and phpfpm_processes plugins were fixed by hardcoding the process:
#PHP_BIN=${phpbin-"php5-fpm"}
PHP_BIN="php-fpm"
And in phpfpm_memory:
#my $PHP_BIN = exists $ENV{'phpbin'} ? $ENV{'phpbin'} : "php5-fpm";
my $PHP_BIN = "php-fpm";
The perl package libcache-cache-perl was needed for the mysql_ plugins.
The /usr/local/bin/mysqltuner.pl script was run again:
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 150)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 67
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 5d 15h 15m 53s (1M q [2.114 qps], 50K conn, TX: 9B, RX: 730M)
[--] Reads / Writes: 80% / 20%
[--] Total buffers: 130.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 535.8M (26% of installed RAM)
[OK] Slow queries: 0% (23/1M)
[OK] Highest usage of available connections: 7% (11/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/285.3M
[OK] Key buffer hit rate: 99.8% (63M cached / 151K reads)
[OK] Query cache efficiency: 25.8% (186K cached / 722K selects)
[!!] Query cache prunes per day: 366
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 55K sorts)
[OK] Temporary tables created on disk: 14% (12K on disk / 89K total)
[OK] Thread cache hit rate: 99% (11 created / 50K connections)
[!!] Table cache hit rate: 0% (128 open / 22K opened)
[OK] Open file limit used: 13% (134/1K)
[OK] Table locks acquired immediately: 99% (701K immediate / 701K locks)
[OK] InnoDB data size / buffer pool: 23.6M/64.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
query_cache_size (> 32M)
table_cache (> 128)
And the following variables in /etc/mysql/my.cnf were changed and mysql restarted:
query_cache_size = 64M table_cache = 256
Regarding collectd it looks like generating stats from things like php-fpm would need some work, but there is a example of how to do it. Personally I think we should probably stick with Munin for now.
comment:42 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.16
- Total Hours changed from 29.97 to 30.13
The multips_memory plugin was enabled to track memory usage on the following processes:
[multips_memory] env.names php5-fpm munin-node nginx mysql tracd php-cli
comment:43 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 30.13 to 30.63
Looking at some page load time stats from sites on penguin on http://tools.pingdom.com/fpt/ the fact that the Piwik webbug is only being served up via https really slows things down, so I have changed the nginx settings so that for http sites the bug can be loaded via http, https sites will still get it via https.
Changing this has reduced to total page load time for http://wiki.transitionnetwork.org/Main_Page from 3.8 seconds to 1.4 seconds.
Looking at the stats for http://www.transitionnetwork.org/ it's the size of the front page has doubled in the last year, it was around 0.5M and it's now 1.1M, but it's being served up in 0.6 of a second when testing from Amsterdam, which isn't bad.
I think the migration of sites to can not be said to be more-or-less done, one outstanding issue is the server resources -- it's got more than it needs, look at the CPU usage:
Load:
And memory usage:
It's currently a 2GB VPS and it could be easilly dropped down to a 1.5GB VPS, server specs are here:
comment:44 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 30.63 to 30.88
The changes to the Piwik config in ticket:470#comment:43 overlooked the requirement for the login to use HTTPS, so this line was added to the port 80 server configuration:
# Redirect front page to HTTPS
rewrite !^/piwik\.*$ https://stats.transitionnetwork.org$request_uri? permanent;
This means that requests for http://stats.transitionnetwork.org/piwik.js and http://stats.transitionnetwork.org/piwik.php (both used for tracking) won't be redirected but all other requests will be.
The mysqltuning script was run again:
perl /usr/local/bin/mysqltuner.pl
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 150)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 66
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 2d 5h 8m 29s (422K q [2.206 qps], 29K conn, TX: 3B, RX: 308M)
[--] Reads / Writes: 76% / 24%
[--] Total buffers: 162.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 567.8M (27% of installed RAM)
[OK] Slow queries: 0% (8/422K)
[OK] Highest usage of available connections: 7% (12/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/290.5M
[OK] Key buffer hit rate: 99.8% (24M cached / 47K reads)
[OK] Query cache efficiency: 29.7% (81K cached / 274K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 25K sorts)
[OK] Temporary tables created on disk: 12% (5K on disk / 41K total)
[OK] Thread cache hit rate: 99% (14 created / 29K connections)
[!!] Table cache hit rate: 9% (256 open / 2K opened)
[OK] Open file limit used: 27% (279/1K)
[OK] Table locks acquired immediately: 99% (267K immediate / 267K locks)
[OK] InnoDB data size / buffer pool: 23.6M/64.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
table_cache (> 256)
The table_cache was increased to 2048.
comment:45 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 30.88 to 31.13
To help keep track of the usage of penguin I have added piwik tracking to the following sites:
- https://tech.transitionnetwork.org/
- http://static.transitionnetwork.org/
- http://2010.archive.transitionnetwork.org/
- http://2011.archive.transitionnetwork.org/ (but not the http://2011.archive.transitionnetwork.org/forum/ part of the site yet)
- http://penguin.transitionnetwork.org/
And gemoved GA from http://2011.archive.transitionnetwork.org/ I hope this is OK.
For Trac it was done by creating /web/tech.transitionnetwork.org/trac/templates/site.html containing:
<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:py="http://genshi.edgewall.org/"
py:strip="">
<body py:match="body" py:attrs="select('@*')">
${select('*|text()')}
<!--! Add site-specific footer -->
<div id="sitefooter">
<!-- Piwik -->
<script type="text/javascript">
var pkBaseURL = (("https:" == document.location.protocol) ? "https://stats.transitionnetwork.org/" : "http://stats.transitionnetwork.org/");
document.write(unescape("%3Cscript src='" + pkBaseURL + "piwik.js' type='text/javascript'%3E%3C/script%3E"));
</script><script type="text/javascript">
try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 7);
piwikTracker.trackPageView();
piwikTracker.enableLinkTracking();
} catch( err ) {}
</script><noscript><p><img src="http://stats.transitionnetwork.org/piwik.php?idsite=7" style="border:0" alt="" /></p></noscript>
<!-- End Piwik Tracking Code -->
</div>
</body>
</html>
Following the notes here http://trac.edgewall.org/wiki/TracInterfaceCustomization#SiteAppearance
comment:46 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 31.13 to 31.23
I have just added piwik stats to the old bbpress forum archive by adding the tracking code to /web/2011.archive.transitionnetwork.org/www/forum/bb-templates/kakumei/footer.php.
comment:47 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 31.23 to 31.48
- Description modified (diff)
The mysqltuning script was run again:
perl /usr/local/bin/mysqltuner.pl
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 150)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 67
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 4d 16h 42m 56s (1M q [3.250 qps], 53K conn, TX: 8B, RX: 819M)
[--] Reads / Writes: 80% / 20%
[--] Total buffers: 162.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 567.8M (27% of installed RAM)
[OK] Slow queries: 0% (20/1M)
[OK] Highest usage of available connections: 7% (12/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/294.0M
[OK] Key buffer hit rate: 99.8% (68M cached / 126K reads)
[OK] Query cache efficiency: 26.6% (252K cached / 951K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 70K sorts)
[OK] Temporary tables created on disk: 14% (17K on disk / 121K total)
[OK] Thread cache hit rate: 99% (22 created / 53K connections)
[!!] Table cache hit rate: 13% (483 open / 3K opened)
[OK] Open file limit used: 12% (530/4K)
[OK] Table locks acquired immediately: 99% (918K immediate / 918K locks)
[OK] InnoDB data size / buffer pool: 23.7M/64.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
table_cache (> 2048)
And the table_cache in /etc/mysql/my.cnf was doubled to 4096 and mysql was restarted.
This probably isn't worth increasing further (and perhaps is now too high) setting it too high can make things slower, currently we have 263 tables in total:
mysql information_schema mysql> SELECT COUNT(*) FROM tables; +----------+ | COUNT(*) | +----------+ | 263 | +----------+ 1 row in set (0.01 sec)
comment:48 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 31.48 to 31.58
I ran the mysqltuning script again:
perl /usr/local/bin/mysqltuner.pl
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 152)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 68
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 35d 0h 46m 6s (14M q [4.759 qps], 399K conn, TX: 63B, RX: 7B)
[--] Reads / Writes: 85% / 15%
[--] Total buffers: 162.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 567.8M (27% of installed RAM)
[OK] Slow queries: 0% (155/14M)
[OK] Highest usage of available connections: 8% (13/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/320.3M
[OK] Key buffer hit rate: 99.8% (649M cached / 1M reads)
[!!] Query cache efficiency: 18.1% (2M cached / 11M selects)
[!!] Query cache prunes per day: 970
[OK] Sorts requiring temporary tables: 0% (5 temp sorts / 677K sorts)
[OK] Temporary tables created on disk: 12% (130K on disk / 1M total)
[OK] Thread cache hit rate: 99% (46 created / 399K connections)
[!!] Table cache hit rate: 2% (488 open / 18K opened)
[OK] Open file limit used: 6% (536/8K)
[OK] Table locks acquired immediately: 99% (11M immediate / 11M locks)
[OK] InnoDB data size / buffer pool: 23.5M/64.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
query_cache_limit (> 1M, or use smaller result sets)
query_cache_size (> 64M)
table_cache (> 4096)
And I dramatically increased these settings as there is RAM to spare, see:
- https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/memory.html
- https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/multips_memory.html
query_cache_limit = 24M query_cache_size = 512M
comment:49 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.22
- Status changed from new to closed
- Resolution set to fixed
- Total Hours changed from 31.58 to 31.8
I ran the mysqltuning script again:
perl /usr/local/bin/mysqltuner.pl
>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[OK] Logged in using credentials from debian maintenance account.
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.66-0+squeeze1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 154)
[--] Data in InnoDB tables: 23M (Tables: 62)
[!!] Total fragmented tables: 68
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 27d 22h 58m 35s (12M q [5.375 qps], 262K conn, TX: 52B, RX: 6B)
[--] Reads / Writes: 86% / 14%
[--] Total buffers: 1.1G global + 2.7M per thread (30 max threads)
[OK] Maximum possible memory usage: 1.2G (58% of installed RAM)
[OK] Slow queries: 0% (151/12M)
[OK] Highest usage of available connections: 40% (12/30)
[OK] Key buffer size / total MyISAM indexes: 512.0M/330.4M
[OK] Key buffer hit rate: 99.9% (553M cached / 673K reads)
[OK] Query cache efficiency: 21.4% (2M cached / 10M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (1 temp sorts / 543K sorts)
[OK] Temporary tables created on disk: 14% (114K on disk / 810K total)
[OK] Thread cache hit rate: 99% (23 created / 262K connections)
[!!] Table cache hit rate: 2% (298 open / 14K opened)
[OK] Open file limit used: 3% (265/8K)
[OK] Table locks acquired immediately: 99% (9M immediate / 9M locks)
[OK] InnoDB data size / buffer pool: 23.2M/64.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
table_cache (> 4096)
Due to the comment on ticket:470#comment:47 it's probably not worth increasing the size of the table_cache
MySQL memory use appears to be stable at around 350MB, see https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/multips_memory.html this seems reasonable with nginx and php-fpm using around 150MB each and server server having 2GB -- there is plenty of slack to cope with load spikes.
I think this ticket can now be closed.
Steps done on the first configuration of puffin ticket:466#comment:2 were followed:
Generated a passphrase less root ssh key pair for rsyncing data off the quince and kiwi:
Added this public key to /root/.ssh/authorized_keys on kiwi and quince with this at the start of it to limit it's use to the penguin IP address:
Created /root/.ssh/config with the following in it:
Created user accounts for all the existing people who need it, matching the user and group ID's to the ones on quince, and copied across their ssh public keys and sent them email with their passwords, using the following:
Edited /etc/sudoers:
Edited /etc/ssh/sshd_config and restarted ssh:
Copied across the wild card ssl cert for *.transitionnetwork.org:
Set up root email:
Edited the TLS settings in /etc/postfix/main.cf and restarted postfix:
# TLS parameters smtpd_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt smtpd_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key smtpd_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem smtpd_use_tls=yes smtpd_tls_auth_only = yes smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_tls_session_cache_timeout = 3600s smtpd_tls_cipherlist = EDH:!EXP:!LOW smtpd_tls_ask_ccert = yes smtpd_tls_req_ccert = no smtp_use_tls = yes smtp_tls_note_starttls_offer = yes smtp_enforce_tls = no smtp_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt smtp_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key smtp_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem smtp_tls_loglevel = 1 smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_tls_session_cache_timeout = 3600sSent a test email to root using mutt and then checked that TLS was used by looking at the mail headers.
Installed mysql and added the root password to /root/.my.cnf:
Installed backupninja and created a user on the remote backup server and set up ssh:
Ran ninjahelper and set up system, mysql and rdiff backups and tested that it was all working.