Changes between Version 42 and Version 43 of PiwikServer


Ignore:
Timestamp:
02/27/13 11:11:41 (4 years ago)
Author:
chris
Comment:

Old information moved to wiki:PiwikServerInstall

Legend:

Unmodified
Added
Removed
Modified

  • PiwikServer

    v42 v43  
    2626They should then [https://stats.transitionnetwork.org/index.php?module=UsersManager create a user] and grant the new user access to the stats for the sites they need access to. 
    2727 
    28 == Issues == 
     28== Tickets == 
    2929 
    30  * ticket:382 
    31  * ticket:477 
    32  * ticket:371#comment:26 
     30[[TicketQuery(status=accepted|new|assigned|reopened&component=Piwik,order=id,desc=1,format=table,col=summary|owner|reporter)]] 
     31Closed Piwik tickets: [[TicketQuery(max=1000,status=closed&component=Piwik,compact)]] 
    3332 
    3433== Updates == 
    … …  
    7069 
    7170 
    72  
    73 == Archive == 
    74  
    75 The following documentation is outdated. 
    76  
    77 === Quince Server === 
    78  
    79 The live install at https://stats.transitionnetwork.org/ on the NewLiveServer, the install was done on ticket:160. 
    80  
    81 ==== Install ==== 
    82  
    83 {{{ 
    84 cd /web 
    85 cd stats.transitionnetwork.org/ 
    86 wget http://piwik.org/latest.zip 
    87 unzip latest.zip 
    88 cd piwik 
    89 chown -R www-data:www-data tmp/ 
    90 chown -R www-data:www-data config/ 
    91 chown -R www-data:www-data themes/ 
    92 }}} 
    93  
    94 ===== Apache ===== 
    95  
    96 Create {{{stats.transitionnetwork.org.conf}}} in {{{cd /etc/apache2/sites-available/}}}: 
    97  
    98 {{{ 
    99 <VirtualHost *:80> 
    100   ServerName stats.transitionnetwork.org 
    101   ServerAlias stats.transitionnetwork.org.webarch.net 
    102   ServerSignature Off 
    103   UseCanonicalName On 
    104  
    105   ErrorLog /var/log/apache2/stats.transitionnetwork.org_error.log 
    106   ErrorLog /var/log/apache2/error.log 
    107   SetEnvIf Remote_Addr "127\.0\.0\.1" loopback 
    108   SetEnvIf Remote_Addr "kiwi\.transitionnetwork\.org" loopback 
    109   CustomLog /var/log/apache2/stats.transitionnetwork.org_access.log combined env=!loopback 
    110   CustomLog /var/log/apache2/access.log combined env=!loopback 
    111  
    112   Redirect / https://stats.transitionnetwork.org/ 
    113 </VirtualHost> 
    114  
    115 <VirtualHost *:443> 
    116   ServerName stats.transitionnetwork.org 
    117   ServerAlias stats.transitionnetwork.org.webarch.net 
    118   ServerSignature Off 
    119   UseCanonicalName On 
    120  
    121   LogLevel warn 
    122   ErrorLog /var/log/apache2/stats.transitionnetwork.org_ssl_error.log 
    123   ErrorLog /var/log/apache2/error.log 
    124   SetEnvIf Remote_Addr "127\.0\.0\.1" loopback 
    125   SetEnvIf Remote_Addr "kiwi\.transitionnetwork\.org" loopback 
    126   CustomLog /var/log/apache2/stats.transitionnetwork.org_ssl_access.log combined env=!loopback 
    127   CustomLog /var/log/apache2/access.log combined env=!loopback 
    128  
    129   SSLEngine on 
    130   SSLCipherSuite HIGH 
    131   SSLProtocol all -SSLv2 
    132   SSLCertificateFile      /etc/ssl/transitionnetwork.org/transitionnetwork.org.pem 
    133   SSLCertificateChainFile /etc/ssl/transitionnetwork.org/gandi.pem 
    134  
    135   DocumentRoot "/web/stats.transitionnetwork.org/piwik" 
    136   <Directory "/web/stats.transitionnetwork.org/piwik"> 
    137     DirectoryIndex index.php 
    138     Options -Indexes 
    139     AddType 'image/x-icon' .ico 
    140     AddHandler php5-script .php 
    141     AddType text/html .php 
    142     SSLOptions +StdEnvVars 
    143     AllowOverride None 
    144     order allow,deny 
    145     allow from all 
    146   </Directory> 
    147   <Directory "/web/stats.transitionnetwork.org/piwik/tmp"> 
    148     Options Indexes 
    149     RemoveHandler .php 
    150     AllowOverride None 
    151     order allow,deny 
    152     allow from all 
    153   </Directory> 
    154   <Location /config> 
    155     Order deny,allow 
    156     Deny from all 
    157     Allow from 127.0.0.1 
    158   </Location> 
    159  
    160   <FilesMatch "\.(cgi|shtml|phtml|php)$"> 
    161     SSLOptions +StdEnvVars 
    162   </FilesMatch> 
    163   <Directory /usr/lib/cgi-bin> 
    164     SSLOptions +StdEnvVars 
    165   </Directory> 
    166  
    167 </VirtualHost> 
    168 }}} 
    169  
    170 Enable it and restart: 
    171  
    172 {{{ 
    173 /etc/apache2/sites-enabled 
    174 ln -s ../sites-available/stats.transitionnetwork.org.conf 35-stats.transitionnetwork.org.conf 
    175 apache2ctl configtest 
    176 apache2ctl restart 
    177 }}} 
    178  
    179 ===== MySQL ===== 
    180  
    181 As MySQL root user: 
    182  
    183 {{{ 
    184 CREATE USER 'stats'@'localhost' IDENTIFIED BY '********'; 
    185 GRANT USAGE ON * . * TO 'stats'@'localhost' IDENTIFIED BY '********' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ; 
    186 CREATE DATABASE IF NOT EXISTS `stats` ; 
    187 GRANT ALL PRIVILEGES ON `stats` . * TO 'stats'@'localhost'; 
    188 FLUSH PRIVILEGES; 
    189 }}} 
    190  
    191 ===== Web Install ===== 
    192  
    193 MySQL settings: 
    194  
    195  * database server: 127.0.0.1 
    196  * login: stats 
    197  * password: ******** 
    198  * database name: stats 
    199  * table prefix: (blank - we don't need one, the default is piwik_) 
    200  * adapter: PDO_MYSQL 
    201  
    202 Super User settings: 
    203  
    204  * super user login: webproject 
    205  * password: ******** 
    206  * email: webproject (at) transitionnetwork.org 
    207  
    208 Suggested JS code: 
    209  
    210 {{{ 
    211 <!-- Piwik --> 
    212 <script type="text/javascript"> 
    213 var pkBaseURL = (("https:" == document.location.protocol) ? "https://stats.transitionnetwork.org/" : "http://stats.transitionnetwork.org/"); 
    214 document.write(unescape("%3Cscript src='" + pkBaseURL + "piwik.js' type='text/javascript'%3E%3C/script%3E")); 
    215 </script><script type="text/javascript"> 
    216 try { 
    217 var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 1); 
    218 piwikTracker.trackPageView(); 
    219 piwikTracker.enableLinkTracking(); 
    220 } catch( err ) {} 
    221 </script><noscript><p><img src="http://stats.transitionnetwork.org/piwik.php?idsite=1" style="border:0" alt="" /></p></noscript> 
    222 <!-- End Piwik Tracking Code --> 
    223 }}} 
    224  
    225 Suggested image code: 
    226  
    227 {{{ 
    228 <!-- Piwik Image Tracker --> 
    229 <img src="https://stats.transitionnetwork.org/piwik.php?idsite=1&rec=1" style="border:0" alt="" /> 
    230 <!-- End Piwik --> 
    231 }}} 
    232  
    233 ===== Configure ===== 
    234  
    235 In the General Settings page: 
    236  
    237  * Allow Piwik archiving to trigger when reports are viewed from the browser: no 
    238  * Reports for today (or any other Date Range including today) will be processed at most every: 1800 
    239  
    240 Set up cron: 
    241  
    242 {{{ 
    243 crontab -u www-data -e 
    244  
    245 5 * * * * /web/stats.transitionnetwork.org/piwik/misc/cron/archive.sh > /dev/null 
    246 }}} 
    247  
    248 Test it: 
    249  
    250 {{{ 
    251 su www-data -c "sh /web/stats.transitionnetwork.org/piwik/misc/cron/archive.sh" 
    252 }}} 
    253  
    254 This generated this error: 
    255  
    256 {{{ 
    257 ALERT - script tried to increase memory_limit to 134217728 bytes which is above the allowed value (attacker 'REMOTE_ADDR not set', file '/web/stats.transitionnetwork.org/piwik/core/Piwik.php', line 947) 
    258 }}} 
    259  
    260 Which can be ignored because in {{{/etc/php5/apache2/php.ini}}} it's set to double that already, 256M, so I don't understand why it tried to raise it. 
    261  
    262 Things added to {{{/web/stats.transitionnetwork.org/piwik/config/config.ini.php}}}, taken from {{{global.ini.php}}}: 
    263  
    264 {{{ 
    265 [General] 
    266 force_ssl_login = 1 
    267 feedback_email_address = "webproject (at) transitionnetwork.org" 
    268 assume_secure_protocol = 1 
    269 }}} 
    270  
    271 ===== GA Import ===== 
    272  
    273 First install Python 2.6 from source: 
    274  
    275 {{{ 
    276 aptitude build-dep python2.5  
    277 cd /usr/local/src 
    278 wget http://www.python.org/ftp/python/2.6.7/Python-2.6.7.tgz 
    279 tar -zxvf Python-2.6.7.tgz 
    280 cd Python-2.6.7 
    281 ./configure --with-threads --enable-shared 
    282 make 
    283 make install  
    284 ln -s /usr/local/lib/libpython2.6.so* /usr/lib/ 
    285 cd /usr/local/src 
    286 wget 'http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz#md5=7df2a529a074f613b509fb44feefe74e' 
    287 tar zxvf setuptools-0.6c11.tar.gz 
    288 cd setuptools-0.6c11 
    289 python setup.py install 
    290 cd /usr/local/src 
    291 wget "https://gdata-python-client.googlecode.com/files/gdata-2.0.14.tar.gz" --no-check-certificate 
    292 tar -zxvf gdata-2.0.14.tar.gz 
    293 cd gdata-2.0.14 
    294 python setup.py install 
    295 cd /usr/local/src 
    296 aptitude purge python-mysqldb 
    297 wget "http://downloads.sourceforge.net/project/mysql-python/mysql-python/1.2.3/MySQL-python-1.2.3.tar.gz" 
    298 tar -zxvf MySQL-python-1.2.3.tar.gz  
    299 cd MySQL-python-1.2.3 
    300 vim site.cfg 
    301  mysql_config = /usr/bin/mysql_config 
    302 aptitude install libmysqlclient-dev 
    303 python setup.py build 
    304 python setup.py install 
    305 }}} 
    306  
    307 Then the GA code and import: 
    308  
    309 {{{ 
    310 cd /web/stats.transitionnetwork.org/ 
    311 wget http://clearcode.cc/google2piwik-1.2.5.tgz 
    312 tar -zxvf google2piwik-1.2.5.tgz 
    313 cd google2piwik-1.2.5 
    314 cp google2piwik.conf.sample google2piwik.conf 
    315 }}} 
    316  
    317 Edit {{{google2piwik.conf}}}:  
    318  
    319 {{{ 
    320  
    321 ## Google Account Credentials 
    322 # If you don't know your site table_id type: 
    323 #                       ./google2piwik.py -p 
    324 [google] 
    325 user_login = transitiontownsnetwork [at] gmail.com 
    326 user_pass  = XXX 
    327 table_id   = ga:XXX 
    328  
    329 # MySQL Piwik Database configuration 
    330 [mysql] 
    331 db = stats 
    332 host = localhost 
    333 port = 3306 
    334 user = stats 
    335 passwd = XXX 
    336 table_prefix = 
    337  
    338 # Set export date range 
    339 [export] 
    340 start = 2008-01-01 
    341 end   = 2011-06-29 
    342  
    343 ## Piwik configuration of site. 
    344 # For default Piwik installation with one website, site_id should be set to 1 
    345 [piwik] 
    346 site_id  = 1 
    347 site_url = http://transitionnetwork.org 
    348 }}} 
    349  
    350 Test and backup db: 
    351  
    352 {{{ 
    353 ./google2piwik.py -p 
    354 ./google2piwik.py -c 
    355 mysql_backup 
    356 }}} 
    357  
    358 Then change the timezone to UTC for the site on https://stats.transitionnetwork.org/ run it in screen: 
    359  
    360 {{{ 
    361 screen 
    362 ./google2piwik.py --verbose=2 
    363 }}} 
    364  
    365 === Kiwi Server === 
    366  
    367 The [http://piwik.org/ Piwik] server that is to be installed on the DevelopmentServer, the install was done on ticket:160 
    368  
    369 ==== Config ==== 
    370  
    371 In {{{/web/piwik.transitionnetwork.org/piwik/config}}} 
    372  
    373 {{{ 
    374 assume_secure_protocol = 1 
    375 }}} 
    376  
    377 ==== Install ==== 
    378  
    379 Site:  https://piwik.transitionnetwork.org/ 
    380  
    381 ===== Download ===== 
    382  
    383 {{{ 
    384 cd /web/ 
    385 mkdir piwik.transitionnetwork.org 
    386 cd piwik.transitionnetwork.org/ 
    387 wget http://piwik.org/latest.zip 
    388 unzip latest.zip 
    389 }}} 
    390  
    391 ===== Apache ===== 
    392  
    393 Then set up apache, see /etc/apache2/sites-available/piwik.transitionnetwork.org and run the web installer and: 
    394  
    395 {{{ 
    396 cd /web/piwik.transitionnetwork.org/piwik/ 
    397 chown -R www-data:www-data tmp/ 
    398 chown -R www-data:www-data config/ 
    399 }}} 
    400  
    401 ===== MySQL ===== 
    402  
    403 Then a MySQL user and database was added using phpMyAdmin which ran the following SQL: 
    404  
    405 {{{ 
    406 CREATE USER 'piwik'@'localhost' IDENTIFIED BY '********'; 
    407 GRANT USAGE ON * . * TO 'piwik'@'localhost' IDENTIFIED BY '********' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ; 
    408 CREATE DATABASE IF NOT EXISTS `piwik` ; 
    409 GRANT ALL PRIVILEGES ON `piwik` . * TO 'piwik'@'localhost'; 
    410 }}} 
    411  
    412 ===== Tracking code ===== 
    413  
    414 The installer suggested this JS tracking code, see the docs at http://piwik.org/docs/javascript-tracking/ : 
    415  
    416 {{{ 
    417 <!-- Piwik --> 
    418 <script type="text/javascript"> 
    419 var pkBaseURL = (("https:" == document.location.protocol) ? "https://piwik.transitionnetwork.org.webarch.net/" : "http://piwik.transitionnetwork.org.webarch.net/"); 
    420 document.write(unescape("%3Cscript src='" + pkBaseURL + "piwik.js' type='text/javascript'%3E%3C/script%3E")); 
    421 </script><script type="text/javascript"> 
    422 try { 
    423 var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 1); 
    424 piwikTracker.trackPageView(); 
    425 piwikTracker.enableLinkTracking(); 
    426 } catch( err ) {} 
    427 </script><noscript><p><img src="http://piwik.transitionnetwork.org.webarch.net/piwik.php?idsite=1" style="border:0" alt="" /></p></noscript> 
    428 <!-- End Piwik Tracking Code --> 
    429 }}} 
    430  
    431 And this image tracking code: 
    432  
    433 {{{ 
    434 <!-- Piwik Image Tracker --> 
    435 <img src="https://piwik.transitionnetwork.org.webarch.net/piwik.php?idsite=1&rec=1" style="border:0" alt="" /> 
    436 <!-- End Piwik --> 
    437 }}} 
    438  
    439 The following parameters can also be passed to the image URL: 
    440  
    441  *   rec - (required) The parameter &rec=1 is required to force the request to be recorded 
    442  *   idsite - (required) Defines the Website ID being tracked 
    443  *   action_name - Defines the custom Page Title for this page view 
    444  *   idgoal - The request will trigger the given Goal 
    445  *   urlref - The Referrer URL: must be set to the referrer URL used before landing on the page containing the Image tracker. For example, in PHP this value is accessible via 
    446     {{{$_SERVER['HTTP_REFERER']}}} 
    447  *   revenue - Used with idgoal, defines the custom revenue for this conversion 
    448  
    449 ===== Logo ===== 
    450  
    451 Customise the logo: 
    452  
    453 {{{ 
    454 cd /web/piwik.transitionnetwork.org/piwik 
    455 chown -R www-data:www-data themes/ 
    456 }}} 
    457  
    458 ===== Cron ===== 
    459  
    460 Set up the [http://piwik.org/docs/setup-auto-archiving/#toc-linuxunix-how-to-setup-a-crontab-to-automatically-archive-the-reports crontab to automatically archive the reports]: 
    461  
    462 {{{ 
    463 crontab -u www-data -e 
    464  
    465 5 * * * * /web/piwik.transitionnetwork.org/piwik/misc/cron/archive.sh > /dev/null 
    466 }}} 
    467  
    468 ===== Python ===== 
    469  
    470 For the Google Analytics data import we need Python 2.6 and we are running with Python 2.5.2, so [http://evan.borgstrom.ca/post/845359613/python-2-6-on-debian-lenny-5-0 following this] a source compile was done on the dev server: 
    471  
    472 {{{ 
    473 aptitude build-dep python2.5  
    474 cd /usr/local/src 
    475 wget http://www.python.org/ftp/python/2.6.7/Python-2.6.7.tgz 
    476 tar -zxvf Python-2.6.7.tgz 
    477 cd Python-2.6.7 
    478 ./configure --with-threads --enable-shared 
    479 make 
    480 make install  
    481 ln -s /usr/local/lib/libpython2.6.so* /usr/lib/ 
    482 cd /usr/local/src 
    483 wget 'http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz#md5=7df2a529a074f613b509fb44feefe74e' 
    484 tar zxvf setuptools-0.6c11.tar.gz 
    485 cd setuptools-0.6c11 
    486 python setup.py install 
    487 cd /usr/local/src 
    488 wget "https://gdata-python-client.googlecode.com/files/gdata-2.0.14.tar.gz" --no-check-certificate 
    489 tar -zxvf gdata-2.0.14.tar.gz 
    490 cd gdata-2.0.14 
    491 python setup.py install 
    492 aptitude purge python-mysqldb 
    493 wget "http://downloads.sourceforge.net/project/mysql-python/mysql-python/1.2.3/MySQL-python-1.2.3.tar.gz" 
    494 tar -zxvf MySQL-python-1.2.3.tar.gz  
    495 cd MySQL-python-1.2.3 
    496 vim site.cfg 
    497  mysql_config = /usr/bin/mysql_config 
    498 aptitude install libmysqlclient-dev 
    499 python setup.py build 
    500 python setup.py install 
    501 }}} 
    502  
    503 ===== GA Import ===== 
    504  
    505 See python notes above and [http://clearcode.cc/offer/open-source-projects/google2piwik/ Google2Piwik]. 
    506  
    507 {{{ 
    508 cd /web/piwik.transitionnetwork.org/ 
    509 wget http://clearcode.cc/google2piwik-1.2.5.tgz 
    510 tar -zxvf google2piwik-1.2.5.tgz 
    511 cd google2piwik-1.2.5 
    512 cp google2piwik.conf.sample google2piwik.conf 
    513 }}} 
    514  
    515 Edit {{{google2piwik.conf}}}: 
    516  
    517 {{{ 
    518 ## Google Account Credentials 
    519 # If you don't know your site table_id type: 
    520 #                       ./google2piwik.py -p 
    521 [google] 
    522 user_login = XXX@gmail.com 
    523 user_pass  = XXX 
    524 table_id   = ga:XXX 
    525  
    526 # MySQL Piwik Database configuration 
    527 [mysql] 
    528 db = piwik 
    529 host = localhost 
    530 port = 3306 
    531 user = piwik 
    532 passwd = XXX 
    533 table_prefix =  
    534  
    535 # Set export date range 
    536 [export] 
    537 start = 2011-05-01 
    538 end   = 2011-06-29 
    539  
    540 ## Piwik configuration of site. 
    541 # For default Piwik installation with one website, site_id should be set to 1 
    542 [piwik] 
    543 site_id  = 1 
    544 site_url = http://www.transitionnetwork.org 
    545 }}} 
    546  
    547 Get the table_id and test and backup db: 
    548  
    549 {{{ 
    550 ./google2piwik.py -p 
    551 ./google2piwik.py -c 
    552 mysql_backup 
    553 }}} 
    554  
    555 Piwik, for the website you are importing data to, [https://piwik.transitionnetwork.org/index.php?module=SitesManager&action=index&idSite=1&period=day&date=yesterday change the timezone to UTC] and run the script: 
    556  
    557 {{{ 
    558 ./google2piwik.py --verbose=2 
    559 }}} 
    560  
    561  
    562 === Requirements === 
    563  
    564 The [http://piwik.org/docs/requirements/ requirements are]: 
    565  
    566  * PHP version 5.1.3 or greater, We highly recommend using at least PHP 5.3 as it is much more memory efficient than previous PHP versions, we have: PHP 5.2.6-1+lenny10 with Suhosin-Patch 0.9.6.2 (cli), when the server is migrated to squeeze we will get a new php, it can wait till then. 
    567  * MySQL version 4.1 or greater, we have 5.0.51a-24+lenny5-log 
    568  * (enabled by default) PHP extension pdo and pdo_mysql, weh have these already installed and they are configured via /etc/php5/apache2/conf.d/pdo.ini and /etc/php5/apache2/conf.d/pdo_mysql.ini 
    569  * PHP GD extension, installed already 
    570  * Increase your PHP memory_limit value in your php.ini config file to be at least memory_limit = 128M, we have 256M already in /etc/php5/apache2/php.ini 
    571