Transition Technology: Ticket #861: Piwik 2.14.0

hours, status, totalhours changed; resolution set

chris — Thu, 09 Jul 2015 10:08:53 GMT

hours changed from 0.0 to 0.35
status changed from new to closed
resolution set to fixed
totalhours changed from 0.0 to 0.35

sudo -i
vi /web/stats.transitionnetwork.org/piwik/config/config.ini.php
cd /web/stats.transitionnetwork.org/
cp piwik/config/config.ini.php .
export PIWIK="2.14.0"
wget "https://builds.piwik.org/piwik-$PIWIK.tar.gz"
wget "https://builds.piwik.org/piwik-$PIWIK.tar.gz.asc"
gpg --verify piwik-$PIWIK.tar.gz.asc
tar -zxvf piwik-$PIWIK.tar.gz
cp config.ini.php piwik/config/
chown -R www-data:www-data piwik/
php /web/stats.transitionnetwork.org/piwik/console core:update
      *** Update ***
      Database Upgrade Required
      Your Piwik database is out-of-date, and must be upgraded before you can continue.
      Piwik database will be upgraded from version 2.13.1 to the new version 2.14.0.
      *** Note: this is a Dry Run ***
      DROP TABLE IF EXISTS `site_setting`;
      CREATE TABLE `site_setting` (
                    idsite INTEGER(10) UNSIGNED NOT NULL AUTO_INCREMENT,
                    `setting_name` VARCHAR(255) NOT NULL,
                    `setting_value` LONGTEXT NOT NULL,
                        PRIMARY KEY(idsite, setting_name)
                      ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
      *** End of Dry Run ***
  A database upgrade is required. Execute update? (y/N) y
  Starting the database upgrade process now. This may take a while, so please be patient.
      *** Update ***
      Database Upgrade Required
      Your Piwik database is out-of-date, and must be upgraded before you can continue.
      Piwik database will be upgraded from version 2.13.1 to the new version 2.14.0.
      The database upgrade process may take a while, so please be patient.
    Executing DROP TABLE IF EXISTS `site_setting`... Done. [1 / 2]
    Executing CREATE TABLE `site_setting` (
                    idsite INTEGER(10) UNSIGNED NOT NULL AUTO_INCREMENT,
                    `setting_name` VARCHAR(255) NOT NULL,
                    `setting_value` LONGTEXT NOT NULL,
                        PRIMARY KEY(idsite, setting_name)
                      ) ENGINE=InnoDB DEFAULT CHARSET=utf8... Done. [2 / 2]
  ****************************************
    Piwik has been successfully updated!
  ****************************************
vim /web/stats.transitionnetwork.org/piwik/config/config.ini.php
/etc/init.d/php5-fpm restart

The System Check was checked and all looks good.

hours, status, totalhours changed; resolution deleted

chris — Thu, 09 Jul 2015 11:15:09 GMT

hours changed from 0.0 to 0.02
status changed from closed to reopened
resolution fixed deleted
totalhours changed from 0.35 to 0.37

Reopening this ticket due to this cron error following the upgrade:

Date: Thu,  9 Jul 2015 12:05:10 +0100 (BST)
From: Cron Daemon <root@penguin.webarch.net>
To: root@localhost
Subject: Cron <www-data@penguin> /web/stats.transitionnetwork.org/piwik/console core:archive --url=http://stats.transitionnetwork.org/ >>
        /var/log/piwik-archive.log
ERROR CoreConsole[2015-07-09 11:05:07] Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was 'a:2:{s:6:"result";s:5:"error";s:7:"message";s:104:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.location_browser_lang' in 'field list'";}'
ERROR CoreConsole[2015-07-09 11:05:07] Empty or invalid response '' for website id 1, Time elapsed: 2.943s, skipping
ERROR CoreConsole[2015-07-09 11:05:09] Got invalid response from API request: ?module=API&method=API.get&idSite=11&period=day&date=last52&format=php&trigger=archivephp. Response was 'a:2:{s:6:"result";s:5:"error";s:7:"message";s:104:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.location_browser_lang' in 'field list'";}'
ERROR CoreConsole[2015-07-09 11:05:09] Empty or invalid response '' for website id 11, Time elapsed: 2.274s, skipping
ERROR CoreConsole[2015-07-09 11:05:09] 4 total errors during this script execution, please investigate and try and fix these errors.
  [Exception]
  4 total errors during this script execution, please investigate and try and fix these errors.
core:archive [--url="..."] [--force-all-websites] [--force-all-periods[="..."]] [--force-timeout-for-periods[="..."]] [--skip-idsites[="..."]] [--skip-all-segments] [--force-idsites[="..."]] [--force-periods[="..."]] [--force-date-last-n="..."] [--force-date-range[="..."]] [--force-idsegments="..."] [--concurrent-requests-per-website[="..."]] [--disable-scheduled-tasks] [--accept-invalid-ssl-certificate]

hours, totalhours changed

chris — Thu, 09 Jul 2015 12:08:41 GMT

hours changed from 0.0 to 0.89
totalhours changed from 0.37 to 1.26

Looks like someone else has this issue, I have been following it up on the Piwik Github tracker.

hours, status, totalhours changed; resolution set

chris — Mon, 13 Jul 2015 10:20:29 GMT

hours changed from 0.0 to 0.65
status changed from reopened to closed
resolution set to fixed
totalhours changed from 1.26 to 1.91

I was hoping for a Piwik update to resolve the missing tables issues but one hasn't come out yet so I have been getting these emails every hour:

Date: Mon, 13 Jul 2015 09:06:05 +0100 (BST)
From: Cron Daemon <root@penguin.webarch.net>
To: root@localhost
Subject: Cron <www-data@penguin> /web/stats.transitionnetwork.org/piwik/console core:archive --url=http://stats.transitionnetwork.org/ >>
        /var/log/piwik-archive.log
ERROR CoreConsole[2015-07-13 08:05:05] Got invalid response from API request:
+?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:05:05] Empty or invalid response '' for website id 1, Time elapsed: 1.892s, skipping
ERROR CoreConsole[2015-07-13 08:05:06] Got invalid response from API request:
+?module=API&method=API.get&idSite=11&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:05:06] Empty or invalid response '' for website id 11, Time elapsed: 1.205s, skipping
ERROR CoreConsole[2015-07-13 08:05:10] Got invalid response from API request:
+?module=API&method=API.get&idSite=2&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:05:10] Empty or invalid response '' for website id 2, Time elapsed: 3.835s, skipping
ERROR CoreConsole[2015-07-13 08:05:34] Got invalid response from API request:
+?module=API&method=API.get&idSite=5&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:05:34] Empty or invalid response '' for website id 5, Time elapsed: 1.064s, skipping
ERROR CoreConsole[2015-07-13 08:05:42] Got invalid response from API request:
+?module=API&method=API.get&idSite=7&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:05:42] Empty or invalid response '' for website id 7, Time elapsed: 1.595s, skipping
ERROR CoreConsole[2015-07-13 08:06:04] Got invalid response from API request:
+?module=API&method=API.get&idSite=13&period=day&date=last52&format=php&trigger=archivephp. Response was
+'a:2:{s:6:"result";s:5:"error";s:7:"message";s:92:"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'log_visit.config_os' in 'field
+list'";}'
ERROR CoreConsole[2015-07-13 08:06:04] Empty or invalid response '' for website id 13, Time elapsed: 1.068s, skipping
ERROR CoreConsole[2015-07-13 08:06:04] 12 total errors during this script execution, please investigate and try and fix these errors.
  [Exception]
  12 total errors during this script execution, please investigate and try and fix these errors.
core:archive [--url="..."] [--force-all-websites] [--force-all-periods[="..."]] [--force-timeout-for-periods[="..."]] [--skip-idsites[="..."]]
+[--skip-all-segments] [--force-idsites[="..."]] [--force-periods[="..."]] [--force-date-last-n="..."] [--force-date-range[="..."]]
+[--force-idsegments="..."] [--concurrent-requests-per-website[="..."]] [--disable-scheduled-tasks] [--accept-invalid-ssl-certificate]

So following the manual fix:

sudo -i
mysql stats
mysql> DESCRIBE log_visit;
+---------------------------+----------------------+------+-----+---------+----------------+
| Field                     | Type                 | Null | Key | Default | Extra          |
+---------------------------+----------------------+------+-----+---------+----------------+
| idvisit                   | int(10) unsigned     | NO   | PRI | NULL    | auto_increment |
| idsite                    | int(10) unsigned     | NO   | MUL | NULL    |                |
| idvisitor                 | binary(8)            | NO   |     | NULL    |                |
| visitor_localtime         | time                 | NO   |     | NULL    |                |
| visitor_returning         | tinyint(1)           | NO   |     | NULL    |                |
| visitor_count_visits      | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_last   | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_order  | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_first  | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_first_action_time   | datetime             | NO   |     | NULL    |                |
| visit_last_action_time    | datetime             | NO   |     | NULL    |                |
| visit_exit_idaction_url   | int(10) unsigned     | YES  |     | 0       |                |
| visit_exit_idaction_name  | int(11) unsigned     | NO   |     | NULL    |                |
| visit_entry_idaction_url  | int(11) unsigned     | NO   |     | NULL    |                |
| visit_entry_idaction_name | int(11) unsigned     | NO   |     | NULL    |                |
| visit_total_actions       | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_searches      | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_events        | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_time          | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_goal_converted      | tinyint(1)           | NO   |     | NULL    |                |
| visit_goal_buyer          | tinyint(1)           | NO   |     | NULL    |                |
| referer_type              | tinyint(1) unsigned  | YES  |     | NULL    |                |
| referer_name              | varchar(70)          | YES  |     | NULL    |                |
| referer_url               | text                 | NO   |     | NULL    |                |
| referer_keyword           | varchar(255)         | YES  |     | NULL    |                |
| config_id                 | binary(8)            | NO   |     | NULL    |                |
| config_os_version         | varchar(100)         | YES  |     | NULL    |                |
| config_device_type        | varchar(100)         | YES  |     | NULL    |                |
| config_device_brand       | varchar(100)         | YES  |     | NULL    |                |
| config_device_model       | varchar(100)         | YES  |     | NULL    |                |
| location_ip               | varbinary(16)        | NO   |     | NULL    |                |
| location_country          | char(3)              | NO   |     | NULL    |                |
| location_region           | char(2)              | YES  |     | NULL    |                |
| location_city             | varchar(255)         | YES  |     | NULL    |                |
| location_latitude         | float(10,6)          | YES  |     | NULL    |                |
| location_longitude        | float(10,6)          | YES  |     | NULL    |                |
| custom_var_k1             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v1             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k2             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v2             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k3             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v3             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k4             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v4             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k5             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v5             | varchar(200)         | YES  |     | NULL    |                |
| location_provider         | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_continent  | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_country    | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_city       | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_latitude   | decimal(7,4)         | YES  |     | NULL    |                |
| location_geoip_longitude  | decimal(7,4)         | YES  |     | NULL    |                |
| user_id                   | varchar(200)         | YES  |     | NULL    |                |
| config_browser_engine     | varchar(10)          | NO   |     | NULL    |                |
| location_browser_lang     | varchar(20)          | NO   |     | NULL    |                |
+---------------------------+----------------------+------+-----+---------+----------------+
55 rows in set (0.00 sec)
mysql> ALTER TABLE log_visit ADD COLUMN config_os char(3) NOT NULL;
Query OK, 2484765 rows affected (1 min 21.69 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_browser_name varchar(10) NOT NULL;
Query OK, 2484765 rows affected (1 min 3.00 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_browser_version varchar(20) NOT NULL;
Query OK, 2484765 rows affected (1 min 21.36 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_resolution varchar(9) NOT NULL;
Query OK, 2484765 rows affected (1 min 8.46 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_pdf tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (59.04 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_flash tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 4.59 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_director tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 13.14 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_quicktime tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 29.80 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_realplayer tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 8.96 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_windowsmedia tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 0.62 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_gears tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 2.21 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_silverlight tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 21.97 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_java tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 13.32 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> ALTER TABLE log_visit ADD COLUMN config_cookie tinyint(1) NOT NULL;
Query OK, 2484765 rows affected (1 min 6.25 sec)
Records: 2484765  Duplicates: 0  Warnings: 0
mysql> DESCRIBE log_visit;
+---------------------------+----------------------+------+-----+---------+----------------+
| Field                     | Type                 | Null | Key | Default | Extra          |
+---------------------------+----------------------+------+-----+---------+----------------+
| idvisit                   | int(10) unsigned     | NO   | PRI | NULL    | auto_increment |
| idsite                    | int(10) unsigned     | NO   | MUL | NULL    |                |
| idvisitor                 | binary(8)            | NO   |     | NULL    |                |
| visitor_localtime         | time                 | NO   |     | NULL    |                |
| visitor_returning         | tinyint(1)           | NO   |     | NULL    |                |
| visitor_count_visits      | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_last   | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_order  | smallint(5) unsigned | NO   |     | NULL    |                |
| visitor_days_since_first  | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_first_action_time   | datetime             | NO   |     | NULL    |                |
| visit_last_action_time    | datetime             | NO   |     | NULL    |                |
| visit_exit_idaction_url   | int(10) unsigned     | YES  |     | 0       |                |
| visit_exit_idaction_name  | int(11) unsigned     | NO   |     | NULL    |                |
| visit_entry_idaction_url  | int(11) unsigned     | NO   |     | NULL    |                |
| visit_entry_idaction_name | int(11) unsigned     | NO   |     | NULL    |                |
| visit_total_actions       | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_searches      | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_events        | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_total_time          | smallint(5) unsigned | NO   |     | NULL    |                |
| visit_goal_converted      | tinyint(1)           | NO   |     | NULL    |                |
| visit_goal_buyer          | tinyint(1)           | NO   |     | NULL    |                |
| referer_type              | tinyint(1) unsigned  | YES  |     | NULL    |                |
| referer_name              | varchar(70)          | YES  |     | NULL    |                |
| referer_url               | text                 | NO   |     | NULL    |                |
| referer_keyword           | varchar(255)         | YES  |     | NULL    |                |
| config_id                 | binary(8)            | NO   |     | NULL    |                |
| config_os_version         | varchar(100)         | YES  |     | NULL    |                |
| config_device_type        | varchar(100)         | YES  |     | NULL    |                |
| config_device_brand       | varchar(100)         | YES  |     | NULL    |                |
| config_device_model       | varchar(100)         | YES  |     | NULL    |                |
| location_ip               | varbinary(16)        | NO   |     | NULL    |                |
| location_country          | char(3)              | NO   |     | NULL    |                |
| location_region           | char(2)              | YES  |     | NULL    |                |
| location_city             | varchar(255)         | YES  |     | NULL    |                |
| location_latitude         | float(10,6)          | YES  |     | NULL    |                |
| location_longitude        | float(10,6)          | YES  |     | NULL    |                |
| custom_var_k1             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v1             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k2             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v2             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k3             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v3             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k4             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v4             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_k5             | varchar(200)         | YES  |     | NULL    |                |
| custom_var_v5             | varchar(200)         | YES  |     | NULL    |                |
| location_provider         | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_continent  | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_country    | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_city       | varchar(100)         | YES  |     | NULL    |                |
| location_geoip_latitude   | decimal(7,4)         | YES  |     | NULL    |                |
| location_geoip_longitude  | decimal(7,4)         | YES  |     | NULL    |                |
| user_id                   | varchar(200)         | YES  |     | NULL    |                |
| config_browser_engine     | varchar(10)          | NO   |     | NULL    |                |
| location_browser_lang     | varchar(20)          | NO   |     | NULL    |                |
| config_os                 | char(3)              | NO   |     | NULL    |                |
| config_browser_name       | varchar(10)          | NO   |     | NULL    |                |
| config_browser_version    | varchar(20)          | NO   |     | NULL    |                |
| config_resolution         | varchar(9)           | NO   |     | NULL    |                |
| config_pdf                | tinyint(1)           | NO   |     | NULL    |                |
| config_flash              | tinyint(1)           | NO   |     | NULL    |                |
| config_director           | tinyint(1)           | NO   |     | NULL    |                |
| config_quicktime          | tinyint(1)           | NO   |     | NULL    |                |
| config_realplayer         | tinyint(1)           | NO   |     | NULL    |                |
| config_windowsmedia       | tinyint(1)           | NO   |     | NULL    |                |
| config_gears              | tinyint(1)           | NO   |     | NULL    |                |
| config_silverlight        | tinyint(1)           | NO   |     | NULL    |                |
| config_java               | tinyint(1)           | NO   |     | NULL    |                |
| config_cookie             | tinyint(1)           | NO   |     | NULL    |                |
+---------------------------+----------------------+------+-----+---------+----------------+
69 rows in set (0.01 sec)

Running the cont task manually to test if this fix has worked:

su - www-data -s /bin/bash
$ /web/stats.transitionnetwork.org/piwik/console core:archive --url=http://stats.transitionnetwork.org/
INFO CoreConsole[2015-07-13 09:21:39] ---------------------------
INFO CoreConsole[2015-07-13 09:21:39] INIT
INFO CoreConsole[2015-07-13 09:21:39] Running Piwik 2.14.0 as Super User
INFO CoreConsole[2015-07-13 09:21:39] ---------------------------
INFO CoreConsole[2015-07-13 09:21:39] NOTES
INFO CoreConsole[2015-07-13 09:21:39] - Reports for today will be processed at most every 1800 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO CoreConsole[2015-07-13 09:21:39] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO CoreConsole[2015-07-13 09:21:39] - Archiving was last executed without error 4 days 0 hours ago
INFO CoreConsole[2015-07-13 09:21:40] - Will process 2 websites with new visits since 4 days 0 hours , IDs: 1, 11
INFO CoreConsole[2015-07-13 09:21:40] - Will process 11 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13
INFO CoreConsole[2015-07-13 09:21:40] ---------------------------
INFO CoreConsole[2015-07-13 09:21:40] START
INFO CoreConsole[2015-07-13 09:21:40] Starting Piwik reports archiving...
INFO CoreConsole[2015-07-13 09:21:40] Will pre-process for website id = 1, period = day, date = last52
INFO CoreConsole[2015-07-13 09:21:40] - pre-processing all visits
INFO CoreConsole[2015-07-13 09:21:45] - pre-processing segment 1/1 pageUrl=@%2Fpse%2F
INFO CoreConsole[2015-07-13 09:21:47] Archived website id = 1, period = day, 1 segments, 0 visits in last last52 days, 0 visits today, Time elapsed: 2.441s
INFO CoreConsole[2015-07-13 09:21:47] Will pre-process for website id = 1, period = week, date = last6
INFO CoreConsole[2015-07-13 09:21:47] - pre-processing all visits
INFO CoreConsole[2015-07-13 09:21:47] - pre-processing segment 1/1 pageUrl=@%2Fpse%2F
INFO CoreConsole[2015-07-13 09:21:52] Archived website id = 1, period = week, 1 segments, 27086 visits in last last6 weeks, 7 visits this week, Time elapsed: 4.853s
INFO CoreConsole[2015-07-13 09:21:52] Will pre-process for website id = 1, period = month, date = last6
INFO CoreConsole[2015-07-13 09:21:52] - pre-processing all visits
INFO CoreConsole[2015-07-13 09:21:52] - pre-processing segment 1/1 pageUrl=@%2Fpse%2F
INFO CoreConsole[2015-07-13 09:21:58] Archived website id = 1, period = month, 1 segments, 155522 visits in last last6 months, 6981 visits this month, Time elapsed: 5.786s
INFO CoreConsole[2015-07-13 09:21:58] Will pre-process for website id = 1, period = year, date = last6
INFO CoreConsole[2015-07-13 09:21:58] - pre-processing all visits
INFO CoreConsole[2015-07-13 09:21:58] - pre-processing segment 1/1 pageUrl=@%2Fpse%2F
INFO CoreConsole[2015-07-13 10:10:14] Archived website id = 1, period = year, 1 segments, 2263093 visits in last last6 years, 192505 visits this year, Time elapsed: 2896.601s
INFO CoreConsole[2015-07-13 10:10:14] Will pre-process for website id = 1, period = range, date = previous30
INFO CoreConsole[2015-07-13 10:10:14] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:25] Archived website id = 1, period = range, 0 segments, 22160 visits in last previous30 ranges, 22160 visits this range, Time elapsed: 10.687s
INFO CoreConsole[2015-07-13 10:10:25] Archived website id = 1, 9 API requests, Time elapsed: 2924.651s [1/13 done]
INFO CoreConsole[2015-07-13 10:10:25] Will pre-process for website id = 3, period = day, date = last2
INFO CoreConsole[2015-07-13 10:10:25] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:27] Archived website id = 3, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:10:27] Will pre-process for website id = 3, period = week, date = last2
INFO CoreConsole[2015-07-13 10:10:27] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:28] Archived website id = 3, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.738s
INFO CoreConsole[2015-07-13 10:10:28] Will pre-process for website id = 3, period = month, date = last2
INFO CoreConsole[2015-07-13 10:10:28] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:30] Archived website id = 3, period = month, 0 segments, 17 visits in last last2 months, 0 visits this month, Time elapsed: 2.016s
INFO CoreConsole[2015-07-13 10:10:30] Will pre-process for website id = 3, period = year, date = last2
INFO CoreConsole[2015-07-13 10:10:30] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:34] Archived website id = 3, period = year, 0 segments, 636 visits in last last2 years, 105 visits this year, Time elapsed: 3.885s
INFO CoreConsole[2015-07-13 10:10:34] Archived website id = 3, 4 API requests, Time elapsed: 9.051s [4/13 done]
INFO CoreConsole[2015-07-13 10:10:35] Will pre-process for website id = 4, period = day, date = last2
INFO CoreConsole[2015-07-13 10:10:35] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:36] Archived website id = 4, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:10:36] Will pre-process for website id = 4, period = week, date = last2
INFO CoreConsole[2015-07-13 10:10:36] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:37] Archived website id = 4, period = week, 0 segments, 1 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.230s
INFO CoreConsole[2015-07-13 10:10:37] Will pre-process for website id = 4, period = month, date = last2
INFO CoreConsole[2015-07-13 10:10:37] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:40] Archived website id = 4, period = month, 0 segments, 6 visits in last last2 months, 2 visits this month, Time elapsed: 2.354s
INFO CoreConsole[2015-07-13 10:10:40] Will pre-process for website id = 4, period = year, date = last2
INFO CoreConsole[2015-07-13 10:10:40] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:42] Archived website id = 4, period = year, 0 segments, 168 visits in last last2 years, 29 visits this year, Time elapsed: 2.350s
INFO CoreConsole[2015-07-13 10:10:42] Archived website id = 4, 4 API requests, Time elapsed: 7.480s [5/13 done]
INFO CoreConsole[2015-07-13 10:10:42] Will pre-process for website id = 5, period = day, date = last52
INFO CoreConsole[2015-07-13 10:10:42] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:45] Archived website id = 5, period = day, 0 segments, 0 visits in last last52 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:10:45] Will pre-process for website id = 5, period = week, date = last6
INFO CoreConsole[2015-07-13 10:10:45] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:47] Archived website id = 5, period = week, 0 segments, 47 visits in last last6 weeks, 0 visits this week, Time elapsed: 2.346s
INFO CoreConsole[2015-07-13 10:10:47] Will pre-process for website id = 5, period = month, date = last6
INFO CoreConsole[2015-07-13 10:10:47] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:50] Archived website id = 5, period = month, 0 segments, 282 visits in last last6 months, 12 visits this month, Time elapsed: 2.347s
INFO CoreConsole[2015-07-13 10:10:50] Will pre-process for website id = 5, period = year, date = last6
INFO CoreConsole[2015-07-13 10:10:50] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:52] Archived website id = 5, period = year, 0 segments, 2385 visits in last last6 years, 344 visits this year, Time elapsed: 2.758s
INFO CoreConsole[2015-07-13 10:10:52] Archived website id = 5, 4 API requests, Time elapsed: 10.098s [6/13 done]
INFO CoreConsole[2015-07-13 10:10:52] Will pre-process for website id = 6, period = day, date = last2
INFO CoreConsole[2015-07-13 10:10:52] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:54] Archived website id = 6, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:10:54] Will pre-process for website id = 6, period = week, date = last2
INFO CoreConsole[2015-07-13 10:10:54] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:55] Archived website id = 6, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.088s
INFO CoreConsole[2015-07-13 10:10:55] Will pre-process for website id = 6, period = month, date = last2
INFO CoreConsole[2015-07-13 10:10:55] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:56] Archived website id = 6, period = month, 0 segments, 0 visits in last last2 months, 0 visits this month, Time elapsed: 1.207s
INFO CoreConsole[2015-07-13 10:10:56] Will pre-process for website id = 6, period = year, date = last2
INFO CoreConsole[2015-07-13 10:10:56] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:10:58] Archived website id = 6, period = year, 0 segments, 20 visits in last last2 years, 12 visits this year, Time elapsed: 1.914s
INFO CoreConsole[2015-07-13 10:10:58] Archived website id = 6, 4 API requests, Time elapsed: 5.354s [7/13 done]
INFO CoreConsole[2015-07-13 10:10:58] Will pre-process for website id = 7, period = day, date = last52
INFO CoreConsole[2015-07-13 10:10:58] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:01] Archived website id = 7, period = day, 0 segments, 0 visits in last last52 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:11:01] Will pre-process for website id = 7, period = week, date = last6
INFO CoreConsole[2015-07-13 10:11:01] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:04] Archived website id = 7, period = week, 0 segments, 706 visits in last last6 weeks, 1 visits this week, Time elapsed: 2.877s
INFO CoreConsole[2015-07-13 10:11:04] Will pre-process for website id = 7, period = month, date = last6
INFO CoreConsole[2015-07-13 10:11:04] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:07] Archived website id = 7, period = month, 0 segments, 3628 visits in last last6 months, 178 visits this month, Time elapsed: 2.871s
INFO CoreConsole[2015-07-13 10:11:07] Will pre-process for website id = 7, period = year, date = last6
INFO CoreConsole[2015-07-13 10:11:07] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:43] Archived website id = 7, period = year, 0 segments, 16678 visits in last last6 years, 4284 visits this year, Time elapsed: 96.458s
INFO CoreConsole[2015-07-13 10:12:43] Archived website id = 7, 4 API requests, Time elapsed: 105.509s [8/13 done]
INFO CoreConsole[2015-07-13 10:12:44] Will pre-process for website id = 8, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:44] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:45] Archived website id = 8, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:45] Will pre-process for website id = 8, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:45] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:46] Archived website id = 8, period = week, 0 segments, 3 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.089s
INFO CoreConsole[2015-07-13 10:12:46] Will pre-process for website id = 8, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:46] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:48] Archived website id = 8, period = month, 0 segments, 17 visits in last last2 months, 4 visits this month, Time elapsed: 1.916s
INFO CoreConsole[2015-07-13 10:12:48] Will pre-process for website id = 8, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:48] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:50] Archived website id = 8, period = year, 0 segments, 300 visits in last last2 years, 57 visits this year, Time elapsed: 1.918s
INFO CoreConsole[2015-07-13 10:12:50] Archived website id = 8, 4 API requests, Time elapsed: 6.020s [9/13 done]
INFO CoreConsole[2015-07-13 10:12:50] Will pre-process for website id = 9, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:50] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:51] Archived website id = 9, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:51] Will pre-process for website id = 9, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:51] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:52] Archived website id = 9, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.086s
INFO CoreConsole[2015-07-13 10:12:52] Will pre-process for website id = 9, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:52] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:53] Archived website id = 9, period = month, 0 segments, 2 visits in last last2 months, 0 visits this month, Time elapsed: 1.099s
INFO CoreConsole[2015-07-13 10:12:53] Will pre-process for website id = 9, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:53] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:55] Archived website id = 9, period = year, 0 segments, 97 visits in last last2 years, 26 visits this year, Time elapsed: 1.777s
INFO CoreConsole[2015-07-13 10:12:55] Archived website id = 9, 4 API requests, Time elapsed: 5.074s [10/13 done]
INFO CoreConsole[2015-07-13 10:12:55] Will pre-process for website id = 10, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:55] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:56] Archived website id = 10, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:56] Will pre-process for website id = 10, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:56] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:57] Archived website id = 10, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.089s
INFO CoreConsole[2015-07-13 10:12:57] Will pre-process for website id = 10, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:57] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:58] Archived website id = 10, period = month, 0 segments, 0 visits in last last2 months, 0 visits this month, Time elapsed: 1.082s
INFO CoreConsole[2015-07-13 10:12:58] Will pre-process for website id = 10, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:58] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:59] Archived website id = 10, period = year, 0 segments, 2 visits in last last2 years, 0 visits this year, Time elapsed: 1.080s
INFO CoreConsole[2015-07-13 10:12:59] Archived website id = 10, 4 API requests, Time elapsed: 4.348s [11/13 done]
INFO CoreConsole[2015-07-13 10:13:00] Will pre-process for website id = 12, period = day, date = last2
INFO CoreConsole[2015-07-13 10:13:00] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:01] Archived website id = 12, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:13:01] Will pre-process for website id = 12, period = week, date = last2
INFO CoreConsole[2015-07-13 10:13:01] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:01] Archived website id = 7, period = day, 0 segments, 0 visits in last last52 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:11:01] Will pre-process for website id = 7, period = week, date = last6
INFO CoreConsole[2015-07-13 10:11:01] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:04] Archived website id = 7, period = week, 0 segments, 706 visits in last last6 weeks, 1 visits this week, Time elapsed: 2.877s
INFO CoreConsole[2015-07-13 10:11:04] Will pre-process for website id = 7, period = month, date = last6
INFO CoreConsole[2015-07-13 10:11:04] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:11:07] Archived website id = 7, period = month, 0 segments, 3628 visits in last last6 months, 178 visits this month, Time elapsed: 2.871s
INFO CoreConsole[2015-07-13 10:11:07] Will pre-process for website id = 7, period = year, date = last6
INFO CoreConsole[2015-07-13 10:11:07] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:43] Archived website id = 7, period = year, 0 segments, 16678 visits in last last6 years, 4284 visits this year, Time elapsed: 96.458s
INFO CoreConsole[2015-07-13 10:12:43] Archived website id = 7, 4 API requests, Time elapsed: 105.509s [8/13 done]
INFO CoreConsole[2015-07-13 10:12:44] Will pre-process for website id = 8, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:44] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:45] Archived website id = 8, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:45] Will pre-process for website id = 8, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:45] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:46] Archived website id = 8, period = week, 0 segments, 3 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.089s
INFO CoreConsole[2015-07-13 10:12:46] Will pre-process for website id = 8, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:46] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:48] Archived website id = 8, period = month, 0 segments, 17 visits in last last2 months, 4 visits this month, Time elapsed: 1.916s
INFO CoreConsole[2015-07-13 10:12:48] Will pre-process for website id = 8, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:48] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:50] Archived website id = 8, period = year, 0 segments, 300 visits in last last2 years, 57 visits this year, Time elapsed: 1.918s
INFO CoreConsole[2015-07-13 10:12:50] Archived website id = 8, 4 API requests, Time elapsed: 6.020s [9/13 done]
INFO CoreConsole[2015-07-13 10:12:50] Will pre-process for website id = 9, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:50] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:51] Archived website id = 9, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:51] Will pre-process for website id = 9, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:51] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:52] Archived website id = 9, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.086s
INFO CoreConsole[2015-07-13 10:12:52] Will pre-process for website id = 9, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:52] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:53] Archived website id = 9, period = month, 0 segments, 2 visits in last last2 months, 0 visits this month, Time elapsed: 1.099s
INFO CoreConsole[2015-07-13 10:12:53] Will pre-process for website id = 9, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:53] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:55] Archived website id = 9, period = year, 0 segments, 97 visits in last last2 years, 26 visits this year, Time elapsed: 1.777s
INFO CoreConsole[2015-07-13 10:12:55] Archived website id = 9, 4 API requests, Time elapsed: 5.074s [10/13 done]
INFO CoreConsole[2015-07-13 10:12:55] Will pre-process for website id = 10, period = day, date = last2
INFO CoreConsole[2015-07-13 10:12:55] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:56] Archived website id = 10, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:12:56] Will pre-process for website id = 10, period = week, date = last2
INFO CoreConsole[2015-07-13 10:12:56] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:57] Archived website id = 10, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.089s
INFO CoreConsole[2015-07-13 10:12:57] Will pre-process for website id = 10, period = month, date = last2
INFO CoreConsole[2015-07-13 10:12:57] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:58] Archived website id = 10, period = month, 0 segments, 0 visits in last last2 months, 0 visits this month, Time elapsed: 1.082s
INFO CoreConsole[2015-07-13 10:12:58] Will pre-process for website id = 10, period = year, date = last2
INFO CoreConsole[2015-07-13 10:12:58] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:12:59] Archived website id = 10, period = year, 0 segments, 2 visits in last last2 years, 0 visits this year, Time elapsed: 1.080s
INFO CoreConsole[2015-07-13 10:12:59] Archived website id = 10, 4 API requests, Time elapsed: 4.348s [11/13 done]
INFO CoreConsole[2015-07-13 10:13:00] Will pre-process for website id = 12, period = day, date = last2
INFO CoreConsole[2015-07-13 10:13:00] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:01] Archived website id = 12, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:13:01] Will pre-process for website id = 12, period = week, date = last2
INFO CoreConsole[2015-07-13 10:13:01] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:02] Archived website id = 12, period = week, 0 segments, 0 visits in last last2 weeks, 0 visits this week, Time elapsed: 1.087s
INFO CoreConsole[2015-07-13 10:13:02] Will pre-process for website id = 12, period = month, date = last2
INFO CoreConsole[2015-07-13 10:13:02] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:03] Archived website id = 12, period = month, 0 segments, 0 visits in last last2 months, 0 visits this month, Time elapsed: 1.076s
INFO CoreConsole[2015-07-13 10:13:03] Will pre-process for website id = 12, period = year, date = last2
INFO CoreConsole[2015-07-13 10:13:03] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:04] Archived website id = 12, period = year, 0 segments, 0 visits in last last2 years, 0 visits this year, Time elapsed: 1.226s
INFO CoreConsole[2015-07-13 10:13:04] Archived website id = 12, 4 API requests, Time elapsed: 4.487s [12/13 done]
INFO CoreConsole[2015-07-13 10:13:04] Will pre-process for website id = 13, period = day, date = last52
INFO CoreConsole[2015-07-13 10:13:04] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:07] Archived website id = 13, period = day, 0 segments, 0 visits in last last52 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-07-13 10:13:07] Will pre-process for website id = 13, period = week, date = last6
INFO CoreConsole[2015-07-13 10:13:07] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:09] Archived website id = 13, period = week, 0 segments, 57 visits in last last6 weeks, 0 visits this week, Time elapsed: 2.045s
INFO CoreConsole[2015-07-13 10:13:09] Will pre-process for website id = 13, period = month, date = last6
INFO CoreConsole[2015-07-13 10:13:09] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:11] Archived website id = 13, period = month, 0 segments, 219 visits in last last6 months, 16 visits this month, Time elapsed: 2.067s
INFO CoreConsole[2015-07-13 10:13:11] Will pre-process for website id = 13, period = year, date = last6
INFO CoreConsole[2015-07-13 10:13:11] - pre-processing all visits
INFO CoreConsole[2015-07-13 10:13:13] Archived website id = 13, period = year, 0 segments, 438 visits in last last6 years, 258 visits this year, Time elapsed: 2.468s
INFO CoreConsole[2015-07-13 10:13:13] Archived website id = 13, 4 API requests, Time elapsed: 9.051s [13/13 done]
INFO CoreConsole[2015-07-13 10:13:13] Done archiving!
INFO CoreConsole[2015-07-13 10:13:13] ---------------------------
INFO CoreConsole[2015-07-13 10:13:13] SUMMARY
INFO CoreConsole[2015-07-13 10:13:13] Total visits for today across archived websites: 8
INFO CoreConsole[2015-07-13 10:13:13] Archived today's reports for 11 websites
INFO CoreConsole[2015-07-13 10:13:13] Archived week/month/year for 11 websites
INFO CoreConsole[2015-07-13 10:13:13] Skipped 2 websites: no new visit since the last script execution
INFO CoreConsole[2015-07-13 10:13:13] Skipped 0 websites day archiving: existing daily reports are less than 1800 seconds old
INFO CoreConsole[2015-07-13 10:13:13] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
INFO CoreConsole[2015-07-13 10:13:13] Total API requests: 49
INFO CoreConsole[2015-07-13 10:13:13] done: 11/13 85%, 8 vtoday, 11 wtoday, 11 wperiods, 49 req, 3093123 ms, no error
INFO CoreConsole[2015-07-13 10:13:13] Time elapsed: 3093.123s
INFO CoreConsole[2015-07-13 10:13:13] ---------------------------
INFO CoreConsole[2015-07-13 10:13:13] SCHEDULED TASKS
INFO CoreAdminHome[2015-07-13 10:13:13] Starting Scheduled tasks...
INFO CoreAdminHome[2015-07-13 10:13:13] Scheduler: executing task Piwik\Plugins\ExamplePlugin\Tasks.myTask...
INFO CoreAdminHome[2015-07-13 10:13:13] Scheduler: finished. Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-13 10:13:13] done
INFO CoreConsole[2015-07-13 10:13:13] ---------------------------

So that has run without a problem, note it almost takes an hour to run, so running it once an hour is not very sensible, so /etc/cron.d/piwik-archive has been edited to:

#5 2-23 * * * www-data /web/stats.transitionnetwork.org/piwik/console core:archive --url=http://stats.transitionnetwork.org/ >> /var/log/piwik-archive.log
5 */2 * * * www-data /web/stats.transitionnetwork.org/piwik/console core:archive --url=http://stats.transitionnetwork.org/ >> /var/log/piwik-archive.log

hours, status, totalhours changed; resolution deleted

chris — Thu, 16 Jul 2015 10:02:31 GMT

hours changed from 0.0 to 0.25
status changed from closed to reopened
resolution fixed deleted
totalhours changed from 1.91 to 2.16

This issue caused data loss:

We have a possibly serious issue with the columns being dropped.

https://github.com/piwik/piwik/issues/8304#issuecomment-121873392

This question has been asked:

Would anyone in this thread know whether you used to have those columns, or do you know whether these columns were never there in your tables?

https://github.com/piwik/piwik/issues/8304#issuecomment-121600534

So, looking at our backups... (ticket:763):

cd .zfs/snapshot/auto-UTC-2015-05-18_06.00/penguin.webarch.net/var/backups/mysql/sqldump/
ls -lah stats.sql
-rw-------  1 tn-penguin  tn-penguin   3.5G May 18 00:15 stats.sql
grep config_os stats.sql | less
  `config_os` char(3) NOT NULL,
  `config_os_version` varchar(100) DEFAULT NULL,
INSERT INTO `log_visit` (`idvisit`, `idsite`, `idvisitor`, `visitor_localtime`, `visitor_returning`, `visitor_count_visits`, `visitor_days_since_last`, `visitor_days_since_order`, `visitor_days_since_first`, `visit_first_action_time`, `visit_last_action_time`, `visit_exit_idaction_url`, `visit_exit_idaction_name`, `visit_entry_idaction_url`, `visit_entry_idaction_name`, `visit_total_actions`, `visit_total_searches`, `visit_total_events`, `visit_total_time`, `visit_goal_converted`, `visit_goal_buyer`, `referer_type`, `referer_name`, `referer_url`, `referer_keyword`, `config_id`, `config_os`, `config_os_version`, `config_browser_name`, `config_browser_version`, `config_device_type`, `config_device_brand`, `config_device_model`, `config_resolution`, `config_pdf`, `config_flash`, `config_java`, `config_director`, `config_quicktime`, `config_realplayer`, `config_windowsmedia`, `config_gears`, `config_silverlight`, `config_cookie`, `location_ip`, `location_browser_lang`, `location_country`, `location_region`, `location_city`, `location_latitude`, `location_longitude`, `custom_var_k1`, `custom_var_v1`, `custom_var_k2`, `custom_var_v2`, `custom_var_k3`, `custom_var_v3`, `custom_var_k4`, `custom_var_v4`, `custom_var_k5`, `custom_var_v5`, `location_provider`, `location_geoip_continent`, `location_geoip_country`, `location_geoip_city`, `location_geoip_latitude`, `location_geoip_longitude`, `user_id`, `config_browser_engine`) VALUES (1,1,

I have posted to the Github ticket to report that it looks like data was lost.

attachment set

chris — Fri, 17 Jul 2015 11:30:58 GMT

attachment set to piwik.png

chris — Mon, 27 Jul 2015 11:22:56 GMT

Closing this ticket as PiwikServer 2.14.12 is out ticket:867 and has a fix for the issue https://github.com/piwik/piwik/issues/8304#issuecomment-124527544

status changed; resolution set

chris — Mon, 27 Jul 2015 11:23:06 GMT

status changed from reopened to closed
resolution set to fixed