Transition Technology: Ticket #382: Measurement and tracking requirements for TN and PSE

jim — Tue, 13 Dec 2011 23:59:47 GMT

Piwik might be needed, but I'd be surprised if GA + Drupal + server logs couldn't do it... Depends on outcome to #371, at least temporarily.

ed — Wed, 21 Dec 2011 00:26:13 GMT

work started on wiki here: https://wiki.transitionnetwork.org/Sharing_Engine/Measurement_requirements

Laura - please add TN.org requirements Ed to keep working on PSE requirements

Need good outline by PSE meet on 10/01/12

hours, totalhours changed

jim — Sun, 11 Mar 2012 20:30:12 GMT

hours changed from 0.0 to 0.1
totalhours changed from 0.0 to 0.1

Following the work on the widget structure, it's clear we're measuring 2 related parts:

the 'view widget'... This will be a view or panels page with responsive theme within an iFrame on a participating website. Piwik/GA will therefore already be present in the iFrame.
the 'entry widget'... Will be an iFrame again, but opened in a bigger popup div... Same as above, will already be tracked by Piwik because Drupal will generate that page.

So the mechanisms are in place, the key thing is ensure we can 'partition' Piwik data from the PSE... I think this is covered too, as the planned URL structure will states all PSE-related pages always start with with '/pse'.

hours, priority, totalhours changed

chris — Fri, 11 May 2012 19:38:56 GMT

hours changed from 0.0 to 0.2
priority changed from major to critical
totalhours changed from 0.1 to 0.3

Replying to jim:

the planned URL structure will states all PSE-related pages always start with with '/pse'.

Is the above still valid?

I have updated the dev site via https://dev.transitionnetwork.org/admin/settings/piwik setting it to use the dev version of pwiki at https://piwik.transitionnetwork.org/ so we can do some testing.

Do we want role specific tracking?

https://dev.transitionnetwork.org/admin/settings/piwik#edit-piwik-visibility-roles-0-wrapper

Are we going to allow opt-outs?

https://dev.transitionnetwork.org/admin/settings/piwik#edit-piwik-custom-0-wrapper

Jim - can the PSE code be installed on the dev server so I can do some testing?

hours, priority, totalhours changed

chris — Mon, 14 May 2012 13:35:54 GMT

hours changed from 0.0 to 0.5
priority changed from critical to blocker
totalhours changed from 0.3 to 0.8

Jim, I have been thinking about this quite a lot and I think the way to go with this is to set up a couple of example transition test sites and related accounts on the dev server and to put up some test widgets, I'm happy to do as much of this as I can to reduce the work for you.

Can you point me in the right direction for these things:

Updating the dev server with the latest PSE code and database updates (if any)
Permissions for new test accounts that can be used with test widgets
Code / URL's for widgets for including in test sites

I'm also happy to help with documenting how to install a widget.

Thanks.

hours, totalhours changed

jim — Sun, 29 Jul 2012 22:30:23 GMT

hours changed from 0.0 to 0.1
totalhours changed from 0.8 to 0.9

Hi there, thought I'd add to this since a few things have changed:

yep, all widget URLs start with /pse
We don't need transition test sites, any site will do. We just need a functioning DEV box. I'll put (have already) it widget on my site, and can add it to a few others if needs be. I'm sure Laura can too.
Re DEV: It'd be good to have its database and files pulled from LIVE ASAP, so we have a proper mirror. I've merged LIVE into DEV as far as the Git repo is concerend, so a git pull origin development in the site root will bring things up to speed with LIVE (and make the database work properly) and include all the widgety bits.
There's basic information on how to install the widget on the 'get the widget' page that participating webmasters have access to. Would be good to add additional instructions to a page on the site that we can link to from the 'get' page.

priority changed

ed — Thu, 18 Oct 2012 14:10:18 GMT

priority changed from blocker to critical

moving to critical as not a blocker, but still needs action

cc, hours, totalhours changed

chris — Sat, 10 Nov 2012 10:16:10 GMT

cc ed added
hours changed from 0.0 to 0.1
totalhours changed from 0.9 to 1.0

I think that perhaps we need to have a discussion on tracking users and the ethics of this and how we want to approach it.

I really don't agree with the latest update from Piwik, see below, I think Piwik should give us the options here:

http://mikebeach.org/2012/11/09/piwik-1-9-2-does-not-honor-donottrack-by-default-blame-ie10/

I think we should respect the DNT header even if browsers start setting it by default.

We still are using two webbugs on the network site and widget, Piwik and GA.

Posted: 08 Nov 2012 11:28 PM PST http://piwik.org/blog/2012/11/piwik-1-9-2/

starting from Piwik 1.9.2, Do Not Track browser setting will be ignored for IE 10 browsers. Because all IE10 browsers have DNT enabled, we cannot afford not to record in Piwik all IE10. So we decided to ignore the DNT setting for all IE10 users.

hours, totalhours changed

chris — Tue, 13 Nov 2012 14:15:00 GMT

hours changed from 0.0 to 0.1
totalhours changed from 1.0 to 1.1

There is now a ticket for the IE10 / DoNotTrack issue on the Piwik trac site.

chris — Tue, 29 Jan 2013 17:32:59 GMT

ed — Mon, 18 Feb 2013 10:52:22 GMT

catching up here - also reading #431, so leaving this comment: as long as we can see this I'm happy bunny:

how many widgets are in action
traffic generated by widdgets
number of projects generated from widgets

chris — Thu, 30 May 2013 22:04:40 GMT

This ticket needs some work, as a munimum documenting how to get the widget stats, which might perhaps use the new segment editor, which is in the latest version, ticket:556, see:

cc, hours, status, totalhours changed

chris — Tue, 04 Jun 2013 12:26:06 GMT

cc laura removed
hours changed from 0.0 to 1.7
status changed from new to accepted
totalhours changed from 1.1 to 2.8

The "segment editor" in Piwik is fairly awesome, this is how I have used it so far:

Find the active PSE URLs by going to Actions > Pages > PSE > View Sidebar in Piwik.
For each URL like https://www.transitionnetwork.org/pse/view/sidebar/16928/6290 (and which has had significant traffic in the last year) use the "Add a new segment" from the pull down menu and create the segment using:
- Name: take this from the user ID in the URL, eg the username for https://www.transitionnetwork.org/pse/view/sidebar/16928/6290 can be found via https://www.transitionnetwork.org/user/16928, eg "PSE Test Initiative"
- Add the condition: Actions > Page URL > Contains > /pse/
- Add the condition: Actions > Page URL > Contains > /16928/
- Edit: This segment is visible to: all users (it defaults to "me")
- Edit: Segmented reports are pre-processed (faster, requires archive.php cron)

I have also set up a Segment report for all URLs containing /pse/ and this might be more useful than the PSE / User segment reports as you can get reports like Actions > Page Titles to see a good overview of PSE activity.

I'm also a bit concerned that segmenting it by the user account that set up the widget isn't the best way to do it? Jim, Ed, thoughts?

PS I have removed Laura from this ticket and accepted it.

jim — Thu, 06 Jun 2013 12:15:48 GMT

Replying to chris:

I'm also a bit concerned that segmenting it by the user account that set up the widget isn't the best way to do it? Jim, Ed, thoughts?

I'd argue the second number in the URL is just as/more important -- that's the node ID of the imitative. E.g. given "https://www.transitionnetwork.org/pse/view/sidebar/16928/6290":

1st = http://www.transitionnetwork.org/user/16928 is our webmaster who owns the widget.
2nd = http://www.transitionnetwork.org/node/6290 is the Initiative node it's for.

So really this is one for Ed: Do we want to segment by both of these, one, or neither?

hours, totalhours changed

chris — Fri, 07 Jun 2013 09:58:58 GMT

hours changed from 0.0 to 0.25
totalhours changed from 2.8 to 3.05

Replying to jim:

I'd argue the second number in the URL is just as/more important

I did wonder about that but was concerned that it might not be able to be done by URL, when matching the URL we have:

Is
Is not
Contains
Does not contain

Consider this URL:

http://www.transitionnetwork.org/pse/view/sidebar/7292/71

If we try to match it with "Contains /71" then we are also going to match any user who has an ID that starts with 71. The ones I have set up use matches like "Contains /7292/".

If we do need stats based on the sites hosting the widget we might be best of using the HTTP Referer field, eg Referrer Is http://transition.example.org/ Or Referrer Is http://www.transition.example.org/

But perhaps the PSE segment is enough as it is and it would be better to delete the user PSE segnemnt?

ed — Wed, 19 Jun 2013 16:55:08 GMT

Sorry for absence, book PR mayhem.
Err. I would like to attend to this in mid July if possible? Perhaps starting with:
1. Jim time: 10/7: current options and future needs for international initiative widget
2. Chris time: w/c 15/7: deeper understanding of piwik and follow up having jim's input
?
On 7 Jun 2013, at 10:58, Transiton Technology Trac <trac@tech.transitionnetwork.org> wrote:
> #382: Measurement and tracking requirements for TN and PSE
> -----------------------------------+--------------------------------------
>           Reporter:  ed           |                      Owner:  chris
>               Type:  enhancement  |                     Status:  accepted
>           Priority:  critical     |                  Milestone:  PSE
>          Component:  Piwik        |                 Resolution:
>           Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0.25         |                  Billable?:  1
>        Total Hours:  2.8          |
> -----------------------------------+--------------------------------------
> Changes (by chris):
>
> * hours:  0.0 => 0.25
> * totalhours:  2.8 => 3.05
>
>
> Comment:
>
> Replying to [comment:14 jim]:
>>
>> I'd argue the second number in the URL is just as/more important
>
> I did wonder about that but was concerned that it might not be able to be
> done by URL, when matching the URL we have:
>
> * Is
> * Is not
> * Contains
> * Does not contain
>
> Consider this URL:
>
> * http://www.transitionnetwork.org/pse/view/sidebar/7292/71
>
> If we try to match it with "Contains {{{/71}}}" then we are also going to
> match any user who has an ID that starts with 71. The ones I have set up
> use matches like "Contains {{{/7292/}}}".
>
> If we do need stats based on the sites hosting the widget we might be best
> of using the HTTP Referer field, eg Referrer Is
> http://transition.example.org/ Or Referrer Is
> http://www.transition.example.org/
>
> But perhaps the PSE segment is enough as it is and it would be better to
> delete the user PSE segnemnt?
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/382#comment:15>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.
~~~~
Ed Mitchell: Transition Network Web manager
web: http://transitionnetwork.org
project blog: http://transitionnetwork.org/blogs/ed-mitchell
mobile: +44 (0)7807 141 828
office hours: Mon, Tues, Wed, Thurs
company no. 6135675  charity no. 1128675
Subscribe to Transition news: http://tinyurl.com/transitionregister
~~~~

priority, type changed

ed — Tue, 09 Jul 2013 13:59:12 GMT

priority changed from critical to major
type changed from enhancement to maintenance

milestone changed

ed — Tue, 09 Jul 2013 13:59:44 GMT

milestone changed from PSE to Maintenance

hours, totalhours changed

chris — Wed, 17 Jul 2013 20:26:39 GMT

hours changed from 0.0 to 4.0
totalhours changed from 3.05 to 7.05

On Tuesday 16th I spent the day with Ed going though Piwik stats and also looking at the server load issue and also some other items, I'm adding this comment here to record 4 hours of time spent.

chris — Wed, 02 Apr 2014 10:26:07 GMT

Think this is OK to close now, new tickets can be opened for new web stat issues.

status changed; resolution set

chris — Wed, 02 Apr 2014 10:35:37 GMT

status changed from accepted to closed
resolution set to fixed