Measurement and tracking requirements for TN and PSE

[ed]

to list the measurement and tracking requirements for TN and PSE so that we can assess piwik vs google and decide which way to go.

[jim]

Piwik might be needed, but I'd be surprised if GA + Drupal + server logs couldn't do it... Depends on outcome to #371, at least temporarily.

[ed]

work started on wiki here:
​https://wiki.transitionnetwork.org/Sharing_Engine/Measurement_requirements

Laura - please add TN.org requirements
Ed to keep working on PSE requirements

Need good outline by PSE meet on 10/01/12

[jim]

Following the work on the ​widget structure, it's clear we're measuring 2 related parts:

1. the 'view widget'... This will be a view or panels page with responsive theme within an iFrame on a participating website. Piwik/GA will therefore already be present in the iFrame.
2. the 'entry widget'... Will be an iFrame again, but opened in a bigger popup div... Same as above, will already be tracked by Piwik because Drupal will generate that page.

So the mechanisms are in place, the key thing is ensure we can 'partition' Piwik data from the PSE... I think this is covered too, as the ​planned URL structure will states all PSE-related pages always start with with '/pse'.

[chris]

Replying to jim:

the ​planned URL structure will states all PSE-related pages always start with with '/pse'.

Is the above still valid?

I have updated the dev site via ​https://dev.transitionnetwork.org/admin/settings/piwik setting it to use the dev version of pwiki at ​https://piwik.transitionnetwork.org/ so we can do some testing.

Do we want role specific tracking?

​https://dev.transitionnetwork.org/admin/settings/piwik#edit-piwik-visibility-roles-0-wrapper

Are we going to allow opt-outs?

​https://dev.transitionnetwork.org/admin/settings/piwik#edit-piwik-custom-0-wrapper

Jim - can the PSE code be installed on the dev server so I can do some testing?

[chris]

Jim, I have been thinking about this quite a lot and I think the way to go with this is to set up a couple of example transition test sites and related accounts on the dev server and to put up some test widgets, I'm happy to do as much of this as I can to reduce the work for you.

Can you point me in the right direction for these things:

• Updating the dev server with the latest PSE code and database updates (if any)
• Permissions for new test accounts that can be used with test widgets
• Code / URL's for widgets for including in test sites

I'm also happy to help with documenting how to install a widget.

Thanks.

[jim]

Hi there, thought I'd add to this since a few things have changed:

1. yep, all widget URLs start with /pse
2. We don't need transition test sites, any site will do. We just need a functioning DEV box. I'll put (have already) it widget on my site, and can add it to a few others if needs be. I'm sure Laura can too.
3. Re DEV: It'd be good to have its database and files pulled from LIVE ASAP, so we have a proper mirror. I've merged LIVE into DEV as far as the Git repo is concerend, so a git pull origin development in the site root will bring things up to speed with LIVE (and make the database work properly) and include all the widgety bits.
4. There's basic information on how to install the widget on the 'get the widget' page that participating webmasters have access to. Would be good to add additional instructions to a page on the site that we can link to from the 'get' page.

[ed]

moving to critical as not a blocker, but still needs action

[chris]

I think that perhaps we need to have a discussion on tracking users and
the ethics of this and how we want to approach it.

I really don't agree with the latest update from Piwik, see below, I
think Piwik should give us the options here:

• ​http://mikebeach.org/2012/11/09/piwik-1-9-2-does-not-honor-donottrack-by-default-blame-ie10/

I think we should respect the DNT header even if browsers start
setting it by default.

We still are using two webbugs on the network site and widget, Piwik and GA.

---

Posted: 08 Nov 2012 11:28 PM PST
​http://piwik.org/blog/2012/11/piwik-1-9-2/

starting from Piwik 1.9.2, Do Not Track browser setting will be ignored for
IE 10 browsers. Because all IE10 browsers have DNT enabled, we cannot
afford not to record in Piwik all IE10. So we decided to ignore the DNT
setting for all IE10 users.

---

[chris]

There is now a ​ticket for the IE10 / DoNotTrack issue on the Piwik trac site.

[chris]

See also wiki:PiwikServer and ticket:371#comment:26

[ed]

catching up here - also reading #431, so leaving this comment:
as long as we can see this I'm happy bunny:

1. how many widgets are in action
2. traffic generated by widdgets
3. number of projects generated from widgets

[chris]

This ticket needs some work, as a munimum documenting how to get the widget stats, which might perhaps use the new segment editor, which is in the latest version, ticket:556, see:

• ​https://piwik.org/blog/2013/05/last-piwik-1-x-release-v1-12-new-features-api-improvements-screenshots/#toc-new-big-feature-segment-editor
• ​https://piwik.org/docs/segmentation/

[chris]

The "segment editor" in Piwik is fairly awesome, this is how I have used it so far:

1. Find the active PSE URLs by going to Actions > Pages > PSE > View Sidebar in Piwik.
2. For each URL like ​https://www.transitionnetwork.org/pse/view/sidebar/16928/6290 (and which has had significant traffic in the last year) use the "Add a new segment" from the pull down menu and create the segment using:
   • Name: take this from the user ID in the URL, eg the username for ​https://www.transitionnetwork.org/pse/view/sidebar/16928/6290 can be found via ​https://www.transitionnetwork.org/user/16928, eg "PSE Test Initiative"
   • Add the condition: Actions > Page URL > Contains > /pse/
   • Add the condition: Actions > Page URL > Contains > /16928/
   • Edit: This segment is visible to: all users (it defaults to "me")
   • Edit: Segmented reports are pre-processed (faster, requires archive.php cron)

I have also set up a Segment report for all URLs containing /pse/ and this might be more useful than the PSE / User segment reports as you can get reports like Actions > Page Titles to see a good overview of PSE activity.

I'm also a bit concerned that segmenting it by the user account that set up the widget isn't the best way to do it? Jim, Ed, thoughts?

PS I have removed Laura from this ticket and accepted it.

[jim]

Replying to chris:

I'm also a bit concerned that segmenting it by the user account that set up the widget isn't the best way to do it? Jim, Ed, thoughts?

I'd argue the second number in the URL is just as/more important -- that's the node ID of the imitative. E.g. given "​https://www.transitionnetwork.org/pse/view/sidebar/16928/6290":

• 1st = ​http://www.transitionnetwork.org/user/16928 is our webmaster who owns the widget.
• 2nd = ​http://www.transitionnetwork.org/node/6290 is the Initiative node it's for.

So really this is one for Ed: Do we want to segment by both of these, one, or neither?

[chris]

Replying to jim:

I'd argue the second number in the URL is just as/more important

I did wonder about that but was concerned that it might not be able to be done by URL, when matching the URL we have:

• Is
• Is not
• Contains
• Does not contain

Consider this URL:

• ​http://www.transitionnetwork.org/pse/view/sidebar/7292/71

If we try to match it with "Contains /71" then we are also going to match any user who has an ID that starts with 71. The ones I have set up use matches like "Contains /7292/".

If we do need stats based on the sites hosting the widget we might be best of using the HTTP Referer field, eg Referrer Is ​http://transition.example.org/ Or Referrer Is ​http://www.transition.example.org/

But perhaps the PSE segment is enough as it is and it would be better to delete the user PSE segnemnt?

[ed]

Sorry for absence, book PR mayhem. 

Err. I would like to attend to this in mid July if possible? Perhaps starting with:  
1. Jim time: 10/7: current options and future needs for international initiative widget
2. Chris time: w/c 15/7: deeper understanding of piwik and follow up having jim's input
? 

On 7 Jun 2013, at 10:58, Transiton Technology Trac <trac@tech.transitionnetwork.org> wrote:

> #382: Measurement and tracking requirements for TN and PSE
> -----------------------------------+--------------------------------------
>           Reporter:  ed           |                      Owner:  chris
>               Type:  enhancement  |                     Status:  accepted
>           Priority:  critical     |                  Milestone:  PSE
>          Component:  Piwik        |                 Resolution:
>           Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0.25         |                  Billable?:  1
>        Total Hours:  2.8          |
> -----------------------------------+--------------------------------------
> Changes (by chris):
> 
> * hours:  0.0 => 0.25
> * totalhours:  2.8 => 3.05
> 
> 
> Comment:
> 
> Replying to [comment:14 jim]:
>> 
>> I'd argue the second number in the URL is just as/more important
> 
> I did wonder about that but was concerned that it might not be able to be
> done by URL, when matching the URL we have:
> 
> * Is
> * Is not
> * Contains
> * Does not contain
> 
> Consider this URL:
> 
> * http://www.transitionnetwork.org/pse/view/sidebar/7292/71
> 
> If we try to match it with "Contains {{{/71}}}" then we are also going to
> match any user who has an ID that starts with 71. The ones I have set up
> use matches like "Contains {{{/7292/}}}".
> 
> If we do need stats based on the sites hosting the widget we might be best
> of using the HTTP Referer field, eg Referrer Is
> http://transition.example.org/ Or Referrer Is
> http://www.transition.example.org/
> 
> But perhaps the PSE segment is enough as it is and it would be better to
> delete the user PSE segnemnt?
> 
> -- 
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/382#comment:15>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.

~~~~
Ed Mitchell: Transition Network Web manager
web: http://transitionnetwork.org
project blog: http://transitionnetwork.org/blogs/ed-mitchell
mobile: +44 (0)7807 141 828
office hours: Mon, Tues, Wed, Thurs
company no. 6135675  charity no. 1128675
Subscribe to Transition news: http://tinyurl.com/transitionregister
~~~~

[chris]

On Tuesday 16th I spent the day with Ed going though Piwik stats and also looking at the server load issue and also some other items, I'm adding this comment here to record 4 hours of time spent.

[chris]

Think this is OK to close now, new tickets can be opened for new web stat issues.