Transition Technology: Ticket #582: TN.org platform and sites

hours, totalhours changed

jim — Mon, 02 Sep 2013 09:31:39 GMT

hours changed from 0.0 to 0.15
totalhours changed from 0.0 to 0.15

This is blocking Ticket 578: Comments subscriptions: enable user alerts if a user has made a comment.

Adding time.

ed — Mon, 02 Sep 2013 11:21:46 GMT

anything about a software as a service to analyse TN.org to find the load spikes?

hours, totalhours changed

jim — Mon, 02 Sep 2013 11:50:48 GMT

hours changed from 0.0 to 0.05
totalhours changed from 0.15 to 0.2

Replying to ed:

anything about a software as a service to analyse TN.org to find the load spikes?

New Relic integration is at the BOA/server level, and all Drupal sites hosted will automatically start reporting their stats once set up... So that's unrelated to this.

ed — Mon, 02 Sep 2013 14:53:20 GMT

open atrium?

jim — Sun, 08 Sep 2013 16:18:23 GMT

Per ticket:590#comment:28, we need to apply the patch for blockcache_alter (no grants) to Drupal core...

hours, totalhours changed

jim — Mon, 09 Sep 2013 12:29:08 GMT

hours changed from 0.0 to 0.05
totalhours changed from 0.2 to 0.25

Regarding Location module patch -- our old version with the changes to the French Departments est ici: https://github.com/i-jk/OBSOLETE-Transition-Network/blob/610d54daa7bd5bc1480157241b7f5f273c6c3ba8/sites/all/modules/location/supported/location.fr.inc

Will compare current Locaton 6.x-dev with our old version and combine the best.

hours, totalhours, description changed

jim — Tue, 10 Sep 2013 10:57:33 GMT

hours changed from 0.0 to 0.1
totalhours changed from 0.25 to 0.35
description modified (diff)

Adding (per Proposal N on #590): need to get latest admin menu 3.x branch.

And these are the changes in functions I patched in LoginToboggan? -- need to make a patch in our repo and submit '@' part back to the project:

---- [Removed LOWER() in the queries ] ----
function logintoboggan_user($op, &$edit, &$user_edit, $category = NULL) {
[[snip]]
      if (isset($edit['name']) && db_result(db_query("SELECT uid FROM {users} WHERE mail = '%s' AND uid <> %d", $edit['name'], $uid))) {
        form_set_error('name', t('This name has already been taken by another user.'));
      }
      // Check that no user is using this email address for their name.
      if (isset($edit['mail']) && db_result(db_query("SELECT uid FROM {users} WHERE name = '%s' AND uid <> %d", $edit['mail'], $uid))) {
        form_set_error('mail', t('This e-mail has already been taken by another user.'));
[[snip]]
}
---- [ added '@' check and removed LOWER() in query] ----
function logintoboggan_user_login_validate($form, &$form_state) {
if (isset($form_state['values']['name']) && $form_state['values']['name'] &&
    strpos($form_state['values']['name'], '@') > 0) {
    if ($name = db_result(db_query("SELECT name FROM {users} WHERE mail = '%s'", $form_state['values']['name']))) {
      form_set_value($form['name'], $name, $form_state);
    }
  }
}

description changed

jim — Tue, 10 Sep 2013 11:15:04 GMT

description modified (diff)

As part of Proposal N #590, I'll be adding https://drupal.org/project/advagg to the site, too.

description changed

jim — Tue, 10 Sep 2013 17:32:01 GMT

description modified (diff)

Include Term Node Count too...

Given the downtime today and the meeting for tomorrow I'll wait for the all-clear before doing these updates. Pushed back 1 week for now...

hours, totalhours, description changed

jim — Mon, 07 Oct 2013 17:26:11 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.35 to 0.6
description modified (diff)

I've changed to use comment_notify instead of notifications_extra - better, simpler module.

Committed: Advagg, comment_notify and node_term_count added, admin_menu updated to 3.x-dev.

hours, totalhours, description changed

jim — Mon, 07 Oct 2013 18:10:03 GMT

hours changed from 0.0 to 0.15
totalhours changed from 0.6 to 0.75
description modified (diff)

Checked modules & update, all good, no changes needed to makefile and all relevant contrib updates will be brought into the next platform.

TO DO here:

GMap revert to our old version
Location and LoginToboggan? patches created and applied

Postponing captions issue on #596 until after this is settled - that'll happen on that ticket.

jim — Thu, 10 Oct 2013 14:04:01 GMT

Also: add views_content_cache!

hours, totalhours changed

jim — Sun, 13 Oct 2013 14:25:41 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.75 to 1.0

Update:

Views content cache added for use in #590.
Committed change to fix French Departments... This uses latest dev of Location plus the patch I submitted at https://drupal.org/node/635958#comment-7961467

Now just the GMap module to look into...

hours, totalhours, description changed

jim — Sun, 13 Oct 2013 14:27:24 GMT

hours changed from 0.0 to 0.05
totalhours changed from 1.0 to 1.05
description modified (diff)

description update.

hours, totalhours changed

jim — Sun, 13 Oct 2013 15:00:50 GMT

hours changed from 0.0 to 0.5
totalhours changed from 1.05 to 1.55

Platform now ready to build (tonight):

LoginToboggan patched per comment 7
Reverting back to our custom Gmap 1.x version as 2.x is still not ready :(

description changed

jim — Sun, 13 Oct 2013 15:01:09 GMT

description modified (diff)

hours, totalhours changed

jim — Mon, 14 Oct 2013 15:30:13 GMT

hours changed from 0.0 to 0.25
totalhours changed from 1.55 to 1.8

Have corrected a couple of minor issues/typos in makefile and am building the updated platform for testing with STG...

hours, totalhours changed

jim — Tue, 15 Oct 2013 10:11:58 GMT

hours changed from 0.0 to 0.5
totalhours changed from 1.8 to 2.3

Had a couple more issues that were my fault (typos) in the patch files I made, now fixed.

STG is NOT migrating though... opening a separate ticket for this as I think we need Octopus updating in line with the recent Barracuda updates.

jim — Tue, 15 Oct 2013 10:27:54 GMT

Blocked by #610 - Aegir migrate tasks hang on Drushrc load.

jim — Thu, 17 Oct 2013 16:49:01 GMT

Please note the patch I submitted before for the French Departments has been accepted and applied! Next time we can just use the stock version of the Location module.

jim — Sat, 30 Nov 2013 16:44:13 GMT

FYI I'm waiting on the BOA update #629 now.

hours, totalhours changed

jim — Tue, 07 Jan 2014 22:08:57 GMT

hours changed from 0.0 to 0.75
totalhours changed from 2.3 to 3.05

This is no longer blocked by #629, but seems still affected by #610 -- more over there on that.

Regarding updates:

Makefile updated to use latest D6 Pressflow core.
Minor tweaks too to clean up patches not needed.
Hang occurred on provision verify of 'Transition Network D6 S008' platform -- seems intact though, deleted and tried again
New platform 'Transition Network D6 S008' created manually (see command below), will clone Stg2 site into it now to test it.

The commands for making a new TN D6 platform is:

[log in as Aegir user, or sudo to root and run totn]
cd static
rm transitionnetwork.org-d6.make ; wget https://raw.github.com/transitionnetwork/transitionnetwork.org-d6.profile/master/transitionnetwork.org-d6.make ; drush make transitionnetwork.org-d6.make transition-network-d6-s008

The second line above removes any old makefile, gets the latest, runs it with Drush Make into directory in ~/static/transition-network-d6-s008 <-- edit the directory name increment number!

hours, totalhours changed

jim — Sun, 12 Jan 2014 14:23:28 GMT

hours changed from 0.0 to 0.35
totalhours changed from 3.05 to 3.4

Since #610 is now fixed, I'll now build a new TN PROD platform via Aegir, then migrate news and www sites to it. The platform and process was tested over on #610 with STG being cloned from PROD then migrated, so no further tests are required on the near-identical PROD sites.

For the benefit of future Aegir users, I'll quickly outline the process as I do it...

Add new platform: Name = Transition Network D6 P009, makefile = https://raw.github.com/transitionnetwork/transitionnetwork.org-d6.profile/master/transitionnetwork.org-d6.make, Drush make option = Normal -> 'Save'.
When built go to news.transitionnetwork.org -> Migrate -> Transition Network D6 P009 -> Migrate.
Test news works ok (it does).
Go to www.transitionnetwork.org -> Migrate -> Choose Transition Network D6 P009 -> Migrate.
Test www works ok (it does).

In addition to the above sites, I've also migrated the Space site, and cleaned up the various STG sites, plus the old (now empty) platforms.

Note that this work is actually around 15 mins to complete, but once I'd set up all the tasks then left Aegir to do the work, it took around 1/2 hour to complete. Hence the process for future Aegir users is:

Do the prep/planning/makefile tweaks
Build the platforms for STG and PROD, migrate STG, test
Set the PROD sites to migrate.
Set the (soon to be) empty platform to delete -- best to do STG only to start, so it's still possible to roll back PROD if needs be.
Make tea, eat a sandwich, have a doobie, cook your scag etc.
Final testing/tweaks/cleanup. Done.

hours, totalhours changed

jim — Sun, 12 Jan 2014 14:36:10 GMT

hours changed from 0.0 to 0.25
totalhours changed from 3.4 to 3.65

Hmm... hit an interesting 'Gotcha' moment during this update:

Everything worked as planned...
BUT the platform was newly-created, and therefore did not have the [platform root]/modules/o_contrib folder symlinked into it -- this happens as part of the /var/xdrago/daily.sh script which is run at 6.45 every day and does a bunch of things.
This meant that though the site was ok and ran fine for the User1 account, anonymous users got a white screen that was caused by the lack of Block Cache Alter module (provided by the o_contrib folder).
Rebuilding the registry fixed the issue, then I ran /var/xdrago/daily.sh when logged in as root to fix the problem.
I then manually re-enabled Block Cache Alter to fix the issue.

There is an issue over on Drupal.org for this, and I added my comment: Add o_contrib modules and Boost cache dir on new platforms when created.

The simple (but slightly annoying) solution (in lieu of a proper fix) is to include the required modules in the Makefile...

So left on this ticket is: A) Add Block Cache Alter (any any other required modules in o_contrib) to the makefile. B) Apply Patch in comment:7 to LoginToboggan. C) Possibly make a GMap patch and add to our makefile so our custom markers appear ok -- per #615.

priority, status changed

jim — Sun, 12 Jan 2014 14:37:07 GMT

priority changed from critical to major
status changed from new to assigned

Downgrading, only task A, B and C in comment:24 required at this point, all other updates done.

ed — Mon, 13 Jan 2014 08:51:59 GMT

where will you document teh simple but annoying solution in lieu of a proper fix ?

hours, totalhours, description changed

jim — Wed, 15 Jan 2014 22:17:45 GMT

hours changed from 0.0 to 0.2
totalhours changed from 3.65 to 3.85
description modified (diff)

Hmmm... Drupal core has just had a security update! 6.30 is now available.

But according to https://drupal.org/SA-CORE-2014-001, none of the risks affect us. We can therefore wait for the next sensible time to update.

I also note that

I'd missed Elysia Cron from the makefile, causing #677, now fixed.
I've left the old GMap (our fork of 1.x) in the makefile too. This means part (C) of comment:24 is necessary to have a proper makefile meet our GMap needs.
And the LoginToboggan? patch in (B) is already done in comment:15.
And I've added Block Cache Alter to the makefile.

Updating summary...

cc, hours, totalhours, description, summary changed

jim — Wed, 15 Jan 2014 22:21:59 GMT

cc sam added
hours changed from 0.0 to 0.05
totalhours changed from 3.85 to 3.9
description modified (diff)
summary changed from Updates to TN.org -- Sept 2013 to TN.org platform and sites

Better title/description.

Also cc Sam FYI.

hours, totalhours changed

jim — Wed, 29 Jan 2014 16:23:11 GMT

hours changed from 0.0 to 0.05
totalhours changed from 3.9 to 3.95

FYI I've added a note to the TN.org Update Status admin page and told it to ignore 6.30 as the security risk/fix simply does not apply to us.

Also, these notes are provided by Update Status Advanced module, and should be kept up to date and mirror any key pins/ignores in the makefile.

I'll try to do the GMap patch and associated makefile tweaks later tonight so we can then have just 'normal' update tracking on this ticket.

hours, totalhours changed

jim — Wed, 05 Feb 2014 14:43:58 GMT

hours changed from 0.0 to 0.5
totalhours changed from 3.95 to 4.45

I've made a few commits to the makefile and patches folder that:

Comments out reference to our old custom Gmap module, uses 6.x-2.x-dev branch with our patch (see below).
Includes a new patch for the custom marker set we use on TN.org for some map views
Renamed the patches to start "tnXX_" where XX is the number used in the makefile to help people find the right ones.
Updated the core to the Aegir Project's recommended 6.30.1 version (per instruction at top of makefile, using this to find current best version: https://github.com/omega8cc/nginx-for-drupal/search?q=pressflow).

Note to make a patch that includes the binary files (marker PNGs) I had to follow the instructions here: http://mediatribe.net/en/node/63 -- e.g.

[after I had made the changes to a freshly cloned gmap-6.x-2.x-dev branch]
git diff --full-index --binary > tn03_gmap-tn-markers.patch

The key bits being the --full-index and --binary to tell Git to include non-text files in the patch.

I'll now try building the new platform to test.

cc, hours, description, owner, totalhours changed

jim — Wed, 05 Feb 2014 15:01:33 GMT

cc jim added
hours changed from 0.0 to 0.1
description modified (diff)
owner changed from jim to sam
totalhours changed from 4.45 to 4.55

The S010 platform built properly, and the 'transition' folder with our markers is present here: /data/disk/tn/static/transition-network-d6-s010/sites/all/modules/contrib/gmap/markers

So the last 'non-standard' thing with our modules has been resolved and this ticket is able to just track the updates as they come in now. Reassigning to Sam for now, and updating main description so the current/best practice TN.org platform build numbers are documented.

Sam: I'll stay CCed on this ticket so can answer questions and help out as needed. Otherwise we're in great shape so it's all yours! (or delegate as you see fit!)

hours, totalhours changed

jim — Sun, 09 Feb 2014 13:38:20 GMT

hours changed from 0.0 to 0.25
totalhours changed from 4.55 to 4.8

There was a simple Tagadelic module security update. The latest would have been covered by the makefile anyway, but I've done a hotfix on PROD anyway as it was a drop-in replacement (no db update). So there is no need to update PRoD now...

STG and STG2 should be migrated to the S010 platform when possible, I've updated the module there too. (they're on S008 now).