Transition Technology: Ticket #363: Bring DEV up to date with LIVE

chris — Wed, 19 Oct 2011 17:42:20 GMT

It should be OK!

I'll run it if you want, it's been a while since I ran it so I dunno what will happen ;-)

It shouldn't cause any harm, if it doesn't work shout!

chris — Thu, 20 Oct 2011 09:08:23 GMT

Did it go OK?

hours, priority, totalhours changed

jim — Thu, 20 Oct 2011 11:31:43 GMT

hours changed from 0.0 to 0.5
priority changed from major to minor
totalhours changed from 0.0 to 0.5

I started running it but realised the script was doing a 'dumb' DB dump... This included many many megabytes of pointless data and the DEV machine, at the time, was running like a pig...

A look at top showed mysqldump and rdiff-backup and other processes doing stuff. It took ages to get a plain SVN update to happen, so I didn't want to cripple or crash DEV by importing huge dumpfiles.

So I aborted the backup2kiwi script it after I saw it trying to dump many unnecessary tables (cache_*, sessions etc etc) and just used the 'Backup and Migrate' module instead. That made a 13Mb gz file (of a 60mb mysql file) and uploaded onto DEV no problems.

So there remains the following minor issues:

sites/default/files folder ideally needs rsyncing from LIVE to DEV
DEV (and by extension LIVE) SVN performance is awful -- at ~1.30am anyway... did this coincide with your own backup processes? Is it the lack of memory on DEV? Does kiwi need a cleanse?
the backup2kiwi script needs some table filtering to avoid unnecessary load
only the main live DB got moved, not the workspaces or sharing engine -- though these are not really needed on DEV for now.

But DEV is now in sync with LIVE... downgrading but will leave open for comments.

chris — Thu, 20 Oct 2011 13:38:57 GMT

OK, I'll do some on it tonight

hours, totalhours changed

chris — Thu, 20 Oct 2011 15:51:44 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.5 to 0.75

I have added this to the mysql database backup script:

@skip_tables = qw[cache cache_block cache_content cache_emfield_xml cache_filter cache_form cache_hierarchical_select cache_location cache_media_youtube_status cache_menu cache_mollom cache_page cache_path cache_rules cache_update cache_views cache_views_data sessions];

And I have checked that neither mediawiki or piwik have any tables with the same names, so I think that has fixed that one.

I can't see anything really that has changed which would make svn slower than before...

I think it would be safe to sync live to dev tonight, I'll do this if you want.

hours, totalhours changed

jim — Thu, 20 Oct 2011 16:06:09 GMT

hours changed from 0.0 to 0.1
totalhours changed from 0.75 to 0.85

Just checking the settings at: https://www.transitionnetwork.org/admin/content/backup_migrate/profile/list/edit/default

Seems the search_* and watchdog tables should also be excluded...

DEV is ready for us to use for now, but I'm sure it'll need updating again in a few weeks so we can try then.

Thanks.

hours, status, totalhours changed; resolution set

chris — Thu, 20 Oct 2011 16:19:31 GMT

hours changed from 0.0 to 0.1
status changed from new to closed
resolution set to fixed
totalhours changed from 0.85 to 0.95

OK I have added:

search_dataset search_index search_node_links search_total watchdog

Closing this ticket now.