Bring DEV up to date with LIVE

[jim]

DEV needs to be brought up to date using Chris' live2dev script so that Laura and Jim can develop against it without needing releases to LIVE for every change.

Jim's gonna try the script tonight after he does the SVN update...

Chris: Shout now if it's gonna blow!

[chris]

It should be OK!

I'll run it if you want, it's been a while since I ran it so I dunno what will happen ;-)

It shouldn't cause any harm, if it doesn't work shout!

[chris]

Did it go OK?

[jim]

I started running it but realised the script was doing a 'dumb' DB dump... This included many many megabytes of pointless data and the DEV machine, at the time, was running like a pig...

A look at top showed mysqldump and rdiff-backup and other processes doing stuff. It took ages to get a plain SVN update to happen, so I didn't want to cripple or crash DEV by importing huge dumpfiles.

So I aborted the backup2kiwi script it after I saw it trying to dump many unnecessary tables (cache_*, sessions etc etc) and just used the 'Backup and Migrate' module instead. That made a 13Mb gz file (of a 60mb mysql file) and uploaded onto DEV no problems.

So there remains the following minor issues:

• sites/default/files folder ideally needs rsyncing from LIVE to DEV
• DEV (and by extension LIVE) SVN performance is awful -- at ~1.30am anyway... did this coincide with your own backup processes? Is it the lack of memory on DEV? Does kiwi need a cleanse?
• the backup2kiwi script needs some table filtering to avoid unnecessary load
• only the main live DB got moved, not the workspaces or sharing engine -- though these are not really needed on DEV for now.

But DEV is now in sync with LIVE... downgrading but will leave open for comments.

[chris]

OK, I'll do some on it tonight

[chris]

I have added this to the mysql database backup script:

@skip_tables = qw[cache cache_block cache_content cache_emfield_xml cache_filter cache_form cache_hierarchical_select cache_location cache_media_youtube_status cache_menu cache_mollom cache_page cache_path cache_rules cache_update cache_views cache_views_data sessions];

And I have checked that neither mediawiki or piwik have any tables with the same names, so I think that has fixed that one.

I can't see anything really that has changed which would make svn slower than before...

I think it would be safe to sync live to dev tonight, I'll do this if you want.

[jim]

Just checking the settings at: ​https://www.transitionnetwork.org/admin/content/backup_migrate/profile/list/edit/default

Seems the search_* and watchdog tables should also be excluded...

DEV is ready for us to use for now, but I'm sure it'll need updating again in a few weeks so we can try then.

Thanks.

[chris]

OK I have added:

search_dataset search_index search_node_links search_total watchdog

Closing this ticket now.