wiki:CodeManagementReleaseProcess
Last modified 4 years ago Last modified on 02/11/13 09:31:49

Table of Contents

  1. Principles
  2. Software & Getting Started
  3. Release Process Detail
    1. 0. GET THE LATEST UPDATES!
    2. 1. Creating/Working? with your own branches
    3. 2. Edit you code, test locally
    4. 3. Merge your code and push back to Github
    5. 4. Test on the DEV server
    6. 5. Merge 'development' into 'live', pull onto LIVE server
    7. 6. Rolling back a release

This page documents the process used prior to the migration to BOA, see BoaCodeManagement

Code Management and Release Process

The code for the website and other projects - like the Sharing Engine(s) - including all Drupal core, contrib and custom files (but excluding settings and uploaded assets) is stored in a set of Git repositories on Github at ​https://github.com/i-jk/Transition-Network

The code was previously in a SVN repository, details of which are at CodeManagementReleaseProcessOld.

Principles

This process relies upon all developers understanding:

  1. You never hack Drupal core
  2. You understand Git -- go to ​http://gitref.org/ and start reading if not.
  3. Developers work on their own machines and push code into into the 'development' or 'live branches ONLY when their code TESTED functional.
  4. The ​development server (DEV) mirrors the LIVE environment in setup, and must be used to test this code before moving to LIVE. This server has the 'development' branch on it.
  5. There is a ​TEST server which can also be used as required, but is optional unless there is a lot of development work going on or many developers need to test their work in preparation for a release to LIVE. TEST does not presently have Git enabled - contact Jim if this is needed.
  6. The LIVE server ONLY EVER gets what is in the 'live' branch pulled onto it. ALL releases must be tested (both on a local developer machine AND on DEV) prior to merging with the 'live' branch and being pushed to the LIVE server.
  7. Database backups must happen before each release to LIVE - this way each release on LIVE has an associated database backup. Go to 'Site building' -> 'Backup & migrate' (or use the command line NewLiveServer#mysql-backup script) to create the backup.
  8. You may occasionally want to update the TEST and DEV site's databases and files to match the live server by using the DevelopmentServer#live2dev script.

Software & Getting Started

You obviously must have Git installed on your machine to work on this project. The basic Git command line software available for most platforms in from ​http://git-scm.com/ - however, some very nice applications are also available that integrate into your operating system's graphical interface. Recommended FREE Git applications with a nice GUI are:

To get started, you need a copy of the code and ideally a web server (Apache), database (MySQL) and Drupal set up on your machine. We assume as a web developer you already have this, but Google and ​http://drupal.org/start are certainly your friends if not. Once you're set up, please run this command to get a copy of the current development (DEV) code.

NOTE: You will also need an SSH account set AND to have your SSH keys added to the GitHub? repository before you will have access to the repositories. 

cd /the/directory/you/want/the/project/folder/in
git clone -b [branch] git@github.com:i-jk/Transition-Network.git dev

Where [branch] is the branch you want to work with... Branches available on GitHub? are: development, live. Use development for all normal work as this branch is available on the DEV server. Use live only for making small, incremental updates based on the code in LIVE -- but avoid this approach unless absolutely necessary.

Once you have this, you will be able to edit files on your machine and then later commit them back to the repository - once TESTED of course! Note that the above checkout includes a .gitignore files that means the repository DOES NOT contain the Drupal settings.php files, nor any files uploaded to the server, nor any other unnecessary files your computer might add. It will contain the site's theme though: you will need to get or create these as needed for your setup and configuration.

NOTE LIVE and DEV servers have had git config core.filemode false set to avoid Git setting the file permissions. You will want to run this command in your copy of the repo on your machine, especially if you're running your own webserver.

Release Process Detail

This process gets you from a basic checkout of the code, through committing and testing, to a proper release on the LIVE server. By now you should have a copy of the code checked out as per the section above.

The process is as follows, and assumes you're in the root of your working directory:

0. GET THE LATEST UPDATES!

You MUST start by getting all the changes in the the origin GitHub? repository...

  • If you've not already cloned the branch you want to work with, follow the instructions above in 'Getting Started'
  • If you have and haven't made many changes, you'll need to pull the latest changes with a git pull

1. Creating/Working? with your own branches

If you're working on something and need to keep these changes from the rest of the development, always make your own branch on your local machine. Read the Git documentation for this, but the process is roughly as follows: 

git branch [newbranch] <--- creates a new branch called [newbranch]
git checkout [newbranch] <--- switches your working directory to it

You can then switch branches with no loss of work or confusion by using git checkout [branch], where [branch] is the branch you want. You can see the branches available and active by using git branch. Use as many branches as you need to work locally, but please only send changes to GitHub? when a) they're on live and development branches, or b) they're part of some new project or collaborative effort.

2. Edit you code, test locally

As you modify the source code and theme files on working branch, the differences between your copy and the original branch (live or development) will grow. Clearly, if several developers are working on the same areas at once, you will have merge your changes carefully.

To add and commit your code and merge 

git add -A
git status <--- will show you what's changed, tidy up if needs be!
git commit -m "a nice description of the commit"

The above makes sure your working directory code changes are copied to the staging area and ready to be merged. Make sure you add a good description in case someone needs to roll back a commit to fix an issue. Then, repeat this step until you're ready to...

3. Merge your code and push back to Github

When you're happy your work is tested and complete, you'll want to merge it back into a the 'development' branch for testing on the DEV server: 

git status  <--- will show you what's changed -- double check!
git checkout [target branch]  <--- should be 'development', but could be 'megaproject5' or even 'live' if you like killing kittens
git merge [working branch]  <--- wherever you've been working and committing

If the above goes well, your changes will automatically be merged in. If not, it's time to read up on how to manually ​changes. Once the merge process is complete, you'll want to push your changes to the 'development' branch back to Github: 

git push -u origin [target branch] <--- pushes all updates back to [target branch] (probably 'development')

4. Test on the DEV server

This step is critical because the DEV server mirrors the setup on LIVE. Hence any issues or differences you didn't spot whilst working on your local machine can be ironed out here. It also allows you to get feedback on your work so far as required.

Note: To get files onto the server, you must have logged into the development server via SSH - for more on how to do this see wiki:DevelopmentServer#SSHaccess. ALSO NOTE the DEV and LIVE servers require the use of sudo to make changes. This means that the Root user on both DEV (Kiwi) and LIVE (Quince) have been given access to the Github account. 

ssh dev.transitionnetwork.org.webarch.net
cd /web/dev.transitionnetwork.org.webarch.net/www
sudo git pull  <--- 'origin' is github in this case

The last command gets all the updates to the current branch ('development' is the only one on DEV) and merges them into the current working directory. Go to ​http://dev.transitionnetwork.org.webarch.net/ and enjoy the fruits of your labour. You might need to run 'update.php', too.

Depending on the changes, you should test properly and then solicit feedback and testing from managers/stakeholders. This is probably on DEV, but could be on TEST if needed.

5. Merge 'development' into 'live', pull onto LIVE server

Please note the documentation for the LIVE server is here: wiki:NewLiveServer - and you should have decided whether or not it's necessary to do a run of testing on the TEST site before moving to LIVE... TEST is not presently set up with a Git repository, but the process for both would be the same. Contact Jim if TEST is actually needed.

5a. Merge your branches by... ON YOUR MACHINE: 

git checkout [the branch you want to merge into] <--- probably 'live' at this stage
git pull <--- belt and braces to make sure you've not forgotten to merge something
git merge [your working branch] <--- probably 'development'
git push origin [the branch you want to merge into] <--- probably 'live'

The above will merge your recent work in [your working branch] into [the branch you want to merge into 'live'], then push it back to GitHub?. The above is usually a case of checking out 'live' and merging 'development' before pushing, but if you've made further edits you'll have to merge them in.

5b. Prepare LIVE

  1. SSH into LIVE (or TEST): ssh www.transitionnetwork.org (OR for TEST: ssh test.transitionnetwork.org.webarch.net)
  2. Go to the site root folder cd /web/transitionnetwork.org/www (OR cd /web/test.transitionnetwork.org.webarch.net/www)

5c. Backup & put Drupal into maintenance mode: You are now ready to take the site down and get the new files onto the server, but first you need to make a backup and prepare in Drupal...

  1. Log in as Transition Admin (user 1 - password available on request)
  2. Go to 'Site configuration' -> 'Site maintenance' and put the site offline. Ideally update message so users can see when the site is likely to be back - allow 15 minutes.
  3. Go to 'Site building' -> 'Backup & migrate' (or use the NewLiveServer#mysql-backup script) to make a manual backup of the database - which is saved on the server.

5d. Deploy the new code: Back in the terminal where you are SSHed into the correct machine AND in the correct site root folder... 

sudo git pull

The server will grab the lastest in the 'live' branch and you can then...

4d. Finish the job: Go back to Drupal in your browser and...

  1. Go to the 'Database update' page at ​http://www.transitionnetwork.org/update.php (or test.transitionnetwork.org/update.php) and complete the process.
  2. Go to the homepage, initiatives directory, blogs etc and check the site is looking and behaving like it should. If it isn't you will need to fix the issue. If you can't you will need to roll back - see the next section.
  3. Go to 'Site configuration' -> 'Site maintenance' and put the back main site online.
  4. Let everyone who needs to know the site is back and updated. Then have a beer.

6. Rolling back a release

If it goes badly, don't panic. You have all the old code in Git and a snapshot of the database just prior to updating it. To roll back, run the following commands on the server you're logged into:

  1. (Assuming you can still use Drupal) Take the site back offline and go to 'Site building' -> 'Backup & migrate' -> 'Restore' (if the NewLiveServer#mysql-backup script has been used then the restore will have to be done via the command line).
  2. Restore the database backup just prior to the and make a manual backup of the database - saved on the server.

If you can't use Drupal you will need to use the command line to restore the database backup you made. Essentially this consists of going to the /web/transitionnetwork.org/www/sites/default/files/backup_migrate/manual folder and getting the database snapshot file you made (or /var/backups/mysql/ if the NewLiveServer#mysql-backup script was used, note this script backups up all the databases not just the Drupal one), then running a command like:

mysql live < TransitionNetwork-TransitionAdmin-2010-07-05T11-08.mysql

If you do this as root you don't need to specify the host, user or password as they will be read from /root/.my.cnf).

Once the database is back, WITHOUT doing anything more in Drupal, switch the files back: 

  PROCESS TO BE LOOKED UP AND DOCUMENTED! (sorry!)

That's the files back so clear the caches and test the site.