hours, status, totalhours changed; resolution set

chris — Tue, 08 Jan 2013 14:35:24 GMT

hours changed from 0.0 to 1.4
status changed from new to closed
resolution set to fixed
totalhours changed from 0.0 to 1.4

This script has been created and tested, details follow.

Create a directory for the script and the script:

mkdir /data/disk/tn/scripts
chown tn:users /data/disk/tn/scripts
chmod 700 /data/disk/tn/scripts
touch /data/disk/tn/scripts/pull-files-from-quince.sh
chown tn:users /data/disk/tn/scripts/pull-files-from-quince.sh
chmod 700 /data/disk/tn/scripts/pull-files-from-quince.sh

The directory /data/disk/tn/static currently has one file in it, EMPTY.txt with the content "empty" -- Jim should we delete this file when we create the /data/disk/tn/static/sites/transitionnetwork.org/files directory?

For the moment I have set the script up so that it writes the files into /tmp/ but of course this can be changed by editing the DEST_DIR variable.

Since there is 2.4G of files in /web/transitionnetwork.org/www/sites/default/files on quince I'd suggest moving the contents of /tmp/transitionnetwork.org/files to the place it's wanted to save time / resources compared with rsyncing the data from scratch.

Following is the content of the script:

#!/bin/bash
# See https://trac.transitionnetwork.org/trac/ticket/472
# The server we are copying the files from, see ~/.ssh/config
SOURCE_SERVER="quince"
# Directory on the source server where the files are
# no / at the end please
SOURCE_DIR="/web/transitionnetwork.org/www/sites/default/files"
# The place we want the files copied to
# no / at the end please
DEST_DIR="/tmp/transitionnetwork.org/files"
# The things we want to exclude
EXCLUDE="--exclude=ctools --exclude=js --exclude=css --exclude=tmp"
# The rsync command
RSYNC="rsync -aq --delete"
# Group and user owner to set on transfered files
CHOWN="tn:users"
# Directory permissions to set
CHMOD_DIR="755"
# File permissions to set
CHMOD_FILE="644"
# Check that the $DEST_DIR exists and if it doesn't then create it
if [[ ! -d "$DEST_DIR" ]]; then
  mkdir -p $DEST_DIR
fi
# The rsync command
echo "rsyncing from $SOURCE_SERVER:$SOURCE_DIR/ to $DEST_DIR/ excluding $EXCLUDE"
$RSYNC $EXCLUDE $SOURCE_SERVER:$SOURCE_DIR/ $DEST_DIR/
# The chown command
echo "chowning $DEST_DIR to $CHOWN"
chown -R $CHOWN $DEST_DIR
#  The chmod commands
cd $DEST_DIR
echo "chmodding directories in $DEST_DIR to $CHMOD_DIR"
find -type d -exec chmod $CHMOD_DIR {} \;
echo "chmodding files in $DEST_DIR to $CHMOD_FILE"
find -type f -exec chmod $CHMOD_FILE {} \;

hours, totalhours changed

jim — Sun, 20 Jan 2013 18:35:44 GMT

hours changed from 0.0 to 0.25
totalhours changed from 1.4 to 1.65

Thanks Chris.

FYI a couple of tweaks:

DEST_DIR now is "/data/disk/tn/static/sites/transitionnetwork.org-PROD/files".
RSYNC="rsync -az --delete" (use compression, don't be quiet [I like to see things happen!])

For some reason it still said nothing while it worked, but it did work! Thanks again.

chris — Sun, 20 Jan 2013 18:50:15 GMT

Replying to jim:

RSYNC="rsync -az --delete" (use compression, don't be quiet [I like to see things happen!])
For some reason it still said nothing while it worked, but it did work!

To see the output you need -va (v for verbose), compression isn't going to speed things up as both virtual servers are on the same switch, glad it worked :-)

Transition Technology: Ticket #472: Quince to Puffin rsync script

hours, status, totalhours changed; resolution set

hours, totalhours changed