Transition Technology: Ticket #812: space.transitionnetwork.org hacked?

hours, totalhours changed

chris — Thu, 27 Nov 2014 11:24:26 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.0 to 0.25

Checking the Drupal version:

su - tn -s /bin/bash
cd /data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1
drush core-status
 Drupal version        :  7.31
 Site URI              :  http://default
 Default theme         :  garland
 Administration        :  garland
 theme
 PHP executable        :  /opt/php53/bin/php
 PHP configuration     :  /data/disk/tn/.drush/php.ini
 PHP OS                :  Linux
 Drush version         :  6.5-dev
 Drush configuration   :  /data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/drush/drushrc.php
                          /data/disk/tn/.drush/drushrc.php
 Drush alias files     :  /data/disk/tn/.drush/platform_014.alias.drushrc.php "
                          There are 26 more alias files. Run with --full to see the full list."
 Drupal root           :  /data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1
 Site path             :  sites/default
 File directory path   :  sites/default/files

Manually patching:

wget https://www.drupal.org/files/issues/SA-CORE-2014-005-D7.patch
patch -p1 < SA-CORE-2014-005-D7.patch
  Reversed (or previously applied) patch detected!  Assume -R? [n] n
  Apply anyway? [n] n
  Skipping patch.

So the patch has already been applied, the file is dated Oct 17 00:02.

So I think this is probably a false alarm, but if the site is still in use it should probably be upgraded at some point.

ed — Thu, 27 Nov 2014 11:29:41 GMT

Good news. could be upgraded - the national hubs have all moved off space and into google docs, so it's more likely that we'll can space - but stand by for future updates...

paul — Thu, 27 Nov 2014 17:48:24 GMT

I forgot about this D7 site.

So the patch has already been applied, the file is dated Oct 17 00:02.

So I think this is probably a false alarm, but if the site is still in use it should probably be upgraded at some point.

No, the patch was almost certainly applied by the hacker - so that no other hacker can take control of the site.

Can this site be deleted?

Investigating ..

paul — Thu, 27 Nov 2014 17:50:14 GMT

I forgot about this D7 site.
* So the patch has already been applied, the file is dated Oct 17 00:02. So
I think this is probably a false alarm, but if the site is still in use it
should probably be upgraded at some point.*
No, the patch was almost certainly applied by the hacker - so that no other
hacker can take control of the site.
Are we still using this site?
On Thu, Nov 27, 2014 at 11:29 AM, Transiton Technology Trac <
trac@tech.transitionnetwork.org> wrote:
> #812: space.transitionnetwork.org hacked?
> -------------------------------------+-------------------------------------
>            Reporter:  chris          |                      Owner:  chris
>                Type:  maintenance    |                     Status:  new
>            Priority:  critical       |                  Milestone:
>           Component:  Live server    |  Maintenance
>            Keywords:                 |                 Resolution:
> Add Hours to Ticket:  0              |  Estimated Number of Hours:  0.0
>         Total Hours:  0.25           |                  Billable?:  1
> -------------------------------------+-------------------------------------
>
> Comment (by ed):
>
>  Good news. could be upgraded - the national hubs have all moved off space
>  and into google docs, so it's more likely that we'll can space - but stand
>  by for future updates...
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/812#comment:2>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.
>

paul — Thu, 27 Nov 2014 17:50:15 GMT

Investigating ..
On Thu, Nov 27, 2014 at 5:45 PM, Booker, Paul <i@paulbooker.co.uk> wrote:
> I forgot about this D7 site.
>
>
>
>
> * So the patch has already been applied, the file is dated Oct 17
> 00:02. So I think this is probably a false alarm, but if the site is still
> in use it should probably be upgraded at some point.*
>
> No, the patch was almost certainly applied by the hacker - so that no
> other hacker can take control of the site.
>
> Are we still using this site?
>
> On Thu, Nov 27, 2014 at 11:29 AM, Transiton Technology Trac <
> trac@tech.transitionnetwork.org> wrote:
>
>> #812: space.transitionnetwork.org hacked?
>>
>> -------------------------------------+-------------------------------------
>>            Reporter:  chris          |                      Owner:  chris
>>                Type:  maintenance    |                     Status:  new
>>            Priority:  critical       |                  Milestone:
>>           Component:  Live server    |  Maintenance
>>            Keywords:                 |                 Resolution:
>> Add Hours to Ticket:  0              |  Estimated Number of Hours:  0.0
>>         Total Hours:  0.25           |                  Billable?:  1
>>
>> -------------------------------------+-------------------------------------
>>
>> Comment (by ed):
>>
>>  Good news. could be upgraded - the national hubs have all moved off space
>>  and into google docs, so it's more likely that we'll can space - but
>> stand
>>  by for future updates...
>>
>> --
>> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/812#comment:2
>> >
>> Transition Technology <https://tech.transitionnetwork.org/trac>
>> Support and issues tracking for the Transition Network Web Project.
>>
>
>

paul — Thu, 27 Nov 2014 17:56:54 GMT

@Chris

Looking for the Nginx vhost config files ..

The database credentials are stored in the Apache or Nginx vhost config

./sites/space.transitionnetwork.org/settings.php

paul — Thu, 27 Nov 2014 18:06:10 GMT

That file has just disappeared. Was that you Chris?

paul — Thu, 27 Nov 2014 18:08:20 GMT

Attempting a backup for offline auditing ..

paul — Thu, 27 Nov 2014 18:14:55 GMT

Not working.

@Chris

I'm looking for the mysql username / password for the database spacetransitionn.

paul — Thu, 27 Nov 2014 18:30:40 GMT

I have access to mysql databases. Investigating ..

paul — Thu, 27 Nov 2014 18:33:36 GMT

So far so good :

MariaDB [(none)]> use newstransitionne; Database changed MariaDB [newstransitionne]> select * from menu_router where access_callback = 'file_put_contents'; Empty set (0.00 sec)

MariaDB [newstransitionne]> select * from menu_router where access_callback = 'assert'; Empty set (0.00 sec)

MariaDB [newstransitionne]> select * from role

-> ;

+-----+--------------------+ | rid | name | +-----+--------------------+ | 1 | anonymous user | | 2 | authenticated user | | 3 | se admin | +-----+--------------------+ 3 rows in set (0.00 sec)

MariaDB [newstransitionne]> select * from users_roles where rid=3

-> ;

+-----+-----+ | uid | rid | +-----+-----+ | 1 | 3 | | 3 | 3 | +-----+-----+ 2 rows in set (0.00 sec)

paul — Thu, 27 Nov 2014 18:36:30 GMT

I can still login fine. Ed can you still login ok?

MariaDB [newstransitionne]> select * from users where uid=1; +-----+------------+----------------------------------+-------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+-------------------------------+-------------------------------------------------------------------------------------+---------------+ | uid | name | pass | mail | mode | sort | threshold | theme | signature | signature_format | created | access | login | status | timezone | language | picture | init | data | timezone_name | +-----+------------+----------------------------------+-------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+-------------------------------+-------------------------------------------------------------------------------------+---------------+ | 1 | paulbooker | 459b96db6834ebe7d1ed4fe4bef48036 | se-site@… | 0 | 0 | 0 | | | 0 | 1293637384 | 1407496908 | 1407496005 | 1 | 3600 | | | se-site@… | a:1:{s:13:"form_build_id";s:48:"form-UiEISaCkHEBQyXqIJpw6BbOjaj975ZmbMrs2jw6k-no";} | Europe/London? | +-----+------------+----------------------------------+-------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+-------------------------------+-------------------------------------------------------------------------------------+---------------+ 1 row in set (0.00 sec)

MariaDB [newstransitionne]> select * from users where uid=3; +-----+-------------+----------------------------------+----------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+----------------------------------+--------------------------------------------------------------------------+---------------+ | uid | name | pass | mail | mode | sort | threshold | theme | signature | signature_format | created | access | login | status | timezone | language | picture | init | data | timezone_name | +-----+-------------+----------------------------------+----------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+----------------------------------+--------------------------------------------------------------------------+---------------+ | 3 | Ed Mitchell | 83c81417a775b6f68c6871c2a9bc00aa | edmitchell@… | 0 | 0 | 0 | | | 0 | 1295364633 | 1376991359 | 1376991111 | 1 | 3600 | | | edmitchell@… | a:1:{s:13:"form_build_id";s:37:"form-d20c69bf075ad599bf7f4bdb50ebb1f8";} | Europe/London? | +-----+-------------+----------------------------------+----------------------------------+------+------+-----------+-------+-----------+------------------+------------+------------+------------+--------+----------+----------+---------+----------------------------------+--------------------------------------------------------------------------+---------------+

hours, totalhours changed

paul — Thu, 27 Nov 2014 18:43:05 GMT

hours changed from 0.0 to 0.75
totalhours changed from 0.25 to 1.0

Back later.

@Chris

Would you have a look to see if any new files have been added or have been modified? I'll then go through these later.

chris — Thu, 27 Nov 2014 19:10:38 GMT

I could look at doing some recursive diffs to compare backup directories tomorrow.

Paul you can mount the backups via SFTP if you need to, your ssh key was added to the backup account for the server a while ago, see wiki:PuffinServer#Backups.

paul — Thu, 27 Nov 2014 22:19:50 GMT

Thanks Chris. I'll give that a go ..

hours, totalhours changed

paul — Thu, 27 Nov 2014 22:43:38 GMT

hours changed from 0.0 to 0.25
totalhours changed from 1.0 to 1.25

@Chris

I don't know how to get everything downloaded (quickly) so I'm going to wait until the morning.

FYI

Remote working directory: /tank/backupclients/tn-puffin/puffin.webarch.net/data/disk/tn/distro/008

ed — Fri, 28 Nov 2014 08:46:01 GMT

I can login. I am also asking Filipa and Ben (national hub co-ordinators who were using space) if we can remove it.

IF we can remove it but they want some backups, what can I tell them? Can we access important items later in some way (offline/whatever)?

chris — Fri, 28 Nov 2014 10:02:03 GMT

Replying to paul:

I don't know how to get everything downloaded (quickly) so I'm going to wait until the morning.

You shouldn't need to download anything, you can use FUSE to mount the backups via SFTP and then do recursive diffs locally, see wiki:PuffinServer#Backups.

paul — Fri, 28 Nov 2014 10:40:40 GMT

Ok, giving this a go ..

paul — Fri, 28 Nov 2014 10:49:34 GMT

$ sudo echo "sshfs#tn-puffin@…:puffin.webarch.net /media/tn-puffin/latest fuse ro,nobootwait 0 0" >> /etc/fstab -bash: /etc/fstab: Permission denied

@Chris

Any thoughts on this error and whether I should be running the commands as root? (I get the same error as a normal user)

hours, totalhours changed

chris — Fri, 28 Nov 2014 10:59:02 GMT

hours changed from 0.0 to 0.7
totalhours changed from 1.25 to 1.95

To save you having to mount the directories I have done a diff on the server, comparing the oldest backup with the newest backup:

cd /tank/backupclients/tn-puffin/.zfs/snapshot
diff -rq auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008 auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008 > /tmp/diff.txt
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/drupal-7.31.1-prod/sites/all/drush/drushrc.php and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/drupal-7.31.1-prod/sites/all/drush/drushrc.php differ
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1: SA-CORE-2014-005-D7.patch
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/drush/drushrc.php and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/drush/drushrc.php differ
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/boa_platform_control.ini and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/boa_platform_control.ini differ
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/default.boa_platform_control.ini and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/default.boa_platform_control.ini differ
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all: permissions-fix-141001-0258.info
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all: permissions-fix-141126-0258.info
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__37dcSK13DUMToVq7_C_zjzrJZehU_30SPuCUcwsq3Vs__9lVXmlNDyWbA-F7gweyD6pq_dkHy54WLLWE6rGMiaak__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__37dcSK13DUMToVq7_C_zjzrJZehU_30SPuCUcwsq3Vs__9lVXmlNDyWbA-F7gweyD6pq_dkHy54WLLWE6rGMiaak__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css.gz
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__DnsfIfnMw9CQlL0B51bIIR1InlsR8mlxYzFJ_lv8fvM__el7OJeHin9FwxhELRIPdBdM0EJzz3xabCVlfYkUiw54__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__DnsfIfnMw9CQlL0B51bIIR1InlsR8mlxYzFJ_lv8fvM__el7OJeHin9FwxhELRIPdBdM0EJzz3xabCVlfYkUiw54__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__DnsfIfnMw9CQlL0B51bIIR1InlsR8mlxYzFJ_lv8fvM__el7OJeHin9FwxhELRIPdBdM0EJzz3xabCVlfYkUiw54__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__DnsfIfnMw9CQlL0B51bIIR1InlsR8mlxYzFJ_lv8fvM__el7OJeHin9FwxhELRIPdBdM0EJzz3xabCVlfYkUiw54__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css.gz
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__FVLOVP1z5I7UXArbCK2jqQENbMaWlChLL2uIAIGG-js__BTkqsmiTb440FCOvdiG6OzeNXz0uR-c-E61jvmUexrk__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__FVLOVP1z5I7UXArbCK2jqQENbMaWlChLL2uIAIGG-js__BTkqsmiTb440FCOvdiG6OzeNXz0uR-c-E61jvmUexrk__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__Im6tnC41VzfIivWpCiqNs4oz77H_gMyMZCLUo--UKos__cogj-_ncBLkAw2w9bUtCHkO4esEZtLwkennq2sYCBx4__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__Im6tnC41VzfIivWpCiqNs4oz77H_gMyMZCLUo--UKos__cogj-_ncBLkAw2w9bUtCHkO4esEZtLwkennq2sYCBx4__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__P8l4gjISAAyhjDLI0jmclbe-LmzIWzTzIX585JtrM-s__dWEJe9609vlq-OQWT6zLaA5l-3rylYBH2Zd_9WT26Rk__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__P8l4gjISAAyhjDLI0jmclbe-LmzIWzTzIX585JtrM-s__dWEJe9609vlq-OQWT6zLaA5l-3rylYBH2Zd_9WT26Rk__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__frXgqMMlXNY5ut1ostmTnUNRLKlE-t4Ve5HrkkMEE8Q__-6IoyaUxE1z-0puvXF9HOodp6KnrGqGCZubiPbreLbw__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__frXgqMMlXNY5ut1ostmTnUNRLKlE-t4Ve5HrkkMEE8Q__-6IoyaUxE1z-0puvXF9HOodp6KnrGqGCZubiPbreLbw__fuaSwME_XtwsiwlzRQ_u2UYxU1ng47lk67Yb5VQaHy4.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__lG0AaaPLEChLbQjnuVcas1Y32FOrto2AzDvkKBDAu_A__qu7oNSrNSuDsNa-uhdnGAlzHKftvHuo64Z7AlHYKLYE__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/advagg_css: css__lG0AaaPLEChLbQjnuVcas1Y32FOrto2AzDvkKBDAu_A__qu7oNSrNSuDsNa-uhdnGAlzHKftvHuo64Z7AlHYKLYE__rGEEOwlVMXHXDy9x0x0d_byGcDo3vEb_dtgoUTR4ivM.css.gz
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/colorizer: oa_radix-ac4264df.css
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/files/colorizer: oa_radix_0-75336d87.css
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/boa_site_control.ini and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/boa_site_control.ini differ
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules: commerce_ubercart_check.info
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/default.boa_site_control.ini and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/default.boa_site_control.ini differ
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/settings.php and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/settings.php differ
Files auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php and auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php differ

Running diffs on the files identified above:

diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/drupal-7.31.1-prod/sites/all/drush/drushrc.php auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/drupal-7.31.1-prod/sites/all/drush/drushrc.php | vim -
  1 1970c1970
  2 <           'version' => '7.x-1.12',
  3 ---
  4 >           'version' => '7.x-1.14',
  5 1972c1972
  6 <           'datestamp' => '1407423547',
  7 ---
  8 >           'datestamp' => '1415893406',
  9 1980c1980
 10 <         'version' => '7.x-1.12',
 11 ---
 12 >         'version' => '7.x-1.14',
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/drush/drushrc.php auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/drush/drushrc.php | vim -
  1 1969c1969
  2 <           'version' => '7.x-1.12',
  3 ---
  4 >           'version' => '7.x-1.14',
  5 1971c1971
  6 <           'datestamp' => '1407423547',
  7 ---
  8 >           'datestamp' => '1415893406',
  9 1979c1979
 10 <         'version' => '7.x-1.12',
 11 ---
 12 >         'version' => '7.x-1.14',
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/boa_platform_control.ini  auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all/modules/boa_platform_control.ini | vim -
  1 40c40
  2 < ;;  may not include all options available after upgrade to BOA-2.3.3
  3 ---
  4 > ;;  may not include all options available after upgrade to BOA-2.3.7
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/boa_site_control.ini auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/boa_site_control.ini | vim -
  1 39c39
  2 < ;;  may not include all options available after upgrade to BOA-2.3.3
  3 ---
  4 > ;;  may not include all options available after upgrade to BOA-2.3.7
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/default.boa_site_control.ini auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules/default.boa_site_control.ini | vim -
  1 14c14
  2 < ;;  BOA-2.3.3
  3 ---
  4 > ;;  BOA-2.3.6
  5 69,70c69,71
  6 < ;;  The more aggressive cache flush mode is not enabled by default, but you can
  7 < ;;  still enable it with TRUE below, if you experience issues with stale caches.
  8 ---
  9 > ;;  The more aggressive cache flush mode is now enabled by default, but you can
 10 > ;;  still disable it with FALSE below, if you wish, after some testing, since
 11 > ;;  it will further improve your site's performance.
 12 92c93
 13 < ;redis_flush_forced_mode = FALSE
 14 ---
 15 > ;redis_flush_forced_mode = TRUE
 16 325a327,330
 17 > ;;  IMPORTANT if you are using self-hosted BOA: _MODULES_FIX=YES must be set
 18 > ;;  in the /root/.barracuda.cnf file (this is default value) to make this
 19 > ;;  feature active.
 20 > ;;
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php | vim -
  1 1968c1968
  2 <           'version' => '7.x-1.12',
  3 ---
  4 >           'version' => '7.x-1.14',
  5 1970c1970
  6 <           'datestamp' => '1407423547',
  7 ---
  8 >           'datestamp' => '1415893406',
  9 1978c1978
 10 <         'version' => '7.x-1.12',
 11 ---
 12 >         'version' => '7.x-1.14',
diff auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php  auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.21-7.31.1/sites/all/drush/drushrc.php | vim -
  1 1968c1968
  2 <           'version' => '7.x-1.12',
  3 ---
  4 >           'version' => '7.x-1.14',
  5 1970c1970
  6 <           'datestamp' => '1407423547',
  7 ---
  8 >           'datestamp' => '1415893406',
  9 1978c1978
 10 <         'version' => '7.x-1.12',
 11 ---
 12 >         'version' => '7.x-1.14',

If we ignore the *.css files, which I assume it is safe to do, the only other differences are the following:

Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1: SA-CORE-2014-005-D7.patch
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all: permissions-fix-141001-0258.info
Only in auto-UTC-2014-11-28_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/all: permissions-fix-141126-0258.info
Only in auto-UTC-2014-10-01_06.00/puffin.webarch.net/data/disk/tn/distro/008/openatrium-7.x-2.19-7.31.1/sites/space.transitionnetwork.org/modules: commerce_ubercart_check.info

The SA-CORE-2014-005-D7.patch was downloaded my me.

The permissions-fix-*.info files simply contain the word fixed.

The commerce_ubercart_check.info file simply contains the word OK.

I can't find any evidence that content has been uploaded which shouldn't have. However there are more things that could be checked, for example I haven't checked the server logs.

chris — Fri, 28 Nov 2014 11:01:05 GMT

Replying to paul:

$ sudo echo "sshfs#tn-puffin@…:puffin.webarch.net /media/tn-puffin/latest fuse ro,nobootwait 0 0" >> /etc/fstab -bash: /etc/fstab: Permission denied

@Chris

Any thoughts on this error and whether I should be running the commands as root? (I get the same error as a normal user)

I don't know why you don't have permission to edit your local /etc/fstab file, try doing ls -lah /etc/fstab to see what the permissions and ownership for it are.

paul — Fri, 28 Nov 2014 11:07:29 GMT

The error was actually because the directory doesn't exist. Investigating what needs to be done on my mac ..

$ ls -lah /etc/fstab

ls: /etc/fstab: No such file or directory

hours, totalhours changed

paul — Fri, 28 Nov 2014 11:24:45 GMT

hours changed from 0.0 to 0.5
totalhours changed from 1.95 to 2.45

Chris,

Brilliant. That looks good. Great to hear that it was you that applied the patch.

Ok, switching channel ..

chris — Fri, 28 Nov 2014 11:27:53 GMT

Replying to paul:

Great to hear that it was you that applied the patch.

It failed actually as it was already applied (by BOA I guess?), see ticket:812#comment:1

chris — Fri, 28 Nov 2014 11:30:42 GMT

Replying to paul:

The error was actually because the directory doesn't exist. Investigating what needs to be done on my mac ..

Sorry I didn't realise that you are not running Linux locally, try the suggestions here?

How to mount SFTP/SSH shares in OS X

hours, totalhours changed

paul — Fri, 28 Nov 2014 11:34:28 GMT

hours changed from 0.0 to 0.125
totalhours changed from 2.45 to 2.575

Replying to chris:

Replying to paul:

Great to hear that it was you that applied the patch.

It failed actually as it was already applied (by BOA I guess?), see ticket:812#comment:1

By you, I meant the aegir system that you're managing.

I already knew the patch failed - I referred to this earlier - when I said the patch was probably already applied by the hacker.

Switching channel ..

chris — Fri, 28 Nov 2014 11:46:39 GMT

Replying to ed:

I can login. I am also asking Filipa and Ben (national hub co-ordinators who were using space) if we can remove it.

IF we can remove it but they want some backups, what can I tell them? Can we access important items later in some way (offline/whatever)?

I'd suggest if they have copied off all the data they need and they don't need the site any more then we can remove it.

If they haven't copied off all the data they need then I'd suggest we still need to host and maintain the site until a time when they don't need it -- because the data isn't public we can't generate a static copy of the site.

ed — Fri, 28 Nov 2014 13:27:24 GMT

Replying to chris:

Replying to ed:

I can login. I am also asking Filipa and Ben (national hub co-ordinators who were using space) if we can remove it.

IF we can remove it but they want some backups, what can I tell them? Can we access important items later in some way (offline/whatever)?

I'd suggest if they have copied off all the data they need and they don't need the site any more then we can remove it.

If they haven't copied off all the data they need then I'd suggest we still need to host and maintain the site until a time when they don't need it -- because the data isn't public we can't generate a static copy of the site.

Awaiting reply from them

hours, totalhours changed

chris — Fri, 28 Nov 2014 18:28:46 GMT

hours changed from 0.0 to 1.0
totalhours changed from 2.575 to 3.575

Replying to paul:

the patch was almost certainly applied by the hacker - so that no other hacker can take control of the site.

I'm not 100% sure this isn't the case, I have done a recursive diff on /etc/ and then tried with the whole filesystem but it's too much data.

Following these suggestions:

cd /var/lib/dpkg/info
cat *.md5sums | sort -k 2 > /run/shm/all.md5
cd /
md5sum -c /run/shm/all.md5 > /run/shm/md5check.txt 2>&1
grep -v ': OK$' /run/shm/md5check.txt

Results in this list:

bin/bzdiff: FAILED
bin/bzexe: FAILED
bin/bzgrep: FAILED
bin/bzmore: FAILED
bin/lesspipe: FAILED
bin/which: FAILED
etc/cron.daily/bsdmainutils: FAILED
etc/init.d/README: FAILED
etc/init.d/rc: FAILED
etc/init.d/rcS: FAILED
etc/init.d/skeleton: FAILED
sbin/fsck.nfs: FAILED
sbin/installkernel: FAILED
sbin/on_ac_power: FAILED
sbin/resolvconf: FAILED
sbin/shadowconfig: FAILED
usr/bin/7z: FAILED
usr/bin/7za: FAILED
usr/bin/Magick-config: FAILED
usr/bin/MagickCore-config: FAILED
usr/bin/MagickWand-config: FAILED
usr/bin/Wand-config: FAILED
usr/bin/anytopnm: FAILED
usr/bin/autoconf: FAILED
usr/bin/autoconf2.13: FAILED
usr/bin/autoheader: FAILED
usr/bin/autoheader2.13: FAILED
usr/bin/autopoint: FAILED
usr/bin/autoreconf: FAILED
usr/bin/autoreconf2.13: FAILED
usr/bin/autoupdate2.13: FAILED
usr/bin/bison.yacc: FAILED
usr/bin/c89-gcc: FAILED
usr/bin/c99-gcc: FAILED
usr/bin/checkbashisms: FAILED
usr/bin/compile_et: FAILED
usr/bin/crypt: FAILED
usr/bin/dcmd: FAILED
usr/bin/debconf-updatepo: FAILED
usr/bin/debsign: FAILED
usr/bin/dehtmldiff: FAILED
usr/bin/dscextract: FAILED
usr/bin/dumphint: FAILED
usr/bin/dvipdf: FAILED
usr/bin/edit-patch: FAILED
usr/bin/eps2eps: FAILED
usr/bin/fakeroot-sysv: FAILED
usr/bin/fakeroot-tcp: FAILED
usr/bin/font2c: FAILED
usr/bin/freetype-config: FAILED
usr/bin/gcore: FAILED
usr/bin/gdbtui: FAILED
usr/bin/getbuildlog: FAILED
usr/bin/gettext.sh: FAILED
usr/bin/gettextize: FAILED
usr/bin/glib-gettextize: FAILED
usr/bin/gpg-error-config: FAILED
usr/bin/gsbj: FAILED
usr/bin/gsdj: FAILED
usr/bin/gsdj500: FAILED
usr/bin/gslj: FAILED
usr/bin/gslp: FAILED
usr/bin/gsnd: FAILED
usr/bin/ifnames2.13: FAILED
usr/bin/igawk: FAILED
usr/bin/install-info: FAILED
usr/bin/lft.db: FAILED
usr/bin/lftpget: FAILED
usr/bin/libmcrypt-config: FAILED
usr/bin/libpng12-config: FAILED
usr/bin/libtool: FAILED
usr/bin/libtoolize: FAILED
usr/bin/libwmf-config: FAILED
usr/bin/lorder: FAILED
usr/bin/lsinitramfs: FAILED
usr/bin/mkfontdir: FAILED
usr/bin/ncurses5-config: FAILED
usr/bin/ncursesw5-config: FAILED
usr/bin/neqn: FAILED
usr/bin/nroff: FAILED
usr/bin/pamstretch-gen: FAILED
usr/bin/pcre-config: FAILED
usr/bin/pdf2dsc: FAILED
usr/bin/pdf2ps: FAILED
usr/bin/pdfopt: FAILED
usr/bin/pf2afm: FAILED
usr/bin/pfbtopfa: FAILED
usr/bin/pnmmargin: FAILED
usr/bin/po2debconf: FAILED
usr/bin/pphs: FAILED
usr/bin/ppmtomap: FAILED
usr/bin/printafm: FAILED
usr/bin/ps2ascii: FAILED
usr/bin/ps2epsi: FAILED
usr/bin/ps2pdf: FAILED
usr/bin/ps2pdf12: FAILED
usr/bin/ps2pdf13: FAILED
usr/bin/ps2pdf14: FAILED
usr/bin/ps2pdfwr: FAILED
usr/bin/ps2ps: FAILED
usr/bin/ps2ps2: FAILED
usr/bin/rgrep: FAILED
usr/bin/routef: FAILED
usr/bin/routel: FAILED
usr/bin/savelog: FAILED
usr/bin/sensible-browser: FAILED
usr/bin/sensible-editor: FAILED
usr/bin/sensible-pager: FAILED
usr/bin/shtool: FAILED
usr/bin/shtoolize: FAILED
usr/bin/ssl-cert-check: FAILED
usr/bin/traceproto.db: FAILED
usr/bin/traceroute-nanog: FAILED
usr/bin/update-mime-database: FAILED
usr/bin/updatedb.findutils: FAILED
usr/bin/valgrind: FAILED
usr/bin/vimtutor: FAILED
usr/bin/wftopfa: FAILED
usr/bin/xlsview: FAILED
usr/bin/xpdf: FAILED
usr/bin/xslt-config: FAILED
usr/bin/zipgrep: FAILED
usr/bin/zxpdf: FAILED
usr/include/mysql/my_config.h: FAILED
usr/lib/ConsoleKit/run-session.d/pam-foreground-compat.ck: FAILED
usr/lib/esmtp/sasl-cram-md5.so: FAILED
usr/lib/esmtp/sasl-login.so: FAILED
usr/lib/esmtp/sasl-plain.so: FAILED
usr/lib/git-core/git-am: FAILED
usr/lib/git-core/git-bisect: FAILED
usr/lib/git-core/git-difftool--helper: FAILED
usr/lib/git-core/git-filter-branch: FAILED
usr/lib/git-core/git-instaweb: FAILED
usr/lib/git-core/git-lost-found: FAILED
usr/lib/git-core/git-merge-octopus: FAILED
usr/lib/git-core/git-merge-one-file: FAILED
usr/lib/git-core/git-merge-resolve: FAILED
usr/lib/git-core/git-mergetool: FAILED
usr/lib/git-core/git-pull: FAILED
usr/lib/git-core/git-quiltimport: FAILED
usr/lib/git-core/git-rebase: FAILED
usr/lib/git-core/git-rebase--interactive: FAILED
usr/lib/git-core/git-remote-testgit: FAILED
usr/lib/git-core/git-repack: FAILED
usr/lib/git-core/git-request-pull: FAILED
usr/lib/git-core/git-stash: FAILED
usr/lib/git-core/git-submodule: FAILED
usr/lib/git-core/git-web--browse: FAILED
usr/lib/gnupg/gpgkeys_curl: FAILED
usr/lib/gnupg/gpgkeys_hkp: FAILED
usr/lib/postfix/post-install: FAILED
usr/lib/postfix/postfix-script: FAILED
usr/lib/postfix/postfix-wrapper: FAILED
usr/lib/postfix/postmulti-script: FAILED
usr/sbin/add-shell: FAILED
usr/sbin/apticron: FAILED
usr/sbin/invoke-rc.d: FAILED
usr/sbin/mkinitramfs: FAILED
usr/sbin/ntpdate-debian: FAILED
usr/sbin/paperconfig: FAILED
usr/sbin/remove-shell: FAILED
usr/sbin/service: FAILED
usr/sbin/sync-available: FAILED
usr/sbin/t1libconfig: FAILED
usr/sbin/tcptraceroute.db: FAILED
usr/sbin/update-fonts-alias: FAILED
usr/sbin/update-fonts-dir: FAILED
usr/sbin/update-fonts-scale: FAILED
usr/sbin/update-gsfontmap: FAILED
usr/sbin/update-icon-caches: FAILED
usr/sbin/update-icon-caches: FAILED
usr/sbin/update-initramfs: FAILED
usr/share/GeoIP/GeoIP.dat: FAILED
usr/share/GeoIP/GeoIPv6.dat: FAILED
usr/share/man/man1/autoconf.1.gz: FAILED
usr/share/man/man1/autoheader.1.gz: FAILED
usr/share/man/man1/autoreconf.1.gz: FAILED
usr/share/munin/plugins/apt_all: FAILED
usr/share/vim/vim73/doc/help.txt: FAILED
usr/share/vim/vim73/doc/tags: FAILED
md5sum: WARNING: 180 computed checksums did NOT match

So some additional checking could be done on the above files.

I have also reviewed the last two months of lfd System Integrity check which detects modified system files and these could be checked against the updates, there are 20 or so of these, for example:

From: root@puffin.webarch.net
Date: Sun, 16 Nov 2014 14:43:30 +0000 (GMT)
To: chris@webarchitects.co.uk
Subject: lfd on puffin.webarch.net: System Integrity checking detected a modified system file
Time:     Sun Nov 16 14:43:30 2014 +0100
The following list of files have FAILED the md5sum comparison test. This means that the file has been changed in some way. This could be a result of an OS update or application upgrade. If the change is unexpected it should be investigated:
/usr/bin/dumpsexp: FAILED
/usr/bin/hmac256: FAILED
/usr/bin/libgcrypt-config: FAILED

I also installed rkhunter and chkrootkit and these are the outputs:

ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `crontab'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not found
Checking `grep'... not infected
Checking `hdparm'... not infected
Checking `su'... not infected
Checking `ifconfig'... not infected
Checking `inetd'... not infected
Checking `inetdconf'... not found
Checking `identd'... not found
Checking `init'... not infected
Checking `killall'... not infected
Checking `ldsopreload'... not infected
Checking `login'... not infected
Checking `ls'... not infected
Checking `lsof'... not infected
Checking `mail'... not infected
Checking `mingetty'... not found
Checking `netstat'... not infected
Checking `named'... not found
Checking `passwd'... not infected
Checking `pidof'... not infected
Checking `pop2'... not found
Checking `pop3'... not found
Checking `ps'... not infected
Checking `pstree'... not infected
Checking `rpcinfo'... not infected
Checking `rlogind'... not found
Checking `rshd'... not found
Checking `slogin'... not infected
Checking `sendmail'... not infected
Checking `sshd'... not infected
Checking `syslogd'... not tested
Checking `tar'... not infected
Checking `tcpd'... not infected
Checking `tcpdump'... not infected
Checking `top'... not infected
Checking `telnetd'... not found
Checking `timed'... not found
Checking `traceroute'... not infected
Checking `vdir'... not infected
Checking `w'... not infected
Checking `write'... not infected
Checking `aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for rootkit HiDrootkit's default files... nothing found
Searching for rootkit t0rn's default files... nothing found
Searching for t0rn's v8 defaults... nothing found
Searching for rootkit Lion's default files... nothing found
Searching for rootkit RSHA's default files... nothing found
Searching for rootkit RH-Sharpe's default files... nothing found
Searching for Ambient's rootkit (ark) default files and dirs... nothing found
Searching for suspicious files and dirs, it may take a while... The following suspicious files and directories were found:
/usr/lib/debug/.build-id /usr/lib/pymodules/python2.6/.path /usr/lib/pymodules/python2.7/.path /usr/lib/jvm/.java-1.7.0-openjdk-amd64.jinfo /usr/lib/jvm/.java-1.6.0-openjdk-amd64.jinfo /usr/lib/xulrunner-1.9.1/.autoreg
/usr/lib/debug/.build-id
Searching for LPD Worm files and dirs... nothing found
Searching for Ramen Worm files and dirs... nothing found
Searching for Maniac files and dirs... nothing found
Searching for RK17 files and dirs... nothing found
Searching for Ducoci rootkit... nothing found
Searching for Adore Worm... nothing found
Searching for ShitC Worm... nothing found
Searching for Omega Worm... nothing found
Searching for Sadmind/IIS Worm... nothing found
Searching for MonKit... nothing found
Searching for Showtee... nothing found
Searching for OpticKit... nothing found
Searching for T.R.K... nothing found
Searching for Mithra... nothing found
Searching for LOC rootkit... nothing found
Searching for Romanian rootkit... nothing found
Searching for Suckit rootkit... nothing found
Searching for Volc rootkit... nothing found
Searching for Gold2 rootkit... nothing found
Searching for TC2 Worm default files and dirs... nothing found
Searching for Anonoying rootkit default files and dirs... nothing found
Searching for ZK rootkit default files and dirs... nothing found
Searching for ShKit rootkit default files and dirs... nothing found
Searching for AjaKit rootkit default files and dirs... nothing found
Searching for zaRwT rootkit default files and dirs... nothing found
Searching for Madalin rootkit default files... nothing found
Searching for Fu rootkit default files... nothing found
Searching for ESRK rootkit default files... nothing found
Searching for rootedoor... nothing found
Searching for ENYELKM rootkit default files... nothing found
Searching for common ssh-scanners default files... nothing found
Searching for suspect PHP files... nothing found
Searching for anomalies in shell history files... nothing found
Checking `asp'... not infected
Checking `bindshell'... not infected
Checking `lkm'...
You have 1 process hidden for readdir command
You have 1 process hidden for ps command
chkproc: Warning: Possible LKM Trojan installed
chkdirs: nothing detected
Checking `rexedcs'... not found
Checking `sniffer'...
lo: not promisc and no packet sniffer sockets
eth0: not promisc and no packet sniffer sockets
Checking `w55808'... not infected
Checking `wted'... chkwtmp: nothing deleted
Checking `scalper'... not infected
Checking `slapper'... not infected
Checking `z2'... chklastlog: nothing deleted
Checking `chkutmp'...
The tty of the following user process(es) were not found in /var/run/utmp !
! RUID PID TTY CMD
! root 3453 pts/1 /bin/bash
chkutmp: nothing deleted
Checking `OSX_RSPLUG'... not infected

And:

 rkhunter -c
[ Rootkit Hunter version 1.4.0 ]
Checking system commands...
  Performing 'strings' command checks
  Performing 'strings' command checks
    Checking 'strings' command                               [ OK ]
  Performing 'shared libraries' checks
    Checking for preloading variables                        [ None found ]
    Checking for preloaded libraries                         [ None found ]
    Checking LD_LIBRARY_PATH variable                        [ Not found ]
  Performing file properties checks
    Checking for prerequisites                               [ OK ]
    /usr/sbin/adduser                                        [ OK ]
    /usr/sbin/chroot                                         [ OK ]
    /usr/sbin/cron                                           [ OK ]
    /usr/sbin/groupadd                                       [ OK ]
    /usr/sbin/groupdel                                       [ OK ]
    /usr/sbin/groupmod                                       [ OK ]
    /usr/sbin/grpck                                          [ OK ]
    /usr/sbin/groupadd                                       [ OK ]
    /usr/sbin/groupdel                                       [ OK ]
    /usr/sbin/groupmod                                       [ OK ]
    /usr/sbin/grpck                                          [ OK ]
    /usr/sbin/nologin                                        [ OK ]
    /usr/sbin/pwck                                           [ OK ]
    /usr/sbin/rsyslogd                                       [ OK ]
    /usr/sbin/tcpd                                           [ OK ]
    /usr/sbin/useradd                                        [ OK ]
    /usr/sbin/userdel                                        [ OK ]
    /usr/sbin/usermod                                        [ OK ]
    /usr/sbin/vipw                                           [ OK ]
    /usr/bin/awk                                             [ OK ]
    /usr/bin/basename                                        [ OK ]
    /usr/bin/chattr                                          [ OK ]
    /usr/bin/curl                                            [ OK ]
    /usr/bin/cut                                             [ OK ]
    /usr/bin/diff                                            [ OK ]
    /usr/bin/dirname                                         [ OK ]
    /usr/bin/dpkg                                            [ OK ]
    /usr/bin/dpkg-query                                      [ OK ]
    /usr/bin/du                                              [ OK ]
    /usr/bin/env                                             [ OK ]
    /usr/bin/file                                            [ OK ]
    /usr/bin/find                                            [ OK ]
    /usr/bin/GET                                             [ OK ]
    /usr/bin/groups                                          [ OK ]
    /usr/bin/head                                            [ OK ]
    /usr/bin/id                                              [ OK ]
    /usr/bin/killall                                         [ OK ]
    /usr/bin/last                                            [ OK ]
    /usr/bin/lastlog                                         [ OK ]
    /usr/bin/ldd                                             [ OK ]
    /usr/bin/less                                            [ OK ]
    /usr/bin/locate                                          [ OK ]
    /usr/bin/logger                                          [ OK ]
    /usr/bin/lsattr                                          [ OK ]
    /usr/bin/lsof                                            [ OK ]
    /usr/bin/dirname                                         [ OK ]
    /usr/bin/dpkg                                            [ OK ]
    /usr/bin/dpkg-query                                      [ OK ]
    /usr/bin/du                                              [ OK ]
    /usr/bin/env                                             [ OK ]
    /usr/bin/file                                            [ OK ]
    /usr/bin/find                                            [ OK ]
    /usr/bin/GET                                             [ OK ]
    /usr/bin/groups                                          [ OK ]
    /usr/bin/head                                            [ OK ]
    /usr/bin/id                                              [ OK ]
    /usr/bin/killall                                         [ OK ]
    /usr/bin/last                                            [ OK ]
    /usr/bin/lastlog                                         [ OK ]
    /usr/bin/ldd                                             [ OK ]
    /usr/bin/less                                            [ OK ]
    /usr/bin/locate                                          [ OK ]
    /usr/bin/logger                                          [ OK ]
    /usr/bin/lsattr                                          [ OK ]
    /usr/bin/lsof                                            [ OK ]
    /usr/bin/lynx                                            [ OK ]
    /usr/bin/mail                                            [ OK ]
    /usr/bin/md5sum                                          [ OK ]
    /usr/bin/newgrp                                          [ OK ]
    /usr/bin/passwd                                          [ OK ]
    /usr/bin/perl                                            [ OK ]
    /usr/bin/pgrep                                           [ OK ]
    /usr/bin/pkill                                           [ OK ]
    /usr/bin/pstree                                          [ OK ]
    /usr/bin/rkhunter                                        [ OK ]
    /usr/bin/runcon                                          [ OK ]
    /usr/bin/sha1sum                                         [ OK ]
    /usr/bin/sha224sum                                       [ OK ]
    /usr/bin/sha256sum                                       [ OK ]
    /usr/bin/sha384sum                                       [ OK ]
    /usr/bin/sha512sum                                       [ OK ]
    /usr/bin/size                                            [ OK ]
    /usr/bin/sort                                            [ OK ]
    /usr/bin/stat                                            [ OK ]
    /usr/bin/strace                                          [ OK ]
    /usr/bin/strings                                         [ OK ]
    /usr/bin/sudo                                            [ OK ]
    /usr/bin/tail                                            [ OK ]
    /usr/bin/test                                            [ OK ]
    /usr/bin/top                                             [ OK ]
    /usr/bin/touch                                           [ OK ]
    /usr/bin/tr                                              [ OK ]
    /usr/bin/uniq                                            [ OK ]
    /usr/bin/users                                           [ OK ]
    /usr/bin/vmstat                                          [ OK ]
    /usr/bin/w                                               [ OK ]
    /usr/bin/watch                                           [ OK ]
    /usr/bin/wc                                              [ OK ]
    /usr/bin/wget                                            [ OK ]
    /usr/bin/sha256sum                                       [ OK ]
    /usr/bin/sha384sum                                       [ OK ]
    /usr/bin/sha512sum                                       [ OK ]
    /usr/bin/size                                            [ OK ]
    /usr/bin/sort                                            [ OK ]
    /usr/bin/stat                                            [ OK ]
    /usr/bin/strace                                          [ OK ]
    /usr/bin/strings                                         [ OK ]
    /usr/bin/sudo                                            [ OK ]
    /usr/bin/tail                                            [ OK ]
    /usr/bin/test                                            [ OK ]
    /usr/bin/top                                             [ OK ]
    /usr/bin/touch                                           [ OK ]
    /usr/bin/tr                                              [ OK ]
    /usr/bin/uniq                                            [ OK ]
    /usr/bin/users                                           [ OK ]
    /usr/bin/vmstat                                          [ OK ]
    /usr/bin/w                                               [ OK ]
    /usr/bin/watch                                           [ OK ]
    /usr/bin/wc                                              [ OK ]
    /usr/bin/wget                                            [ OK ]
    /usr/bin/whatis                                          [ OK ]
    /usr/bin/whereis                                         [ OK ]
    /usr/bin/which                                           [ OK ]
    /usr/bin/who                                             [ OK ]
    /usr/bin/whoami                                          [ OK ]
    /usr/bin/unhide.rb                                       [ Warning ]
    /usr/bin/gawk                                            [ OK ]
    /usr/bin/lwp-request                                     [ OK ]
    /usr/bin/locate.findutils                                [ OK ]
    /usr/bin/heirloom-mailx                                  [ OK ]
    /usr/bin/w.procps                                        [ OK ]
    /sbin/depmod                                             [ OK ]
    /sbin/fsck                                               [ OK ]
    /sbin/ifconfig                                           [ OK ]
    /sbin/ifdown                                             [ OK ]
    /sbin/ifup                                               [ OK ]
    /sbin/init                                               [ OK ]
    /sbin/insmod                                             [ OK ]
    /sbin/ip                                                 [ OK ]
    /sbin/ip                                                 [ OK ]
    /sbin/lsmod                                              [ OK ]
    /sbin/modinfo                                            [ OK ]
    /sbin/modprobe                                           [ OK ]
    /sbin/rmmod                                              [ OK ]
    /sbin/route                                              [ OK ]
    /sbin/runlevel                                           [ OK ]
    /sbin/sulogin                                            [ OK ]
    /sbin/sysctl                                             [ OK ]
    /bin/bash                                                [ OK ]
    /bin/cat                                                 [ OK ]
    /bin/chmod                                               [ OK ]
    /bin/chown                                               [ OK ]
    /bin/cp                                                  [ OK ]
    /bin/date                                                [ OK ]
    /bin/df                                                  [ OK ]
    /bin/dmesg                                               [ OK ]
    /bin/echo                                                [ OK ]
    /bin/egrep                                               [ OK ]
    /bin/fgrep                                               [ OK ]
    /bin/fuser                                               [ OK ]
    /bin/grep                                                [ OK ]
    /bin/ip                                                  [ OK ]
    /bin/kill                                                [ OK ]
    /bin/less                                                [ OK ]
    /bin/login                                               [ OK ]
    /bin/ls                                                  [ OK ]
    /bin/lsmod                                               [ OK ]
    /bin/mktemp                                              [ OK ]
    /bin/more                                                [ OK ]
    /bin/mount                                               [ OK ]
    /bin/mv                                                  [ OK ]
    /bin/netstat                                             [ OK ]
    /bin/ping                                                [ OK ]
    /bin/ps                                                  [ OK ]
    /bin/pwd                                                 [ OK ]
    /bin/readlink                                            [ OK ]
    /bin/sed                                                 [ OK ]
    /bin/sh                                                  [ OK ]
    /bin/su                                                  [ OK ]
    /bin/touch                                               [ OK ]
    /bin/uname                                               [ OK ]
    /bin/which                                               [ OK ]
    /bin/kmod                                                [ OK ]
    /bin/websh                                               [ Warning ]
[Press <ENTER> to continue]
  Performing check of known rootkit files and directories
    55808 Trojan - Variant A                                 [ Not found ]
    ADM Worm                                                 [ Not found ]
    AjaKit Rootkit                                           [ Not found ]
    Adore Rootkit                                            [ Not found ]
    aPa Kit                                                  [ Not found ]
    Apache Worm                                              [ Not found ]
    Ambient (ark) Rootkit                                    [ Not found ]
    Balaur Rootkit                                           [ Not found ]
    BeastKit Rootkit                                         [ Not found ]
    beX2 Rootkit                                             [ Not found ]
    BOBKit Rootkit                                           [ Not found ]
    cb Rootkit                                               [ Not found ]
    CiNIK Worm (Slapper.B variant)                           [ Not found ]
    Danny-Boy's Abuse Kit                                    [ Not found ]
    Devil RootKit                                            [ Not found ]
    Dica-Kit Rootkit                                         [ Not found ]
    Dreams Rootkit                                           [ Not found ]
    Duarawkz Rootkit                                         [ Not found ]
    Enye LKM                                                 [ Not found ]
    Flea Linux Rootkit                                       [ Not found ]
    Fu Rootkit                                               [ Not found ]
    Fuck`it Rootkit                                          [ Not found ]
    GasKit Rootkit                                           [ Not found ]
    Heroin LKM                                               [ Not found ]
    HjC Kit                                                  [ Not found ]
    ignoKit Rootkit                                          [ Not found ]
    IntoXonia-NG Rootkit                                     [ Not found ]
    Irix Rootkit                                             [ Not found ]
    Jynx Rootkit                                             [ Not found ]
    KBeast Rootkit                                           [ Not found ]
    Kitko Rootkit                                            [ Not found ]
    Knark Rootkit                                            [ Not found ]
    ld-linuxv.so Rootkit                                     [ Not found ]
    Li0n Worm                                                [ Not found ]
    Lockit / LJK2 Rootkit                                    [ Not found ]
    Mood-NT Rootkit                                          [ Not found ]
    MRK Rootkit                                              [ Not found ]
    Ni0 Rootkit                                              [ Not found ]
    Ohhara Rootkit                                           [ Not found ]
    Optic Kit (Tux) Worm                                     [ Not found ]
    Oz Rootkit                                               [ Not found ]
    Phalanx Rootkit                                          [ Not found ]
    Phalanx2 Rootkit                                         [ Not found ]
    Phalanx2 Rootkit (extended tests)                        [ Not found ]
    Portacelo Rootkit                                        [ Not found ]
    R3dstorm Toolkit                                         [ Not found ]
    RH-Sharpe's Rootkit                                      [ Not found ]
    RSHA's Rootkit                                           [ Not found ]
    Scalper Worm                                             [ Not found ]
    Sebek LKM                                                [ Not found ]
    Shutdown Rootkit                                         [ Not found ]
    Shutdown Rootkit                                         [ Not found ]
    SHV4 Rootkit                                             [ Not found ]
    SHV5 Rootkit                                             [ Not found ]
    Sin Rootkit                                              [ Not found ]
    Slapper Worm                                             [ Not found ]
    Sneakin Rootkit                                          [ Not found ]
    'Spanish' Rootkit                                        [ Not found ]
    Suckit Rootkit                                           [ Not found ]
    Superkit Rootkit                                         [ Not found ]
    TBD (Telnet BackDoor)                                    [ Not found ]
    TeLeKiT Rootkit                                          [ Not found ]
    T0rn Rootkit                                             [ Not found ]
    trNkit Rootkit                                           [ Not found ]
    Trojanit Kit                                             [ Not found ]
    Tuxtendo Rootkit                                         [ Not found ]
    URK Rootkit                                              [ Not found ]
    Vampire Rootkit                                          [ Not found ]
    VcKit Rootkit                                            [ Not found ]
    Volc Rootkit                                             [ Not found ]
    Xzibit Rootkit                                           [ Not found ]
    zaRwT.KiT Rootkit                                        [ Not found ]
    ZK Rootkit                                               [ Not found ]
[Press <ENTER> to continue]
  Performing additional rootkit checks
    Suckit Rookit additional checks                          [ OK ]
    Checking for possible rootkit files and directories      [ None found ]
    Checking for possible rootkit strings                    [ None found ]
  Performing malware checks
    Checking running processes for suspicious files          [ None found ]
    Checking for login backdoors                             [ None found ]
    Checking for suspicious directories                      [ None found ]
    Checking for sniffer log files                           [ None found ]
  Performing Linux specific checks
    Checking loaded kernel modules                           [ OK ]
    Checking kernel module names                             [ OK ]
[Press <ENTER> to continue]
Checking the network...
  Performing checks on the network ports
    Checking for backdoor ports                              [ None found ]
    Checking for hidden ports                                [ Skipped ]
  Performing checks on the network interfaces
    Checking for promiscuous interfaces                      [ None found ]
Checking the local host...
  Performing system boot checks
    Checking for local host name                             [ Found ]
    Checking for system startup files                        [ Found ]
    Checking system startup files for malware                [ None found ]
  Performing group and account checks
    Checking for passwd file                                 [ Found ]
    Checking for root equivalent (UID 0) accounts            [ None found ]
    Checking for passwordless accounts                       [ None found ]
    Checking for passwd file changes                         [ None found ]
    Checking for group file changes                          [ None found ]
    Checking root account shell history files                [ OK ]
  Performing system configuration file checks
    Checking for SSH configuration file                      [ Found ]
    Checking if SSH root access is allowed                   [ Not allowed ]
    Checking if SSH protocol v1 is allowed                   [ Not allowed ]
    Checking for running syslog daemon                       [ Found ]
    Checking for syslog configuration file                   [ Found ]
    Checking if syslog remote logging is allowed             [ Not allowed ]
  Performing filesystem checks
    Checking /dev for suspicious file types                  [ None found ]
    Checking for hidden files and directories                [ Warning ]
[Press <ENTER> to continue]
System checks summary
=====================
File properties checks...
    Files checked: 138
    Suspect files: 2
Rootkit checks...
    Rootkits checked : 307
    Possible rootkits: 0
Applications checks...
    All checks skipped
The system checks took: 4 minutes and 43 seconds
All results have been written to the log file (/var/log/rkhunter.log)
One or more warnings have been found while checking the system.
Please check the log file (/var/log/rkhunter.log)

The warnings in the log file:

[18:17:33]   /usr/bin/unhide.rb                              [ Warning ]
[18:17:33] Warning: The command '/usr/bin/unhide.rb' has been replaced by a script: /usr/bin/unhide.rb: Ruby script, ASCII text
[18:17:54]   /bin/websh                                      [ Warning ]
[18:17:54] Warning: The command '/bin/websh' has been replaced by a script: /bin/websh: Bourne-Again shell script, ASCII text executable, with very long lines
[18:21:22] Warning: Hidden directory found: '/etc/.java'

The second of these, /bin/websh is a BOA thing.

I don't know the origin of /etc/.java, it contains two empty files:

/etc/.java/.systemPrefs/.systemRootModFile
/etc/.java/.systemPrefs/.system.lock

The bottom line is I don't think the server is compromised but I'm not 100% sure and could spend more time on this, but I'm not sure I would find anything if I did...

chris — Fri, 28 Nov 2014 18:37:10 GMT

/usr/bin/unhide.rb is a script to find hidden processes, I don't know where it's from, it doesn't find any:

/usr/bin/unhide.rb
Scanning for hidden processes...
No hidden processes found!

It's part of a debian package:

dpkg -S /usr/bin/unhide.rb
unhide.rb: /usr/bin/unhide.rb
aptitude search unhide | grep ^i
i A unhide.rb                       - Forensic tool to find processes hidden by rootkits

ed — Mon, 01 Dec 2014 12:02:07 GMT

DELETE space.transitionnetwork.org entirely - confirmed that no-one needs anything.

paul — Mon, 01 Dec 2014 12:04:34 GMT

I'm on it ..

hours, totalhours changed

paul — Mon, 01 Dec 2014 12:19:37 GMT

hours changed from 0.0 to 0.25
totalhours changed from 3.575 to 3.825

Deleted the site and platform:

space.transitionnetwork.org / Open Atrium 2.19 7.31.1 P.008

Also deleted the empty platforms:

Open Atrium 2.21 7.31.1 P.008 Open Atrium 2.22 7.32.1 P.009 Open Atrium 2.24 7.33.1 P.010

Also scheduled deletion of my failed platforms from earlier this morning.

ed — Mon, 01 Dec 2014 13:06:07 GMT