Transition Technology: Ticket #544: CSF / LDF false positive blocks on Puffin

hours, totalhours changed

chris — Sat, 04 May 2013 11:04:45 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.0 to 0.25

Our server monitoring if puffin is up or down was blocked:

csf -g 81.95.52.66
Chain            num   pkts bytes target     prot opt in     out     source               destination
DENYIN           99     247 19164 DROP       all  --  !lo    *       81.95.52.66          0.0.0.0/0
DENYOUT          99       0     0 DROP       all  --  *      !lo     0.0.0.0/0            81.95.52.66
csf.deny: 81.95.52.66 # lfd: (sshd) Failed SSH login from 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org): 5 in the last 300 secs - Fri May  3 21:30:38 2013

I have unblocked it:

csf -dr 81.95.52.66
Removing rule...
DROP  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0
DROP  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66

milestone set

ed — Thu, 09 May 2013 09:41:01 GMT

milestone set to Maintenance

hours, totalhours changed

chris — Fri, 24 May 2013 10:21:35 GMT

hours changed from 0.0 to 0.15
totalhours changed from 0.25 to 0.4

Our monitoring server was blocked again:

csf -g 81.95.52.66
  Chain            num   pkts bytes target     prot opt in     out     source               destination
  DENYIN           95      50  3768 DROP       all  --  !lo    *       81.95.52.66          0.0.0.0/0
  DENYOUT          95       0     0 DROP       all  --  *      !lo     0.0.0.0/0            81.95.52.66
  csf.deny: 81.95.52.66 # lfd: (sshd) Failed SSH login from 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org): 5 in the last 300 secs - Thu May 23 12:43:04 2013

So I have unblocked it again:

csf -dr 81.95.52.66
  Removing rule...
  DROP  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0
  DROP  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66
csf -g 81.95.52.66
  Chain            num   pkts bytes target     prot opt in     out     source               destination
  No matches found for 81.95.52.66 in iptables

I need to work out how to permanently whitelist this server.

I have added a link from wiki:PuffinServer#CSFLDF to this ticket.

hours, totalhours changed

chris — Wed, 07 Aug 2013 10:14:10 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.4 to 0.65

The firewall blocking the webarch monitoring server has been an ongoing issues resulting in lots of false positive emails, following the notes here: http://hostinghints.co.uk/2012/05/notes-on-csf-and-lfd-whitelisting-and-blacklisting/

csf -a 81.95.52.66
  Adding 81.95.52.66 to csf.allow and iptables ACCEPT...
  ACCEPT  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0
  ACCEPT  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66
csf -g 81.95.52.66
  Chain            num   pkts bytes target     prot opt in     out     source           destination
  ALLOWIN          1        5   387 ACCEPT     all  --  !lo    *       81.95.52.66          0.0.0.0/0
  ALLOWOUT         1        4   908 ACCEPT     all  --  *      !lo     0.0.0.0/0            81.95.52.66

In /etc/csf/csf.conf already had IGNORE_ALLOW = "0".

hours, status, totalhours changed; resolution set

chris — Fri, 15 Nov 2013 13:28:29 GMT

hours changed from 0.0 to 0.1
status changed from new to closed
resolution set to fixed
totalhours changed from 0.65 to 0.75

We have this in /root/.barracuda.cnf to ensure that the variables we change are not clobbered by BOA:

_CUSTOM_CONFIG_CSF=YES

This has been added to wiki:PuffinServer#CSFLDF so now closing this ticket.

hours, totalhours changed

chris — Thu, 16 Jan 2014 13:45:17 GMT

hours changed from 0.0 to 0.17
totalhours changed from 0.75 to 0.92

The CSF firewall tried to block the Webarchitects monitoring server again:

From: root@puffin.webarch.net
Date: Thu, 16 Jan 2014 13:16:50 +0000 (GMT)
To: chris@webarchitects.co.uk
Subject: lfd on puffin.webarch.net: blocked 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org)
Time:     Thu Jan 16 13:16:50 2014 +0000
IP:       81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org)
Failures: 5 (sshd)
Interval: 300 seconds
Blocked:  Permanent Block (IP match in csf.allow, block may not work)
Log entries:
Jan 16 13:08:18 puffin sshd[19375]: Did not receive identification string from 81.95.52.66
Jan 16 13:09:33 puffin sshd[19440]: Did not receive identification string from 81.95.52.66
Jan 16 13:11:15 puffin sshd[19522]: Did not receive identification string from 81.95.52.66
Jan 16 13:12:32 puffin sshd[19580]: Did not receive identification string from 81.95.52.66
Jan 16 13:16:13 puffin sshd[20671]: Did not receive identification string from 81.95.52.66

This happened at the same time as a load spike:

From: root@puffin.webarch.net
Date: Thu, 16 Jan 2014 13:16:55 +0000 (GMT)
To: chris@webarchitects.co.uk
Subject: lfd on puffin.webarch.net: High 5 minute load average alert - 73.53
[-- Attachment #1 --]
[-- Type: text/plain, Encoding: 7bit, Size: 0.2K --]
Time:                    Thu Jan 16 13:16:55 2014 +0000
1 Min Load Avg:          59.08
5 Min Load Avg:          73.53
15 Min Load Avg:         41.10
Running/Total Processes: 5/430

The block doesn't appear to have actually been added:

csf -g 81.95.52.66
Chain            num   pkts bytes target     prot opt in     out     source               destination
ALLOWIN          1       48  3220 ACCEPT     all  --  !lo    *       81.95.52.66          0.0.0.0/0
ALLOWOUT         1       36  8708 ACCEPT     all  --  *      !lo     0.0.0.0/0            81.95.52.66

So I don't think there isn't anything to do here apart from note what happened.