Ticket #544 (closed maintenance: fixed)

Opened 4 years ago

Last modified 3 years ago

CSF / LDF false positive blocks on Puffin

Reported by: chris Owned by: chris
Priority: major Milestone: Maintenance
Component: Live server Keywords:
Cc: ed Estimated Number of Hours: 0.0
Add Hours to Ticket: 0 Billable?: yes
Total Hours: 0.92

Description

Ticket to keep track of CSF /LDF issues on Puffin, see wiki:PuffinServer#CSFLDF

Change History

comment:1 Changed 4 years ago by chris

  • Add Hours to Ticket changed from 0.0 to 0.25
  • Total Hours changed from 0.0 to 0.25

Our server monitoring if puffin is up or down was blocked:

csf -g 81.95.52.66

Chain            num   pkts bytes target     prot opt in     out     source               destination         

DENYIN           99     247 19164 DROP       all  --  !lo    *       81.95.52.66          0.0.0.0/0           

DENYOUT          99       0     0 DROP       all  --  *      !lo     0.0.0.0/0            81.95.52.66

csf.deny: 81.95.52.66 # lfd: (sshd) Failed SSH login from 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org): 5 in the last 300 secs - Fri May  3 21:30:38 2013

I have unblocked it:

csf -dr 81.95.52.66
Removing rule...
DROP  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0  
DROP  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66  

comment:2 Changed 4 years ago by ed

  • Milestone set to Maintenance

comment:3 Changed 3 years ago by chris

  • Add Hours to Ticket changed from 0.0 to 0.15
  • Total Hours changed from 0.25 to 0.4

Our monitoring server was blocked again:

csf -g 81.95.52.66
  
  Chain            num   pkts bytes target     prot opt in     out     source               destination
  
  DENYIN           95      50  3768 DROP       all  --  !lo    *       81.95.52.66          0.0.0.0/0
  
  DENYOUT          95       0     0 DROP       all  --  *      !lo     0.0.0.0/0            81.95.52.66
  
  csf.deny: 81.95.52.66 # lfd: (sshd) Failed SSH login from 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org): 5 in the last 300 secs - Thu May 23 12:43:04 2013

So I have unblocked it again:

csf -dr 81.95.52.66
  Removing rule...
  DROP  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0
  DROP  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66

csf -g 81.95.52.66
  
  Chain            num   pkts bytes target     prot opt in     out     source               destination         
  No matches found for 81.95.52.66 in iptables

I need to work out how to permanently whitelist this server.

I have added a link from wiki:PuffinServer#CSFLDF to this ticket.

comment:4 Changed 3 years ago by chris

  • Add Hours to Ticket changed from 0.0 to 0.25
  • Total Hours changed from 0.4 to 0.65

The firewall blocking the webarch monitoring server has been an ongoing issues resulting in lots of false positive emails, following the notes here: http://hostinghints.co.uk/2012/05/notes-on-csf-and-lfd-whitelisting-and-blacklisting/

csf -a 81.95.52.66
  Adding 81.95.52.66 to csf.allow and iptables ACCEPT...
  ACCEPT  all opt -- in !lo out *  81.95.52.66  -> 0.0.0.0/0  
  ACCEPT  all opt -- in * out !lo  0.0.0.0/0  -> 81.95.52.66  
csf -g 81.95.52.66
  Chain            num   pkts bytes target     prot opt in     out     source           destination         
  ALLOWIN          1        5   387 ACCEPT     all  --  !lo    *       81.95.52.66          0.0.0.0/0
  ALLOWOUT         1        4   908 ACCEPT     all  --  *      !lo     0.0.0.0/0            81.95.52.66

In /etc/csf/csf.conf already had IGNORE_ALLOW = "0".

comment:5 Changed 3 years ago by chris

  • Add Hours to Ticket changed from 0.0 to 0.1
  • Status changed from new to closed
  • Resolution set to fixed
  • Total Hours changed from 0.65 to 0.75

We have this in /root/.barracuda.cnf to ensure that the variables we change are not clobbered by BOA:

_CUSTOM_CONFIG_CSF=YES

This has been added to wiki:PuffinServer#CSFLDF so now closing this ticket.

comment:6 Changed 3 years ago by chris

  • Add Hours to Ticket changed from 0.0 to 0.17
  • Total Hours changed from 0.75 to 0.92

The CSF firewall tried to block the Webarchitects monitoring server again:

From: root@puffin.webarch.net
Date: Thu, 16 Jan 2014 13:16:50 +0000 (GMT)
To: chris@webarchitects.co.uk
Subject: lfd on puffin.webarch.net: blocked 81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org)

Time:     Thu Jan 16 13:16:50 2014 +0000
IP:       81.95.52.66 (GB/United Kingdom/nsa.rat.burntout.org)
Failures: 5 (sshd)
Interval: 300 seconds
Blocked:  Permanent Block (IP match in csf.allow, block may not work)

Log entries:

Jan 16 13:08:18 puffin sshd[19375]: Did not receive identification string from 81.95.52.66
Jan 16 13:09:33 puffin sshd[19440]: Did not receive identification string from 81.95.52.66
Jan 16 13:11:15 puffin sshd[19522]: Did not receive identification string from 81.95.52.66
Jan 16 13:12:32 puffin sshd[19580]: Did not receive identification string from 81.95.52.66
Jan 16 13:16:13 puffin sshd[20671]: Did not receive identification string from 81.95.52.66

This happened at the same time as a load spike:

From: root@puffin.webarch.net
Date: Thu, 16 Jan 2014 13:16:55 +0000 (GMT)
To: chris@webarchitects.co.uk
Subject: lfd on puffin.webarch.net: High 5 minute load average alert - 73.53

[-- Attachment #1 --]
[-- Type: text/plain, Encoding: 7bit, Size: 0.2K --]

Time:                    Thu Jan 16 13:16:55 2014 +0000
1 Min Load Avg:          59.08
5 Min Load Avg:          73.53
15 Min Load Avg:         41.10
Running/Total Processes: 5/430

The block doesn't appear to have actually been added:

csf -g 81.95.52.66

Chain            num   pkts bytes target     prot opt in     out     source               destination         

ALLOWIN          1       48  3220 ACCEPT     all  --  !lo    *       81.95.52.66          0.0.0.0/0

ALLOWOUT         1       36  8708 ACCEPT     all  --  *      !lo     0.0.0.0/0            81.95.52.66

So I don't think there isn't anything to do here apart from note what happened.

Note: See TracTickets for help on using tickets.