Spam account war

[ed]

Aim:

tell drupal (and server level stuff?) to sniff out and destroy spam accounts without them knowing we did it, and ban them from doing it again

Wiki page:

​https://wiki.transitionnetwork.org/Spam_accounts

[mark]

On City of Sanctuary, I've built a couple of Views which make it easier to spot the spam comments that get past Mollom, and to spot spam user accounts. I find Drupal's built in comment moderation and user listing admin pages to be insufficient for quick decisions and processing.

Both my Views use Views Bulk Operations so that I can quickly take care of multiple things. I'll attach screen shots of both Views before a recent clean-up.

This approach works OK if somebody can spare 5-10 minutes each week to keep on top of the situation. I just left City of Sanctuary alone for a month, and it took almost an hour to clean up spam comments and users, even with these tools.

One problem I've found is that Mollom (when I last checked, about a year go) doesn't integrate with Actions. If it did, I could create the View I'd really like, which would allow me to:

• View comments and delete them in bulk, AND, as they're being deleted, a Rule would report those comments to Mollom and block their authors.

If I was cleverer (or had clever friends :) I could envisage adding in other actions to be executed when spam comments are bulk-deleted - like blocking the IP address used to post the comment, or blocking that email address from registering again.

Another thing which has helped spot spam user accounts on other sites is to add a simple question to the user account/profile, for example, "Why are you interested in the Transition Movement?". The info here can be private, and if shown on a list of users along with the username and email address, can make it really easy to tell genuine users from spammers.

So anyway, don't know what you all think about this. The basic issue for me is that it is hard to envision a system for perfectly automating the blocking of spam, because humans are clever and can post spam which gets past systems. So, put a human at our end too, filtering the spam that sneaks past the automated defences with tools that make it as quick and painless as possible. Views, VBO and Actions make quite a lot possible on this front.

[mark]

City of Sanctuary - bulk delete comments (for spam fighting)

[mark]

City of Sanctuary - bulk delete users (for spam fighting)

[ed]

Thanks Mark - the comments I'm basically on top of, it's the user accts (e.g. ​https://www.transitionnetwork.org/people/vu-vu) that have to be treated with extreme prejudice as per the wiki page here: ​https://wiki.transitionnetwork.org/Spam_accounts My serious suggestion are therein.

[ed]

moving this to critical as jim's on today and i'd like to see some brutal action

[ed]

Following conversations with Jim and Mark about this, Mark's suggestion about using VBOs for admin to delete the users concurs with Jim's suggestion.

I want this done asap. We've got a massive comms thing about to kick off and this is a vital beginning. We are going to have to make it easier for users to comment and track stuff (all of Rob Hopkins' work is coming onto TN.org), so need to start prepping for the spam that will come with that.

Please can either Jim or Mark sort this out? I am going to go on about this until it is done.

[jim]

FYI I'd already created this view a while back, and am spending a few minutes finishing the job...

The view admin_users_spam is available here in 'Reports -> Spam users -> Spam users - same first/last names' and at ​https://www.transitionnetwork.org/admin/reports/spam/same-names

IMPORTANT NOTE: Just deleting a user WILL NOT remove them from any mailing lists on MailChimp?. If we want this we'll need to patch the MC module or add custom code.

Ed?

[ed]

Awesome. I am itching to nuke those arsewipes - important question first:

1. What are the specific differences between block and delete?
2. Will blocking the account delete the profile and add them to the ban list for later?
3. Will deleting the account delete the profile but not ban them for later?

2. Can we set a rule about unpublishing profiles/blocking accounts if they have not authorised within a specified period of time?

3. Mailing list clean up will be needed. Or would we have more flexibility with another newsletter install on puffin (as discussed elsewhere)?

[jim]

1. Block sets them to 'blocked' so can't log in, delete is nuclear.
2. Blocking: No delete profile, yes ban them
3. Deleting: Yes delete profile, no ban them
4. Yes, login Toboggan already does this... See: ​https://www.transitionnetwork.org/admin/user/logintoboggan "Delete unvalidated users after:" section. Pick a timespan, or tell me what you want - days to 1 year available.
5. Not necessarily. MC Drupal doesn't by default remove subscriptions on delete, though it can certainly be done via Rules and some custom code (I've seen example code on drupal.org). Isn't MC cutting it any more?

[jim]

Or, if the functions exist in MC, we could export the above emails first as CSV so you can get MC to unsubscribe/remove too?

[ed]

1, 2, 3: I want nuclear war with extreme prejudice. Can I block the account *and* delete the profile page? I want to ban them from ever coming back and delete the profiles.

4. I will reset logintoboggan to to delete users after one week next week.
5. Mailchimp cleaning. I'll look into if we can export the blocked list for MC cleaning purposes

5a. MC options: MC is brilliant in my book. Chris keen on moving away from corporates, TN quite keen on re-considering the cost - there's some tactical rumbling here which I'm pondering.

[jim]

The ban will only work against emails -- and these are either random or got from some poor soul... There's no shortage of them. So a ban is only partly useful without the IP being logged and other logic happening.

I'd propose killing the users for now and looking at other additional modules to kill off spammers e.g. ​http://drupal.org/project/spambot and ​http://drupal.org/project/badbehavior and ​http://drupal.org/project/botcha <-- this last one looks very interesting.

4&5: OK, shout if you need more. Ideally we can auto-unsubscribe accounts that get though after we tighten security.

Shall I install Botcha and Spambot? Bad Behaviour on standby...

[ed]

1,2,3. Monday morning - I'll delete all the spam accounts with same firstname lastname as per your view. I can do multi-unsubs in mailchimp too, so a .xls download of this view would be good. Can I add the .xls download myself to the spammers view? I'll try if Jim doesn't get to it before Monday morning. Also can I see more than 50 at a time?

6. Install Botcha and spambot - in the next few months we'll need to try a few approaches so that when we move Transition Culture (with a healthy commenter culture), we have a relatively easy system for them to use. We will also need to review the registration and commenting patterns for TN.org as we move to having a real editor - that will be part of a next phase.

7. What does bad behaviour do that the others don't do?

8. Then it will be about how to extract the silly names with personal profiles but that's a next step.

9. Patching MC module to unsub spam accounts is longer term but desirable.

10. Comments - mollom is doing fine afaict - the only spam comments getting through look to be humans (?)

[jim]

1,2,3 CSV file download added -- see foot of view.

NOTE some users have no names (as to plenty of spammers), but these "I've got no profile" users are worthy of deletion by virtue of not following site guidelines... I say nuke them all from orbit. It's the only way to be sure.

6. Spambot, Botcha (plus required other modules) now added to server and platform makefile.
7. Bad Behaviour is similar to Spambot, looks like another way to check submissions, IPs and other stuff using this library: ​http://bad-behavior.ioerror.us/download/ -- basically it's something we can try if these two don't work.
8. I reckon the new modules will minimise this faff...
9. It should be a hook implementation to add to our Transition Extras module... Or we can try patching... For a future ticket!
10. Humans are dicks. Some of the worst people I've met were human. ;)

[jim]

Spambot set up here: ​https://www.transitionnetwork.org/admin/settings/spambot -- I've set to check 50 user accounts per cron run (hourly) and block those that match known spammers automatically. It will also block all emails/IPs that are registered as spammer scum. Note that all 'trusted' users (i.e. above grunt level) are immune from scanning.

Botcha in place too (it's pretty much plug and play though there is config if we need to change what forms it looks at).

This ticket and #307 are done IMHO... I'll let you decide what needs doing or if you want to close these.

It's a new anti-spam dawn, Ed. Enjoy...

[ed]

1. Spam accounts (same last/first name) deleted today from report
2. Spam accounts unsubscribed from mailchimp. NB: this was not many. ie the spam accounts are generally *not* subbing to the newsletter.
3. I will keep an eye on the blocked list that will grown from spambot and delete them all in a week when spambot has done it's thing
4. Not clear to Ed if botcha replaces captcha - it sounds like it could be a replacement but I'm not sure - for later

taking down to minor for tracking for ed

thanks jim

[jim]

Botcha and Captcha tackle the same thing from two sides... Suggest you read the Botcha page as there's lots of handy stuff here: ​http://drupal.org/project/botcha

It might be that Botcha is better than Captcha on registration pages... Time and tinkering will tell.

[ed]

Yes. Had read and enjoyed Botcha page but not got direct answer so leaving this here as a trail. Wiki page updated here: ​https://wiki.transitionnetwork.org/Spam_accounts