[admin (no pm's please)]

I have spotted and eliminated a forum that had accumulated over 200,000 spam postings   This was I think automated and the sort of thing that was probably coming in bursts and probably contributing to some of the problems we have been seeing on the Slarti server recently

[admin (no pm's please)]

More investigation leads me to believe that this has been a widespread and insidious cancer on the entire system developing over the last several months since the phpbb3 CAPCHA system was cracked  

Quite possibly ending up with as much 10,000 spam postings daily, something like 25% of the legitimate total. Now that is "only" 6 or so posts a minute, but with bots it would not be spread out but done in fashion that would create load spikes.

Combine that with the spam increasing pressure on the already problematic backup system and don't get a good result

Hopefully there has been a whole host of actions this week that will improves things.

1) Removing the IO bottle neck caused by backups, by making them bypass the disks piping directly to their destination.
2) Disabling and in extreme cases deleting badly spammed forums.
3) Making the gallery more efficient.
4) A major purge of forums that have been inactive a very long time.
5) Making downsizer.net forums (one of the most active on the system and not a myff forum) more efficient.

I'm hoping these measures will delay the need for a new server, every server we have when responding normally is perfectly fast, as I am typing this at what is peak access time, I have looked at a large forum on every server and it has been running at a perfectly decent speed. It has been the load spikes that have caused issues.

[panther_dust]

i had this on one of my forums.... or an attemt anyway to stop this i simple set the user sighn up to user confirms by email. spambot doesnt use legit email so it cant retrieve it therefor cannot get access.

I hadover 150 malicius member sighn ups with names like ccccqcqqiugfkgv which is pointless.
if you wanna take alook admin its http://www.crumpled-tapes.co.nr

[Zudane]

A lot of forums had those problems, but I think it's the extreme cases that are the problem.

[admin (no pm's please)]

In our case the sheer accumulation combined with the fact that it is bots doing it.

We are running three powerful servers and that has the unfortunate side effect that some issues need to get to fairly extreme levels before the problems really start flagging themselves.

We have been seeing increased load in recent months, but we have also been seeing a totally valid rise in the number of active forums on the system. We have highlighted the spam issue to phpbb3 forum owners and last time the issue was looked at, it only looked like a minor pain in the neck that phpbb3 admins would by and large deal with themselves.

Instead it is plain that a lot have simply abandoned ship leaving forums open to spammers  

[panther_dust]

i left them there becouse it makes the forum look bigger haha but i am still beeing very caucious about it and i always keep my eye on all the forums i help run..

[admin (no pm's please)]

Slarti again got his last night and the services needed to be restarted

It may be that whilst the boards effected have been disabled, the attack is still heavy enough to cause a problem.

I will suspend said boards rather than disabling them which will mean the admins will need to come here to get them reinstated, but will lighten any attack load by a large factor.

[panther_dust]

can u not simply do what i did? turn all the bords to user controlled registartion so they have to click a link in an email...

[admin (no pm's please)]

We don't override owner choices unless there is a totally compelling reason.

[panther_dust]

server overload... sounds good enough to me..

[admin (no pm's please)]

Good enough to cause radical action yes. In fact forum suspension, but forum suspension whilst more severe is not the same as overriding owner choices and just overriding the choices does not stop the problems on the forums where the issue is occurring. e.g. the spammers are registered and active.

[panther_dust]

very true. so it would be a case of erase the spambots and ban the ip from the server then lots of modifications.. i see.....

[Zudane]

The spam bots come from hundreds of IPs.  IP bans actually aren't that useful, because each time a typical person reconnects to the internet their IP changes slightly.  Someone else gets that IP and is wrongfully banned, while the person you wanted is free to visit.  Ban an IP range and you ban a collection of people for the sake of one.

All in all, I like admin's idea, because it doesn't change how the forum is run, it simply disallows access unless the owner is willing to make the needed changes.

[admin (no pm's please)]

IP bans don't work for the reasons stated.

The important thing is to lesson the impact on any spambots. A suspension does this 1000x more effectively than a board being disabled.

[admin (no pm's please)]

Having wiped out the spammers, they continue to creep back onto other abandoned forums.

Of course now the problem is more recognized as a chronic one it can be part of a daily review, and the new management code will create a tighter net.

[Zudane]

I know your stance on this already, but I still really thing that changing the CAPTCHA for both phpbb2 and 3 to reCAPTCHA which is an external CAPTCHA system run by a US University that uses it to help in the process of scanning in books, using words that a computer has trouble reading, then distorts it and puts it along side a second word that is already known.

The main bonus of this is that they keep records of it all and if they detect that it's been cracked, they will tweak the code which means that you wouldn't have to worry about anybody breaking the code.

But still, that's just my idea about it.

[admin (no pm's please)]

It is a good point, but then equally so is the point that with the new phpbb3 release in due, which one has to assume will also address this issue, then now is not the time to jump ship with our own fix.

I'm not adverse to doing an alternative, as our own in house and uncracked phpbb2 CAPCHA demonstrates. But logically the phpbb3 team should do a good solution with all the same benefits and the more we stay in line with the main stream phpbb3 code the better.

[Zudane]

[admin (no pm's please)]

Which is why your point is a good one, even if not one I am willing to (on balance) accept at this point.

I would not make a bet that you won't be able to say "told you so" in 6 months time!

[admin (no pm's please)]

Seems this forum itself is now being pummeled by 100's of mainly Eastern European IP's

We have hit 398 online.

[admin (no pm's please)]

With 315ips firewalled   the attack has momentarily abated.

It may be some legit users have been hit, but whilst I apologize for this, there had to be a fairly broad brush used.

[admin (no pm's please)]

Now up to 462 bans.

I have my suspicions as to what may be going on here, I think it may be an attempt to crack our CAPTCHA system.

[Zudane]

Wow.  Someone must not like this system O_o

[admin (no pm's please)]

Like does not come into it, this is not a DDOS attack.

[admin (no pm's please)]

This morning all looks superficially quiet...

But turn on CAPTCHA and within seconds there are a flood of attempts At least they do get quickly blocked.