My security system

Posted: Sat Jan 14, 2006 9:42 pm

I've wrote a new security system for Dragonfly that detects flooding, referer spam, bots, bad user-agents and more...

Due to that some things triggered my mind and i started to investigate and these are some results:

1. Russian referer spambot changes IP on each call
2. Google doesn't obey Crawl-delay in robots.txt
3. Medusa browser is a website harvester
4. Some bad bots identify as a normal browser and the most famous string is

Code:

Mozilla/4.0 (compatible; MSIE 6.0; Win32)

But there doesn't exists such thing as 'Win32' it's either 'Windows 95/98/ME' or 'Windows NT 5.0/5.1'

By putting the protection in place my anonymous visitor counts dropped from 200+ to 5+

Hereby i warn you all not to get fooled by your visitor counts in php-nuke and if you see excessive resource usage add a flood protector.

Sells PC To Pay For Divorce Joined: Posts: 5661

Win32 is the Windows API on Microsoft Windows 9X, NT, and 2000.

Okay and API means:Application Program Interface. A good API provides a group of routines, protocols and tools that programmers use to develop a program with greater ease. By doing so, an API provides consistency across applications by providing the same basic tools for all programmers to use. Operating systems such as Windows NT have an API, as do most popular web servers. (Also see Win32, WSAPI, ISAPI and NSAPI )

lol

Posted: Sat Jan 14, 2006 10:15 pm

duh i know that but did you find a IE browser that identifies as Win32 ?
I didn't talk about apps i talked about ua's Laughing

Posted: Sun Jan 15, 2006 5:25 am

im just joking around dj...
but i found this....

http://msdn.microsoft.com/library/default.asp?url=/archive/en-us/dnarwebtool/html/msdn_inet.asp

Posted: Sun Jan 15, 2006 10:40 am

I know that also, i use those commands often in C++ apps that i write.
But it doesn't explain why many japanese bots use 'Win32' Wink

Site Admin/Owner Joined: Aug 27, 2002 Posts: 17088

djmaze,

Would you be willing to share that code for public use?

Posted: Sun Jan 15, 2006 6:37 pm

http://dragonflycms.org/cvs/html/includes/classes/security.php
http://dragonflycms.org/cvs/html/install/sql/data/agents.php

This is an result of a previous system i wrote called "Trap_Crawler"

Trap_Crawler had an .htaccess rewrite

Code:

RewriteRule ^robots\.txt$ /trap_crawler/robots.php [L]

robots.php stores each unique robots.txt request (IP and User-agent).
I also had a hidden url to /trap_crawler/index.php which logged bots that ignored my robots.txt settings.

After 2 months i had a nice listing which produced my security system.

Posted: Wed Jan 18, 2006 11:25 am

Raven

Do you get it or is it still a lot of the unknown?

Posted: Wed Jan 18, 2006 2:13 pm

djmaze, sorry for not getting back to you on this. I have this on my list of things to look at as soon as I get time. Thanks for posting it!