Google News Search Blocked

New Member Joined: Aug 07, 2004 Posts: 8

I am working with Google to have my site tapped for any news. I've received a few "blocked" emails from Sentinel, and I'm wondering if Sentinel blocks Google from searching areas of my site.

From Google wrote:

We regret that we are having trouble including articles from your site in Google News. While www.legalizefreedom.us is working, an error message indicates that we are being blocked from viewing your content.

Are they having problems viewing my content due to Sentinel?

Thanks.

Posted: Sun Jan 09, 2005 2:33 pm

Are you getting alerts that its Sentienl blocking them? If not what User Agent? LWP::Simple maybe? I have no idea what the grabber they use for news releases uses. Too bad they don't have something like Froogle where you can directly submit to them.

Posted: Sun Jan 09, 2005 5:50 pm

Colorado State University wrote:

Date & Time: 2005-01-02 13:43:27
Blocked IP: 129.82.47.151
User ID: Anonymous (1)
Reason: Abuse-Harvest
String Match: webcollage
--------------------
User Agent: mywebcollage/1.107
Query String: www.legalizefreedom.us/index.php
Forwarded For: none
Client IP: none
Remote Address: 129.82.47.151
Remote Port: 36596
Request Method: GET

Googlebot wrote:

Date & Time: 2004-12-19 15:37:44
Blocked IP: 66.249.66.238
User ID: Anonymous (1)
Reason: Abuse-Script
--------------------
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Stanford wrote:

Date & Time: 2005-01-02 13:43:30
Blocked IP: 128.12.51.32
User ID: Anonymous (1)
Reason: Abuse-Harvest
String Match: webcollage
--------------------
User Agent: webcollage/1.107
Query String: www.legalizefreedom.us/index.php
Forwarded For: none
Client IP: none
Remote Address: 128.12.51.32
Remote Port: 38909
Request Method: GET

All three of these are networks I had asked to browse through my site. Now, perhaps the Colorado and Stanford sites were IPs from some student hacker, or something. There are more networks coming through successfully though, so I'm not really sure what the problem is.

I'm not sure what "abuse-harvest" means. Also, I don't know what was wrong with the script it says the Googlebot tried; I'm afraid to post it here, I don't want to be blocked. Very Happy

registered · Posted: Mon Jan 10, 2005 9:53 am

"Abuse-harvest" is a spider that scans your sites. 99% of the ones blocked by Sentinel are spam bots looking for email addresses.

You're blocking for something called "webcollege." I've never heard of it. Here's a link to info about it. *shrug* I'll let you deside if you want to unblock it or not.

http://www.pgts.com.au/cgi-bin/psql?robot_info=4656

-drmike

Posted: Mon Jan 10, 2005 10:21 am

Thanks doc. I checked it out. BTW, not to be critical, but it's "collage" - like putting a lod of pictures together into one giant pic. I ran a search for webcollage/1.107 and found that this is some sort of program, gather images from various sites and makes a collage or screen saver from the collage.

I guess as long as they aren't hot-linking, they can copy images. I didn't make any of them anyway...

Posted: Mon Jan 10, 2005 10:48 am

You can narrow it down using the Agent Inspector:
http://www.ravenphpscripts.com/modules.php?name=Agent_Inspector

Posted: Mon Jan 10, 2005 5:03 pm

Racine wrote:

Thanks doc. I checked it out. BTW, not to be critical, but it's "collage" - like putting a lod of pictures together into one giant pic.

Sorry for the typo. It's kind of hard to type when you only have 35% use in your arms. Smile

That's actually what I searched for when i went looking for it. I just mistyped.

If your ok with graphic searches, IIRC Sentinel blocks Google-Image I believe. I don't know if you want to remove that block. I used to get a lot of hits from that. (I have nearly 2k screen captures on my site)

Thanks,
-drmike

Posted: Mon Jan 10, 2005 10:05 pm

Oh, well, I had always read it college, until I went to search for it and actually looked at the bloody word. Like I said, I wasn't meaning to be critical, and mis-spelt something in my own post (I typed lod instead of load ... doh!) and you quoted it!! LOL!

Okay, thanks for the data and inspiring me to research this stuff more.