Robots meta tag

Regular Joined: Jun 09, 2004 Posts: 79

I have a site I don't want in the search engines. Now that meta.php is gone, where do I edit the robots and revisit meta tags?

Thanks.

Posted: Wed Oct 21, 2009 1:54 am

I think kguske is the only one who can answer that one authoritatively as I'm not seeing anywhere that those 'default' settings can be overwritten - they maybe a way but I'm just not seeing it at the moment.
You can try setting them in the nuke_seo_dh_master table directly but not all spiders obey those meta tags or even read them, you probably have more chance just editing the robots.txt file which all good spiders should read and obey.

registered · Site Admin Joined: Jun 04, 2004 Posts: 6437

Actually, it would need to exist in both the nuke_seo_dh_master and nuke_seo_dh tables. The master table defines the META tag, but the values are defined in nuke_seo_dh.

For example, the default installation has (META tag ID) mid=23 for the REVISIT tag. In nuke_seo_dh, this exists as:

dhid = 12
levelsort = 0 (site)
title = (null)
id = 0
mid = 23
lang = (null)
active = 1
metavalue = '1 DAY'

Just change the metavalue in the _dh table for the mid that corresponds to the desired tag, and you're good.

If you need to add a meta tag, that needs to be done in both tables.

Eventually, we'll have an admin tool that lets you edit that online, but that is too much error checking, etc. for the first release.

Posted: Fri Oct 23, 2009 12:31 am

Thanks, kguske, I'll try that.

Just excluding in the robots.txt file isn't good enough when it comes to Yahoo. I tried this once before, and my pages got indexed by Yahoo anyway. When I contacted them, they told me every page has to have a robots tag of "noindex" in order to be excluded. (So, they don't always check robots.txt.)

Posted: Fri Oct 23, 2009 2:24 am

The problem with Yahoo is that they allow third parties to use their bots. These can be identified with the useragent xxxx.inktomi.com
Where xxxx is a series of random letters.