According to SEO guru Matt Cutts (he works at google and is quite renowned for helping us web devs out) a promising new link element that is now being supported by major SEO companies (all we really care about these days is google, isn’t it? I’m sure that’ll change someday. Maybe.) called the canonical link.
This link tag is used to specify how search engines (not to people, although who knows how a user agent might use it in the future) should treat those weird, argument filled URLs that get generated by scripts (like phpbb or this blog) that end in something like mydomain.com/forum/viewtopic.php?t=343&f=2 or maybe a session ID at the end of the link like ?SID=3429vhr32if298fg.
For a long time now we’ve been forced to modify our software in all kinds of complicated ways to stop search engines from treating these pages as separate pages when they are often the same page accessed with a different argument.
No longer! Well, not so much anyway. We’ll still need to modify software if we want search friendly URLs. And, actually, google seems to be pretty smart when it comes to detecting duplicate content as a result of variables appended to URLs. Still, I would choose this option over heavy modifications… Actually the more I think about it, the more I realize this is far from a magic bullet… Oh well.
By specifying a canonical link element to a page, crawlers will ignore all that junk at the end of the URL and only index the one, canonical link. It’s also called “normalizing” your URLs. How to do it? Well where you actually implement the code on your software may vary (doing it for phpbb3 culd be tricky if you want all the pages to still get indexed) but the code itself is simple as pie.
<link rel="canonical" href="http://example.com/page.html"/>
Done. Now they know the page to index is page.html and not page.html?doodle=234975ai398r&biddle=34y
Sweet. Oh, and you can use it to deal with that pesky www subdomain issue. Don’t want google treating www.yourdomain.com and yourdomain.com as different URLs? Just normalize it as above. Woohoo!
Incidentally, the google webmaster tools also has a relatively new option in there to specify a preferred subdomain. I would think these two tools go hand in hand quite nicely.
Just go read his blog entry on it if you need more info. It’s got a cute slideshow that breaks it down for ya.