It’s not a perfect solution because there is a bit of content on the themed seattleweekly page that isn’t on the corresponding Trulia page and vice versa, but if Google’s purpose is to understand the structure of a set of pages, having the Trulia system is the closest thing that can be done to showing Googlebot the real pages (again, probably incurring a dupe penalty). I would think that this is how Google would want you to do it if you asked them.

Can you please explain why a conditional redirect is superior to a noindex/nfollow?

The biggest issue has to do with the urls staying in the database. So when Google collects urls to a page as it crawls, but then can’t index that page because of robots.txt or noindex, it doesn’t flush the urls it finds. If there are enough links pointing to those non indexed urls, they will even on occasion start returning them as search results.

Using a 301 fixes all of that. The urls get purged, and Google is better able to understand what the original source is. And it isn’t just something that happens in relation to syndicated content. The same applies to internal structures where very similar content can be viewed from multiple urls. (Things like product sort functions). You could add nofollow to those links, or use wildcard robots.txt, but that isn’t the best approach for purging crap urls or strengthening internal link structures. The better aspproach is to pick one master page for Google and then 301 al the variations to that page. Since those pages do have value for a human visitor, you have to make the 301 conditional.

Here’s another quick example:

Client has to move to a new domain. The current site is and has been indexed in G for quite some time. The new site on the new domain is functional for humans, but not quite ready to be merged, so no index/follow is applied to everything on the new domain because they can’t postpone the press release. The release hits the wires and the new domain gets a lot of press (and links) But Google can’t index the pages people are linking to, so it just hangs on to a bunch of urls with no title and description. Now, all of a sudden, the original site vanishes for their brand name, and the SERPs end up listing the urls that aren’t supposed to be indexed instead. So now their brand name serp looks like shit and they are losing traffic because they “followed the rules” and left everything up to Google to figure out.

Doing that is the kiss of death. The less decision making you leave up to Google the better off you will be. In this particular case a conditional 301 would have been a much better solution, but the client was too afraid to take the proper steps to have the smoothest transition possible give the set of circumstances, because they’ve spent too much time listening to all the “evil cloaking” bullshit being published by people who aren’t even close to be qualified to know what is or isn’t cloaking, or more importantly, how Google might feel/react to it. (Again Eric, I’m being general here, not directing that at you personally).

To me, that’s sad. Large scale projects like that are probably 90% of the kind of work I do. And when you spend that much time doing that kind of work, you constantly see examples of how Google doesn’t always work like the say it should, or even worse, are completely incapable of figuring out things they claim are easy. So to me, making decisions about what might be the best way to address a particular problem based completely on FUD is just a rediculous thing to do.

That cred is why Trulia is kicking your ass, and they will continue to kick your ass

It’s pretty arguable whether or not Trulia is kicking my ass in the SERPs. Kicking the general real estate industry’s ass in the SERPs? Maybe.

If you honestly think noindex/nofollow/robots.txt is the golden ticket, you have never dealt with a project of that magnitude.

Absolutely. Can you please explain why a conditional redirect is superior to a noindex/nfollow?

I also agree that they did this for the SEO - there really isn’t any other plausible explanation. Now for the completely unbelievable part- you out Trulia on this tactic- others outed them for invisible text (how lame…) and the nofollows and yet agents are still facilitating their own SERP demise by adding content to Trulia communities and putting links and widgets on their pages back to Trulia. It’s hard for me to feel sorry for people crying poor mouth over Trulia jumping above them in SERPs when they are part of the cause. It’s pretty cut and dry for me- if you don’t want som