SEO FAQ 4

How do I write the robots.txt?

As you may, know the robots.txt is a tool used by the search engines to see which pages not to index. It is useful if you have sensitive information or other pages you don't wish to get indexed - that is all it does.

It is simply a text file names robots.txt located in the root.

It contains a line for user-agent and then what to disallow or allow.

Example:

User-agent: *
Disallow: /password.html
Disallow: /temp/

As the * is a wildcard the above means that all user-agents will index all pages except the file password.html and the directory temp and it's content.

Here is another example when disallowing all from a bad spider.

User-agent: badspider
Disallow: /

Does a site keep its PR when ownership changes?

The good thing about PageRank is that it is blind. Unless something inappropriate is going on with an individual, Google does not care about the humans who own/buy/sell websites. They focus entirely on the domain name and the website behind it. Changes in ownership are inconsequential to Google.

Pagerank rises and falls entirely on the merits of the website and Goggle's algorithms. The risks of PageRank falling or rising is purely technical.

If anything, Google is very forgiving of websites that go inactive. There is a huge aftermarket of entrepreneurs seeking inactive domains with PageRank attached for its reaale value. Google often times is slow to identify inactive domains. This is mostly due to the fact that just because a PageRanked website goes inactive, it's links can remain active for years.

Last year I purchased an inactive domain name with a PageRank of 6 with several hundred links showing up in Google/Yahoo/MSN. Six months later, its PageRank dropped to a 4. It all honesty, it really should have been a 4. I just got to enjoy a PageRank of 6 for a few months. What this tells us is that Google is going to sooner or later correct a website's PageRank. But Google isn't the bad guy here, they are just applying the rules as strictly as they can with what they know. Google doesn't really worry about buying and selling because they know that the technical factors will keep everything even in the long run.

Purchasing a website in itself is no more risky than having owned it for years. A potential buyer needs to understand what the websites SEO strategy has been and how it gets its traffic. If the former owner spent a fortune purchasing traffic and the new buyer isn't planning on pursuing the same kind of traffic then it isn't unreasonable to expect the performance of the website to degrade. If the former owner was engaged in "black hat" or other inappropriate activites in creating traffic and search engine visibility and Google catches on, then the website will suffer along with whoever might own it. When purchasing anything of value, (including websites) the buyer must be diligent to protect themselves

What are all the Google Operators and what do they do?

Trust level: Proven and confirmed

The advanced operators, as far as SEO is concerned can be divided into 2 categories: Alternate Query Types and Query Modifiers.

Alternate Query Types

cache: If you include other words in the query, Google will highlight those words within the cached document. For instance, [cache:www.google.com web] will show the cached content with the word "web" highlighted.

link: The query [link:] will list webpages that have links to the specified webpage. For instance, [link:www.google.com] will list webpages that have links pointing to the Google homepage. Note there can be no space between the "link:" and the web page url.

related: The query [related:] will list web pages that are "similar" to a specified web page. For instance, [related:www.google.com] will list web pages that are similar to the Google homepage. Note there can be no space between the "related:" and the web page url.

info: The query [info:] will present some information that Google has about that web page. For instance, [info:www.google.com] will show information about the Google homepage. Note there can be no space between the "info:" and the web page url.

This functionality is also accessible by typing the web page url directly into a Google search box.

Query Modifiers

site: If you include [site:] in your query, Google will restrict the results to those websites in the given domain. For instance, [help site:www.google.com] will find pages about help within www.google.com. [help site:com] will find pages about help within .com urls. Note there can be no space between the "site:" and the domain.

allintitle: If you start a query with [allintitle:], Google will restrict the results to those with all of the query words in the title. For instance, [allintitle: google search] will return only documents that have both "google" and "search" in the title.

intitle: If you include [intitle:] in your query, Google will restrict the results to documents containing that word in the title. For instance, [intitle:google search] will return documents that mention the word "google" in their title, and mention the word "search" anywhere in the document (title or no). Note there can be no space between the "intitle:" and the following word.

Putting [intitle:] in front of every word in your query is equivalent to putting [allintitle:] at the front of your query: [intitle:google intitle:search] is the same as [allintitle: google search].

allinurl: If you start a query with [allinurl:], Google will restrict the results to those with all of the query words in the url. For instance, [allinurl: google search] will return only documents that have both "google" and "search" in the url.

Note that [allinurl:] works on words, not url components. In particular, it ignores punctuation. Thus, [allinurl: foo/bar] will restrict the results to page with the words "foo" and "bar" in the url, but won't require that they be separated by a slash within that url, that they be adjacent, or that they be in that particular word order. There is currently no way to enforce these constraints.

inurl: If you include [inurl:] in your query, Google will restrict the results to documents containing that word in the url. For instance, [inurl:google search] will return documents that mention the word "google" in their url, and mention the word "search" anywhere in the document (url or no). Note there can be no space between the "inurl:" and the following word.

Putting "inurl:" in front of every word in your query is equivalent to putting "allinurl:" at the front of your query: [inurl:google inurl:search] is the same as [allinurl: google search]