So, you have a website and you’re thinking why you’re not getting the result that you wanted from the big “G”. You can scour the entire websphere to find tips on how to go about it or you can read SEOChat’s Basic SEO Troubleshooting. It’s a little long but here’s the gist of the entire article.
Sure, Google’s not the only search engine there is but remember that it is the leading search engine that is being used these days (it’s a little old, but you get the idea) and until it is toppled by another search engine (which is unlikely in the near future) we have no other choice but to follow them.
The search engine just recently updated these guidelines to be clearer and include more information. If you scroll down to “Quality guidelines – specific guidelines,” you’ll see that many of the bulleted points now contain hyperlinks that take you to more information about specific issues, such as hidden text. Take your time with these to make sure you fully understand the guidelines.
After all’s that is done, all you have to do is maintain your site properly and update your site with fresh content as much as possible.
A robots.txt file is a good thing; it tells the search engine spiders whether or not to crawl a particular page. That can be important if you have certain content set up to be seen by subscribers only. But if your robots.txt file is set up wrong, the spiders could be avoiding web pages you actually want them to see, thus preventing the pages from being indexed.
Likewise, a search engine spider can’t follow a broken link on your site. Neither can a human visitor. Make sure all of your links work perfectly.
Do you have any pages without content? While a site is always “under construction,” you never know when the search engine spiders will be paying a visit to index your site. You want to show them your best face. Keep those oddball pages to a minimum.
How good is your site navigation? It doesn’t have to be fancy, but for the sake of both your human visitors and the search engines, it should be consistent, with general categories leading to more specific topics within the categories.
Speaking of content, you’ll also want to check for duplicate content. You need to find out whether someone else is duplicating your site’s content (in which case Google, who can’t really tell which site was there first, might be penalizing you by mistake). You also need to find out whether you have duplicate content on your own site – whether you’re duplicating someone else’s content, and whether some of your pages are so similar that Google sees the two pages as identical, and chooses to index just one. There are a variety of tools you can use to check this; just Google “duplicate content check.” Or you can just Google some key phrases from the content that you think has been copied, and see what comes up.
Sure flash, javascript and ajax adds all the pizzaz a site could ever want however, it limits the pages that the robots could crawl and it may deem devastating to you as a site owner. Likewise, all pages in a site must be linked to each other because unless you do that, search engines will not be able to index the entirety of your site.
You can make Google’s work even easier by submitting a sitemap. Do it in XML, and keep it up to date. You can find the details for how to do this here; the page lists the protocol and explains how to submit your sitemap. Keep in mind that your sitemap should not be larger than 100 links. If it is larger than 100 links, you will have to break it up into more than one page – and you can include another page on your sitemap that links to both of those pages, so Google can keep crawling. Incidentally, if possible, you really should have less than 100 links on each page of your web site as well.
Speaking of your site’s structure, you might want to take another look at it, especially if you’re not using a template. Cookie cutter templates may look boring, but GaryTheScubaGuy (aka Gary Beal) cited one possible reason to at least create and use your own unique template: it may help keep Google from seeing very little or duplicate content on your site and then backing out. He notes that this is rare, but he has seen it when a novice builds a site one page at a time with no template. “In a correctly built site most robots will parse the template and crawl the content and see unique content. This allows them to crawl deeper and faster,” he explained.
Make sure that all of the links that redirect to your homepage is your “official one”. There’s no sense in using http://domain.com when you’ve linked to your site as http://www.domain.com when you were link building it before.
It’s not like the crawlers will be turned off if your site is too slow just like us humans, it’s just that it slows the crawlers down in indexing your site. On a related note, you should also be aware of how fast or slow the server is delivering your site because no matter how optimized your site is, if your server is lagging then your site will not be indexed as fast as you want.