I recently launched my new web design website that consisted of a WordPress blog and 8-10 static, optimized, unique pages with 100% valid handwritten code.
As always as soon as I put the site live I did the usual; added the site to webmaster tools (not submitting the site), built and submitted a sitemap and tweeted some of my core pages to create a buzz and generate some social traffic. Additionally I asked Google to display my URLs without the “www.” prefix as I had a .htaccess rule to remove it from my URLs.
Within 24 hours my /blog/ directory and one handwritten page (home page) had indexed. I was a little surprised as my blog was probably the least internally linked content from my homepage, there were plenty of sales like pages that should have been crawled and indexed at the same time if not before my blog.
2 weeks later the situation remained. I started to doubt my code, I felt as though I had missed something obvious. I checked for no-index tags in my content, there were none. I rechecked my robots.txt file, it validated and “Fetch as Google-bot” tool confirmed my pages were crawlable. I the however looked at the crawl errors list and the handwritten pages (apart from my homepage) were noted as returning a 404, however if I visited those pages directly they appeared fine.
The further strange issue was the thumbnail in my webmaster tools control panel for the website showed what appeared to be a black screen with white text, possibly an Apache server error?
In the end I resulted to using the Crawl URL Tool in Google webmasters, the next day the 3 pages I requested to be indexed were indexed and the black/white thumbnail had changed to my homepage thumbnail.
It is obvious why the index page had indexed, what was strange was that all the handwritten pages linked from the homepage returned false “404′s” and my blog was A-okay.
I can’t be that WordPress built pages get higher priority over other pages because my handwritten pages had crawl attempts upon them, they just apparently didn’t exist.
What on earth happened? Any ideas welcome.