There are literally thousands upon thousands of blogs on the internet. There are blogs about every niche imaginable. As such, there is a great deal of competition out there. Bloggers all have a variety of techniques they use to obtain traffic. However, most people agree that nothing beats organic, natural, search engine traffic.
Blogs and web pages are picked up regularly by Google and ranked in order of relevance. Google crawls some blogs more quickly than others, depending on relevance, backlinks, etc. The pages that are indexed quickly referred to as “authority” pages. If you want your blog to be indexed quickly, you need to increase its relevancy and value.
Here are a few tips for optimizing your blog so that Google will crawl it more often:
- Publish new unique blog posts as aften as possible. At least twice a week. Web pages with duplicate content are indexed very slowly, if at all. The content should be informative, beneficial to your reader, have some targeted keywords (a keyword ratio of under 2%), and consist of no “fluff” or “padding”.
- Optimize even the images on your blog. Don’t just label them as “pic1.jpg” or “pic2.gif”, describe what they are. You should also name each image as an important keyword. Read this post on the best way to optimize your images: How To Optimize Your Images for Improved Search Engine Optimization
- Make sure your posts are internally interlinked (i.e. link from one blog post to another). The Insights plugin makes this task easier. And ensure you use the correct anchor text to link from one blog post to another.
- Turn on the trackback and pingback options for every single post. Many people don’t realize the importance of this, especially in regards to a Google crawler. With these options on, you will be directing a Googlebot to crawl and find your new posts. Use the MaxBlog Ping Optimizer plugin.
You can see exactly how to set up the plugin here: Increase blog traffic
- Try getting as many quality backlinks from relevant sites as possible. Two ways to do this is via commenting and publishing guest posts on relevant sites
- Keep your URLs simple, yet effective. Long, complicated URLs take longer to find and index. You need to make them easy to remember and to the point.
- There is some dispute as to whether black hat SEO methods are worthwhile. While using such methods probably will get your blog indexed quickly, they’re extremely risky and your blog can end up being banned. It’s therefore safer to use white hat methods (mentioned above), as they will ensure that your blog will be indexed more quickly over time.
In summary, you can get Google to crawl your blog more quickly by making it as search engine friendly as possible. This means all of your URLs need to be organized, your content needs to be unique, the keywords need to fit in naturally and not overstuffed, etc. Google loves authority websites, so try to make yours as relevant as possible.
What tips do you have to make Google crawl your blog more often? Share them in the comments below.
_______________________________________________________________________________________
Thanks! Im going to be using these tips from now on
best described nice post and informative post
can anyone tell write in robot.txt
Disallow:/tag/
Disallow:/category/
can block my site for crawling by google
Thank you for these tips. I have updated the meta tags on my blog and waiting for the crawler to re-index it again. I will surely try to implement your methods.
I use google webmaster tool to make search engine index my latest post. It is like, fetch your latest post as googlebot and request to index it.
Great info in this post! I agree that quality baclinks together with quality content posted in regular basis gets you content ranked!
Thanks for the post. I was worried because my site is being crawled less often. However, now I realize that instead of focusing on crawl rate, I should rather focus on creating good content.
Akash
Great (not good) content is what makes the difference.
Andrew
I’m having a problem, pls help me..
I use wordpress cms & from starting day it work very well..
all my blog post was getting indexed in google with in 12 hours & sometimes with in 5 to 10 mins.
But now I noticed that my last 3 post didn’t got indexed in google , it have been 5 days. I m waiting for that post to get indexed, but no success…
I resubmitted sitemap , still no use…
Plz help…
Abhijit
Have you followed all the ideas in this post?
Andrew
yes I did still no use…
then I tried one more thing that is google url fetch & it worked..
so if you want, then edit your awesome post & add that google fetch method tooo..
it will help other fellas like me. 🙂
Abhijit
What is google url fetch ?
Andrew
Nice dude. Thanks that really clears a lot of things up my page has been ranked zero forever.
Thanks for posting this. The article and subsequent conversation via the comments is awesome.
I’m curious about a piece of this. I’ve recently been doing some SEO clean up on my posts using a WordPress plug in and it’s really helped me if nothing else to be aware of simple key things that often get missed. There’s a problem though. Since I changed titles, slugs, added pictures, etc… my posts are not showing up in Google. For example, if I copy the new URL that is on my site for a given article into Google search, the article doesn’t come up. What am I doing wrong? Or is this just a matter of Google catching up. I did the latest round of tweaks about a day ago. Will this eventually update or do I have something jacked up in some setting somewhere?
Thanks!
Gary
It can take a Google a few days to re-crawl your site and re-index all your edited posts.
If you changed the slug – did you add in 301 redirects so the old links are redirected to the new links?
Andrew
I used to get my posts appear on Google within 10 mins of writing them, I stopped working on my site for a bit and now Google hardly crawls my site. It takes days for my new posts to show up in Google nowadays.
Desperate to get Google to crawl my site the way they used to. Maybe these tips will help.
Chris
Hopefully, they will help.
Being consistent in your blog publishing will help.
Andrew
Hi,
Andrew
All the points you mentioned are totaly awesome. I read most of the above comment & found that not only me, many others facing this google indexing problem.
So can you please tell me which is the best google sitemap plugin ???
Well I use Yoast’s SEO plugin which has a in built sitemap included.
Andrew
Hi Andrew.
I believe posting via Google Plus and other social media sites gets your site indexed quicker.
Admittedly I don’t check often, but when I do I find the articles I feature on my blog appear in search the same day they are published.
Social media is the centre of my marketing strategy so reasonable to assume that is a reason?
Mark,
Posting on G+ and other social media sites certainly helps…especially if others do also!
Andrew
I am so pleased there are technical experts out there Andrew. I love running my business and I enjoy blogging lots. As for the technical elements I pass my ball to you!
Thanks for all the useful tips!
Steve Miller
Steve,
More business owners who own websites and blogs should follow your approach. Far too many try and do everything themselves. They should be running and marketing their business…not dealing with technical difficulties.
Andrew
Great article! I am definitely going to use these tips on my blog.
These are some great tips to make Google crawl your site often.
I normally make sure that I publish a post at least twice a week so that it looks fresh all the time. But sometimes it takes me weeks to publish the one post.
Thanks for sharing all these tips with us 🙂
Arbaz,
Regular posting helps as does social media posting…as Mark says above.
Andrew
A little Bible for any blog! 🙂 Going to print it and place on the wall! Thanks for reminding simple but not evident things! 🙂
Thanks, Robert…glad it helps.
I think sometimes we forget the simple things!
Andrew
i believe presence of “stop words” such as (a , an , the) in the permalink also affects crawl rate. also whenever i post a fresh content say an article the crawl rate doesn’t significantly improving. But some times it will be exceptionally high and i don’t know why.Also i’m following the above said things in my blogs and i get a crawl rate of 85/day. please any one say are there any miscellaneous things to increase crawl rate. ALSO don’t forget to mention what is the optimum crawl rate for a site??
As far as i concern i understand many thing from your single post. thank you . I use blogspot and try to focus on quality not quantity but still not in good list of google . try to work hard to crawl google but it left . i thought i had unique posting but they did not work for me . well it was nice to read the good article . Try to gether my energy again and focus what you taught .
Posting quality content, frequently is the sure shot way to get more love from Google and making the Google crawler crawl your website frequently.
Any other “shady” Blackhat SEO tip is not worth taking the risk for all the hard work you are willing to do on your blog/website.
Awesome tips!
Writing unique and quality content everyday makes Google crawl our blog frequently. Link building and internal linking lets Google bot crawl our blog further more frequently.
Hi Andrew,
First of all, let me tell you that it was a well written and useful article. I’ve been implementing some of the suggestions you provided above. But some are new to me. Will start making use of them soon.
Above in the backlinks section you have mentioned that can get backlinks by commenting blogs, but those links always give backlinks with nofollow? Is nofollow backlinks are good for search engines?
Sohail
Some blogs do give ‘follow’ backlinks for comments. But my opinion is comment because you want to comment…not because you want the backlink.
Andrew
Hi
I too agree that “nothing beats organic, natural, search engine traffic”.
Well as far as I know having a regular unique content posting and having quality back-links are the two important practices if done then Google would crawl your blog more often .
Nice post . Thank you for sharing .
Nice and informative post
The best way for Google to index your blog is using proper and basic SEO techniques in a better way. SEO techniques means only white hat SEO methods. Some of them are discussed on the above article like having unique content and posting them on daily basis. Having better internal linking with in the blog.
Having back links from good sites rather than weak sites. All these methods automatically makes your blog indexed by search engines.Another important thing is use of images in blog posts and optimization of those images .
One thing you should remember is never use black hat SEo methods never do over optimization as it could lead to spam.
Thank you for sharing.
Sagar
What you say is exactly true. It’s shame so many people want the quick fix and go down the route of black hat SEO.
Andrew
Hey Andrew,
Nice post and thanks for sharing these effective tips with us. Yes, publishing post consistently is very important and these days image also plays very important role. Interlinking helps in decreasing bounce rate. These tips really gonna help all the newbies.
Well it’s essential to build backlinks and also submit the posts at search engines for easy crawling of blogs. You’ve shared some really informative tips.
Thank
I think the best way to rank well on google is to write good content
Great info in this post! I agree that quality back links together with quality content posted in regular basis gets you content high ranked !
thanks for sharing this wonderful knowledge.
Thanks andrew for such a great insight on web page crawling. I have a blog too and my hosting provider provides me the service to ping every blog post to major search engines and you mentioned this thing too. Thanks so much for this sir.
Here are 7 tips for making Google crawl your blog more often:
Create high-quality content on a regular basis.
Use internal linking to help Google discover new content on your blog.
Use descriptive, keyword-rich URLs for your blog posts.
Use social media to promote your blog and attract more traffic.
Use Google Search Console to monitor your blog’s performance and identify any crawling issues.
Make sure your blog’s design is mobile-friendly and optimized for speed.
Submit a sitemap to Google to help it understand the structure of your blog and crawl it more efficiently.