So you’re having problems indexing your website? Don’t worry. Everyone’s been through this at one point or another. And the solution is actually really, really simple as you will see in a minute. Even for newer websites (1 – 2 months old), the techniques that I’m about to share with you, will help index new blog posts and pages in a matter of hours – or even minutes.
What You’ll Learn
- What does it mean when a search engine crawls and indexes a page – what happens on the back-end and why this process is so important for SEO.
- Vital factors that influence the speed of indexing – why do some pages get indexed faster than others?
- The ultimate list of tactics that will get your blog posts indexed phenomenally fast – depending on the site’s age and authority, this can vary from a few hours to just a few minutes.
- Secret WordPress tricks to index your content even faster – for bloggers using the WordPress platform, this section is an absolute must.
Understanding Crawling & Indexing
Most people in the SEO world are already aware of that so if you are one of those people, simply skip this section and go to the next one. But, if you have no idea what search engines do and what indexing is, pay close attention.
Basically, when you publish a new blog post or any page for that matter, in order for it to appear in the SERPs of Google and other search engines, it has to get indexed by their respective crawlers. That’s basically all there is to it. Now, I can talk about that process which will take a few sentences to understand, or, I can just show you an image (courtesy of HowStuffWorks) that explains it all:
So the web spider a.k.a. the search engine crawler is lurking around the Internet looking for new and updated pages to crawl by hopping from one backlink to another. Once it finds something it hasn’t indexed yet, it scans it from top to bottom, takes in keywords and puts the new/updated data into the database of the search engine where it is now considered indexed. Why?
Because the SERPs that we all so want to rank high in are built taking in data from those databases that the search engine crawlers fill with information. And that is why it is extremely important that your content gets indexed as fast as possible.
Now that we have cleared that out of the way, let’s take a look at some of the most important factors that influence search engine indexing.
Vital Factors That Influence The Speed Of Indexing
Right from the start, I just want to say that indexing is not an exact science. You can’t apply the same methods to different sites, or even different pages and expect the same results. No, it just doesn’t work that way.
Sites and pages vary from one another in important metrics that are taken into account when the web crawler is doing its thing. Now, a very important page I urge everyone to read from top to bottom is this one from Google Webmaster Central.
It has everything you will ever need to understand crawling, indexing and even ranking including information on how to do it effectively. This of course concerns only the Google search engine and you should keep in mind that other search engines act differently although in a similar fashion. It’s just the metrics that are taken into account that vary from search engine to search engine.
Okay, now that we’ve established the fact that indexing is not a constant but rather a variable and that it mainly depends on how often a website is crawled by the search engine’s spider, let’s take a look at some of the most important factors that influence web page indexing:
- Site authority – we’re not talking about Moz metrics here, but rather the authority of a website in the eyes of the search engine. If a website has a lot of inbound links from other respectable websites, it is bound to be crawled much more often by the web spider thus making it easier and faster for its new and updated pages to be indexed. On the other hand, newer and less authoritative websites are crawled less often leading to indexing times in the range of a few days to a few months.
- Site age – basically, the older the website the more often it will get crawled. This is of course considering the fact that the website is still active and pushing out new content.
- Traffic – websites with more traffic tend to get crawled more often as it makes it easier for the search engine crawlers to find new content on more active sites.
- The amount of recent links to a certain page – for example, new blog posts tend to get indexed faster if they are shared across social media and are linked to by other websites. After all, the search engine crawler follows links so the more links point at a certain page, the higher the chance for the crawler to find it and index it.
- High quality content – contrary to popular belief, not all pages that are crawled get indexed (exhibit A). And more often that not, the deal breaker is poor content. Google and other search engines are striving for high quality so don’t be surprised when your spun-looking content doesn’t get indexed even if you’ve submitted a sitemap.
Those five are basically the most vital factors when it comes to indexation. Now, keep in mind that I haven’t ranked them in importance (from top to bottom) and that is because there isn’t a most important factor. They all contemplate to create a website that has high index rates.
But to sum it all up, it basically comes down to being respected by the search engine spider. As we saw, the more often a crawler visits your website, the faster your pages will get indexed. Your main job is to make sure the crawler visits a high quality page.
Of course, that doesn’t mean sitting around waiting for this to happen. No no. In the next section, we will list all of the methods and tactics that you can use to make sure phenomenally fast index times and rates – even if your website is completely brand new.
The Ultimate List Of Indexation Tactics & Methods Which Will Bring In Phenomenal Results
Okay. Now that you know how important it is to index your content, how it works, and what website metrics are taken into account, it’s time to take advantage of a few tactics and methods that are tried and tested. Just keep in mind that not all apply for every single website.
For instance, you don’t want to register your PBN sites (how to setup PBN safely ultimate tutorial) on Google Search Console. However, you still want to index the content of those sites as well. It’s pretty intuitive which method applies to which type of website so I won’t be clarifying that any further. Now onto the list:
- Register your website on Google Search Console and submit your sitemap – that is probably one of the most effective methods to ensure quick indexation of your content.
- Create an RSS feed – just another easy way to tell search engine crawlers that your website has something new on it that requires attention – more info on RSS feeds here.
- Implement Google Analytics – this will not only help you with index rates and speed, but also show you very important data about your website and its visitors, which you can use to further grow.
- Blog – search engines (especially Google) love websites that have blogs. Blogging means new content regularly which means Google has more information to distribute to its users. So, having a regularly updated blog on your website is an absolute must nowadays as it will not only help you with indexation (exhibit A), but also make you a lot of passive income if done right.
- Share your content on social media – for this one, and I believe this is one of the main reasons why our content started getting indexed really fast even though this website was brand new (a couple of weeks old), I used IFTTT. For those unaware, just check out the tool, you won’t regret it. We basically use it to distribute each and every new blog post to the following social media sites, automatically of course:
- Get other websites to link to your pages and blog posts – we use this method and it works like a charm. However, you can also simply reach out to relevant websites and influencers and talk them into checking out, and ultimately, linking back to your website.
- High quality content – this is the one thing that will help you the most in the beginning. New websites don’t have authority in the eyes of search engines and one of the fastest ways to earn that authority is through high quality content. This website used to rank, in the beginning, solely on good content and internal backlinks + the IFTTT thing. And speaking of internal backlinks…
- Internal backlinks – usually, every website has a few pages that rank very well on search engines. So, when you create new content that is relevant to those older well-ranking pages, simply go back and edit them to link to the new content. It’s an easy win-win situation.
- Share your content on relevant forums – if your niche happens to have some very popular communities and forums (not all niches do), go there and share your content. Usually people frown upon self-promotion however, if your content is good as we talked about in point 7 of this list, then you will get a positive reaction and result. That’s at least in my experience.
- Guest posting – creating articles for other relevant sites in your niche in exchange for a contextual link back to your website is another great way to ensure high authority in the eyes of search engines the direct results of which is, I don’t know how many times I’ve said this thus far, faster indexation.
Those are your must-follow 10 commandments when it comes to the indexation of your website. It doesn’t matter if it is old and established or brand new and looking for expansion. Simply employ all 10 of the above indexation methods and you might still have 99 problems but indexing won’t be one.
And now that we’ve wrapped up the general indexation methods that work for any website, I have a few additional methods that will even further help with indexation specifically if you are using the WordPress platform.
Secret WordPress Tricks To Index Your Content Even Faster
If you are using WordPress and you want to optimize your indexation to the absolute maximum, the the below list will help you achieve exactly that. Check it:
- Google XML Sitemaps – this plugin generates a specific sitemap that helps all search engines (not only Google) index your website much faster. The sitemap makes it easier for crawlers to scan the complete structure of your website and better see what pages and content there is.
- NextScripts: Social Networks Auto-Poster – while IFTTT will take care of most of the social media sites for you, it cannot post to some of them which is where this plugin comes in. In addition to all of the sites that IFTTT posts our new blog posts to, we use this plugin to post to LiveJournal, Plurk, and StumbleUpon.
- Yoast SEO – this plugin really helps a lot with metadata required when sharing your content on social media. Plus, it also helps with various SEO-related things that indirectly influence indexation such as custom meta description for your blog posts, on-page analysis (although not the best), RSS optimization, etc.
And those are basically the tricks we use to further boost the indexation of our WordPress sites. They certainly have helped us a lot so I’m pretty sure they will help you as well.
While it still is a fact that high quality content and authority backlinks are aces when it comes to SEO, you still want to keep in mind indexation rates and speed. Indexing your content shouldn’t be your number one priority, however, it doesn’t have to be.
If you simply follow all of the methods and tactics we walked through in this article, it becomes an automatic process. You won’t have to do absolutely anything besides writing high quality content and reaching out to people for good backlinks in order to index your blog posts and pages phenomenally fast.