Hi guys! Welcome to part 2 of my Digital Marketing for Beginners series. In part 1 we looked at Domain Name Considerations that you need to think about when choosing the web address for your site, so by now you should have a rough idea of what your web address (domain) is going to be. The next step is to build your website, which is a task in and of itself, and something I could write an entire series of posts on as well… I might do that at a later date, actually.

But anyway, once you have your website and you’ve filled it with content or products, one thing that you really need to understand it what “indexation” by search engines (specifically Google, because no one uses Yahoo anymore) and how it can affect your site’s performance. A lack of understanding of indexation can lead to you missing some major technical SEO issues on the website, in turn potentially leading to lower traffic and profitability… So yeah, it’s a pretty big thing, and it can take time to get your head around.

However, because it’s such an important aspect of getting your website right, I would still class it as something every beginner in Digital Marketing should at least have an understanding about.

So, What The Bloody Hell Is Indexation?

Indexation refers to how Google is able to crawl a website, understand the pages and add them to its database of web pages that fuel search results. As such, it is imperative that indexation errors are rectified as quickly as possible to ensure that all pages that should be indexed have been.

If you think of Google as a young child trying to find its way around a massive bookstore, attempting to understand and catalogue everything, you can see that the child will need all the help they can get!

[ctt template=”5″ link=”60t4v” via=”yes” ]Think of #Google as a child trying to find its way around a bookstore, attempting to catalogue everything. #DigitalMarketing @marketing_punk[/ctt]

Effectively, Google has come to your website (the bookstore) with no idea as to what you’re actually trying to sell. It has then been giving the task of reading through everything on your website, learning what it is about and what sort of people would be interested in it, based on the relevance of the content and the topic of the page.

That’s a pretty tall order for anyone, or anything, to do. And if your site isn’t structured correctly, then you’re just making it more difficult!

SEMrush
The thing to remember here is that Google’s crawlers really do act like children in some respects. If you make a site too difficult to crawl and index, then after a while, the crawlers will just give up! Because, let’s face it, it makes no real difference to them whether your site gets indexed. Therefore a millions, if not billions, of websites out there, so there’s bound to be another few hundred that give the same sort of content as yours.

Even if Google doesn’t index your site, it will find another one of good quality to show to its customers (the people searching). The only person who stands to lose out from this is you, the website owner!

So, it really is in your best interest to make it as quick and easy for Google to get to every page it needs to. Of course, this can be tricky, and a variety of factors can affect things.

 

For example, I have seen an ecommerce website that only had 30,000 products in its inventory, with each product supposed to have its own page. When you add categories and landing pages into that, the site was supposed to have around 30,500 pages. Now, this is still a huge site that would take Google a long time to get through, but when looking at the amount of pages that had been indexed over the course of a year, I found that only 5,000 were crawled.

That is a massive loss in visibility for that particular site, as only 16% of the pages that were supposed to be there had been indexed. Why?

Because there were actually 2.5 million individual URLs on that website! Due to technical errors in the database structure of the site, and in the product upload system, whenever a new product was uploaded to the website, it created new product number URLs for every product on that site!

So, by looking at the indexation of the site, I was able to find the source of a much larger problem, and the reason that the website had never taken off in terms of Organic Search or revenue generation.

Now I can almost hear you asking how to check indexation, and it’s actually really easy to do once you know!

How Do You Find Indexation Problems?

To begin with, you need to find out how many pages of your website appear in Google’s Search Results. To do this, you need to use one of the little search tricks available for Google, called a Site Search. Effectively, you search for your domain (without the www, unless you want to ignore subdomains) with site: in front of like, like this.

site:example.com

This tells Google that you want it to show all pages it has in its search results for that specific website. It will then show you the results of this site search with the pages ranked by relevance and important, at least in Google’s eyes (but we’ll get onto that later).

[ctt template=”5″ link=”1q98z” via=”yes” ]Use a site:domain search in #Google to get an idea of how many pages of your site have been indexed! #DigitalMarketing @marketing_punk[/ctt]

For now, the important part is to see the total number of pages that the search returned. You can see this written in a very small font, just above the first search result (or PPC advert if there are any). In the following example (fig.1), there are “about 2,870,000” pages indexed for the BBC.

Now, the BBC itself is an absolutely massive website, which is why it is a rough estimate number. For smaller, more normal sized websites, it will actually give you a specific number. For example, Fig.2 shows the site search for That Marketing Punk as of the end of November 2016.

As you can see, whilst it still says “about”, the number itself is accurate due to the fact that the website is much (much) smaller. Getting this number is the first part of seeing if there is an indexation problem.

The second part involves logging into Google Search Console (or Webmaster Tools as I still, lovingly, call it) and going to Index Status, which is found under the Google Index section. Here, you can see that total amount of indexed pages that Search Console has placed into Google’s database.

If the numbers are vastly different, that would be your first warning sign that there is an issue with indexation, and you should move on to the next step (but only if you have at least 3 months of data in Google Analytics)

SEMrush
Basically, go to Google Analytics’ Landing Pages section, found within Behaviour > Site Content, and set the date range as large as you can to see all of the data, and set the Segment to Organic users only.

This will show you how many pages were landed on from Google Search Results over the set period of time. Specifically, you want to look at the bottom right where it says something like “1 – 10 of 610”. In that example, the total number of pages that were landed on would be 610.

If this number is widely different that those found in the site search and in Search Console, then it’s time to start investigating why!

[ctt template=”5″ link=”0P9aw” via=”yes” ]Compare a site:domain search, pages in #Google Analytics and indexed pages in Search Console to find indexation issues @marketing_punk[/ctt]

If the number shown in Google Analytics is vastly lower, it shows that Google is not valuing your pages, which is likely an issue with either the content or site architecture. On the other hand, if the number is vastly higher, then it means that Google has indexed pages that no longer exist and are being redirected.

There are, of course, other possibilities (like the product upload issue I mentioned earlier) but these are good starting places to begin your investigation!

And That’s A Basic Overview

If you’ve got through all of that, well done! There’s quite a bit to get your head around, but if you can, you’ve got yourself a basic understanding of indexation – enough to start looking for issues and researching how to solve them! I hope you’ve enjoyed this post and found it useful! If you have any questions or things you want to share. feel free to leave a comment!

Liked this post? Want your say on what games I review or feature? Take control of the future of 16-Bit Dad by supporting on Patreon!