Improve blog post indexation

Google’s Search Console is useful for uncovering lagging performance in web pages, including for blogs. I tend to write terse posts that address very specific issues. Often, these pages perform well, but I saw a few percent of pages being marked in Search Console as status “crawled - currently not indexed”. The underlying theme on these pages was they had too little ordinary paragraph text. If there are too many lists, headers, or preformatted text relative to plain paragraphs, this “not indexed” status is likely to be applied.

A few of these type of pages also suffered from “soft 404” status. I found these were very short pages that contained text with “error” or “missing”. I reworded those articles to avoid those terms. I made sure the titles didn’t include those terms. I also ensured there were not too many header tags relative to the text–perhaps one header at most per “page” of text.

The fix to these issues is generally to include more meaningful text–be sure an article is at least one or two full paragraphs. Add context that would help a more novice user understand why you applied that solution or approach. Avoid sensational or colloquial text as the search engines are smart enough to recognize this as low quality writing. As always, maintaining good spelling and adequate grammar help the search engine better appraise the quality of your content. Also consider short (less than 50 character) but meaningful page titles.

For long-lived blogs such as this one, there is inevitably content that is no longer relevant to anyone except for historical purposes. You may not want to simply delete these posts that you took time to research and share, but realize these old posts are costing your current performance by wasting crawler budget. The approach I take is to mark these old pages with “noindex” metadata. That allows reminiscing about old technology such as Blackberry OS 10 without degrading the performance of my currently relevant content. I think of it as a soft deprecation of the content.