I am going to preface this post by saying one thing: I love technical SEO. The thought of performing an SEO audit to help identify site architecture issues for a client is a satisfying one. It’s one of my favorite parts of my job!
In our industry, there has always been quite a bit of controversy about whether or not it takes technical skills to do modern SEO. Some even argue that no technical skills are required at all, so I would like to weigh in: Technical skills are crucial when optimizing and growing your site (Rand agrees!) Understanding how Google crawlers see your site and how to improve your site’s crawl-ability is vital. I feel as though every SEO today has the ability to become more technical with access to the right tools.
Dan Sharp – Director at Screaming Frog
Response Codes:
Screaming Frog allows you to group your site’s URLs by their response code. This can help you easily figure out where you have used 301s vs 302s. Oftentimes you will see a 302 temporary redirect being used when a 301 permanent redirect is necessary. This can lead to issues when Google is trying to figure out which page it should be indexing.
The response code section can also help identify which pages are resulting in a 404. Having pages 404 when there are valuable links pointing to them is lost value. This is an opportunity to have those 404 pages pass through a 301 redirect, thereby regaining the value of the original link.
Redirect Chains:
Screaming Frog generates several reports from a crawl, one of which is the redirect-chain report. This report allows you to look at all the chains, redirects that lead to other redirects, that have been created. Crawlers can get stuck within redirect chains and cause serious site-speed issues. These chains can also lead to the loss of link value since every redirect does not pass the full value that a direct link would.
Uppercase:
Using uppercase in your URL structure can be detrimental to your site’s health. Screaming Frog allows you to isolate URLs with capitalization and identify if you have a problem. Servers handle capitalization very differently. Some servers will see www.example.com/page and www.example.com/Page as the same page, while others see them as separate pages. When these are seen as separate pages you have several issues: These pages are seen as duplicate content. They compete against each other in the SERPs and they split your traffic in the process.
Parameters:
Being able to reference a full list of the URLs generated with parameters can be very useful when trying to understand how Google is crawling and indexing your site. Pairing this list with what’s included in Search Console’s URL Parameters section will give you a good start. Search Console lets you categorize your parameters so that Google crawls your site more efficiently. By identifying the effect a parameter has on a page and selecting which URLs with parameters you want crawled makes Google’s job easier. Using this tool, you can clarify where duplicate content exists on your site and ensure Google is indexing the URL you want.
Canonicals:
Screaming Frog pulls together a list of all URLs and their corresponding canonical URLs. These tags are very important in making sure search engines know what your preferred URLs are. The canonical error report lays out which pages are not using the correct canonical and helps identify if any canonicals are pointing to dead pages. Making sure these canonicals are going to the correct page can be critical. Having all of your site’s SEO value attributed to your home page or a 404 page through the use of canonicals can cause major problems for your site’s health.
Title Tags & Meta Descriptions:
Having unique meta titles and descriptions across your site is important for optimization and click-through rate. Screaming Frog locates and highlights where you are missing titles or descriptions, where they are duplicated, and where they go over the character limit. There is also a handy feature that allows you to simulate what the SERP snippet would look like in real time on desktop or mobile. You can simply choose a URL and adjust the title and meta description for it.
Analytics Code:
When you are migrating your site, changing domain/cms or merging with another site, it’s always good to make sure your Google Analytics code is transferred properly. Crawling is an effective way to ensure that the correct Google Analytics code has been applied across all the pages of your site. Running a custom configuration that includes your unique tracking id will provide you a list of all the pages containing the code, as well as a list of the pages that are missing that code.
XML Sitemap:
Screaming Frog allows you to easily generate a sitemap that contains only the URLs that you want crawled. Once you have generated a new sitemap you are able to manually adjust the priority and the frequency of the pages included.
An XML sitemap is the roadmap for crawlers to ensure they have reached all the pages that you want indexed. This file can be saved from Screaming Frog and then uploaded to your Search Console account.
Images:
Screaming Frog allows you to take a quick inventory of images on your site as well. Making sure all your images are in the right format can really make an impact on your site speed. Your image file type is listed in the content column when you select the image tab. Images are also an easy place to optimize and target your keywords (as long as they make sense). This generated list allows you to check file names for optimization as well as check for images missing alt text. Screaming Frog has endless possibilities when it comes to aiding your SEO efforts. As modern digital marketers, we are not doing our job fully if we are not using the tools available to us. Sharp agrees “SEO is maturing all the time and continues to evolve; the principles are much the same, while technologies, methods and tactics change.” Exploring new tools and finding creative ways to develop your digital marketing strategy is central to performing modern SEO effectively.
Big thanks to Dan Sharp at Screaming Frog for his contribution.
