18 Jun Airo Security Stated: Top-Level Domain Bias in Search Engine Indexing and Ranking…
As a search engine optimization (SEO) and domain name consultant, one of the questions I get asked most often about domain names is whether or not the domain name or TLD (Top-Level Domain) matters. Will the domain name ending have an effect on SEO or search engine rankings. Are certain domain name endings preferred by the search engines over other domain name extensions? I decided to answer this question based on search engine optimization testing and not just on my personal and professional experience.
In order to begin to answer this question, we have to register domain names, set up websites, set up unbiased SEO testing software, and track the results. That’s exactly what I did — and I used the help of two different organizations, Digitaleer and SIA (SEO Intelligence Agency). Clint Butler from Digitaleer help set up and monitor the testing, SIA helped provide the software and testing, and Hartzer Consulting provided the analysis for this research.
SEO TLD Domain Test Methodology
In order to properly test, in an unbiased way, whether or not the domain name ending has any impact on indexing and search engine rankings, we set up SEO testing. We will call them TLDs (Top-Level Domains) going forward in this document. The methodology we used was the following:
Pick a list of TLDs based on popularity and the number of domain name registrations. Pick both old and new TLDs, as well as ccTLDs (country code TLDs). Make up a word and register that word in all those TLDs. Choose one made-up keyword per domain name. Set up one website per domain name. Track the indexing of each website. Track the rankings of each keyword on each website. Track the progress and report the results.
Before I chose the domain names and the TLDs for the testing, I consulted with several domain name industry professionals and domain name investors. I wanted to make sure that the TLDs that were picked were acceptable for the test. Several recommended that .COM, .NET, and .ORG be used in the testing, as well as ccTLDs be used, as well. I chose the most popular new gTLDs based on domain name registrations, as well as popular ccTLDs (country code TLDs). I used data from nTLDSTats to help with these decisions.
Ultimately, the following TLDs were chosen for the test:
I “randomly” chose a made-up word that does not exist. In picking the word for each domain name, I chose the name of a popular domain name conference, NamesCon, and spelled it backward. At the time of domain name registration, there were only 16 search engine results for this made-up word in Google.com’s search engine results, so it was a good prospect to use. It’s important to note that the keywords chosen (one per website) are not related in any way to the made-up word (NamesCon backward). We didn’t try to rank for “nocseman” as a keyword, and that keyword is only present in the domain names. It is not present anywhere on the websites that were built.
Initially, we wanted to test the indexing of each of the websites. We let the websites sit for a certain number of days, and then when they weren’t indexed (no pages were indexed), we verified the websites in Google Search Console and manually submitted one page per site. Then, after a certain number of days, we used SEO indexing software to help index the rest of the pages that were not indexed. There were 25 pages per domain name for a total of 375 web pages.
We also tracked the search engine rankings of each keyword — one keyword was assigned per domain name.
We monitored the progress and reported the results in a few ways:
- The SEO Testing Software we used checked indexing and rankings of the assigned keyword for each domain name. We strongly believe in the method we used to check the indexing and rankings using the software.
- I manually performed a search at designated times using the site: search operator in Google to see how many pages were indexed.
- The other way I used was to review Google Search Console data and the reports that are provided in Google Search Console.
Timeline of the SEO Testing
We took several days to perform the SEO testing of these TLDs. We initially registered the domain names, created the websites, watched the indexing process of all pages, and the looked and analyzed the search engine rankings. Here is the timeline for the SEO testing:
|December 27, 2019||domain names registered|
|December 30, 2019||websites went live|
|December 31, 2019||SEO test software set up and started tracking/monitoring|
|January 9, 2020||No pages indexed|
|January 9, 2020||All sites verified in Google Search Console using html verification file|
|January 9, 2020||One page of each site manually submitted via URL Inspection Tool in Google Search Console|
|January 9, 2020||All submitted pages indexed within minutes|
|January 13, 2020||All pages on all domains submitted via a search engine indexer tool|
|January 15, 2020||SEO Testing concluded. SEO TLD Domain Test Results|
Once the domain names were registered, the websites were set up, and the SEO testing software was in place, I tracked the progress, made some adjustments to the indexing process. For example, it was necessary to verify the websites in Google Search Console and manually submit one page using the URL Inspection Tool in Google Search Console. We previously had also tried to force indexing by submitting one page from each website using the Mobile-Friendly Test tool by Google. That did not work, so we were forced to use the URL Inspection Tool.
Previously, once a domain name was registered, the search engines would start crawling the websites, looking for new pages. This has changed drastically in the past few years, making it now necessary to “force” Google to crawl and index pages. For example, I now recommend using the URL Inspection Tool to ask Google to crawl a new page or re-crawl a web page that has been updated.
During the testing phase, and when the testing was complete, we discovered many different…