Google is improving its robots.txt checker

Google employee Gary Ilsh confirmed on Twitter that the company is working to improve robots.txt checking. This tool is currently only available in the older version of Search Console.

This information appeared in response to a complaint about a possible bug in the work of robots.txt or incorrect reading of this file by the service.

Gary Ilsh tweeted, “We’re in the process of integrating the tester with the production parser,” adding that he thinks the problem lies with the tool itself.

we’ll look, but it’s likely a tool error. we’re currently in the process

of integrating the tester with the production version of the parser which should fix this, if indeed it is a tool error.

* – it’s my procrastination project, so no holding breaths – Gary 鯨 理 ／ 경리 Illyes (@methode)

January 6, 2021

In early 2019, Google announced that it did not plan to disable its robots.txt checker. The company promised to eventually migrate the tool to the new version of Search Console, however, judging by Ilsh’s answer, this is not an urgent task for Google.

Source: Search Engine Roundtable Follow us