What is robots.txt?
robots.txt is a small text file at the root of your site, at yourdomain.co.uk/robots.txt, that tells search engines which parts to skip and where to find your sitemap. Most business sites need very little in it, but search engines look for it on every visit and a missing file is a small mark against you. The danger is the opposite: a robots.txt that blocks the entire site.
What it is for
It is a list of instructions for crawlers. A typical file for a business site says: all crawlers are welcome, please skip the admin area, and here is the sitemap. It is a request rather than a lock; well-behaved crawlers such as Google obey it, but it does not hide anything from people, so never use it to protect private content.
A sensible file
For a WordPress site, three lines are enough: "User-agent: *", "Disallow: /wp-admin/" and "Sitemap: https://yourdomain.co.uk/wp-sitemap.xml". WordPress generates a file like this automatically if there is no physical file in place, and SEO plugins let you edit it.
How to check
Our check requests your robots.txt and reports whether it exists, whether it blocks everything, and whether it names a sitemap. You can open it in a browser by adding /robots.txt to your domain.
How to fix it
If the file is missing, create it with the three lines above, or let WordPress generate it by making sure no empty physical file is in the way. If it names no sitemap, add the Sitemap line so search engines find every page. If it contains "Disallow: /" on its own, remove that line immediately: it is blocking the whole site from search.
Not sure how your own site does on this? Run the free check: it takes about twenty seconds and reports on this and thirty other things.