Robots.txt is a text file that tells search engine crawlers which parts of your site they can or cannot access. It sits at your domain root and guides crawl behavior.
Robots.txt gives instructions to crawlers using "Allow" and "Disallow" rules. It is the first file bots check when visiting your site.
No, it controls crawling. Use noindex to keep pages out of the index.
At the root: yoursite.com/robots.txt.