WordPress Robots.txt: What Should You Include? searchenginejournal.com

Slash unnecessary crawl activity and index bloat by upgrading your WordPress robots.txt file. Here’s what to include, exclude, and never block.