Sitemap Discovery

Discover sitemaps from robots.txt Sitemap: lines and common paths; count URL loc entries.