User-agent: * Disallow: /admin Disallow: /assets/whitepapers Crawl-Delay: 4