User-agent: * Allow: /ads.txt Disallow: /App_Code/ Disallow: /Bin/ Disallow: /rev Disallow: /css/ Disallow: /webstats/ Disallow: /mobapp/ Disallow: /se_catcher/