# -------------------------------------------- # robots.txt - instructions for web robots/spiders, # should always be present in site root, to avoid # 404 errors logged by its absence. # -------------------------------------------- User-agent: * Disallow: /hidden/ Disallow: /1918566.asp