Robots.txt

Code snippets to add to your robots.txt file