The Best robots.txt File
![[link]](http://s3-ak.buzzfed.com/static/images/public/icons/external-link2.png){kind=icon}
A robots.txt file is a way for a site administrator to give instructions to web crawlers (robots) that are crawling the site. They are not usually anywhere near this funny.
MORE GREAT STUFF
Related Links
Craig Ferguson’s Robot Sidekick
Grant Imahara, the robot guru of Mythbusters , is building a robot sidekick for Craig Ferguson while will debut tonight on The Late Late Show .
Suicide Robot
An Australian man built a robot to kill himself.
Sock Folding/Masturbation Robot In Action
This robot was designed to sort socks, but after watching it in action, I can't help imagine what other illicit functions it could be used for.
Facebook Conversations
5 Responses So Far
- Beastmaster The Best robots.txt File and thinks it’s Geeky
- Nathan C. The Best robots.txt File
- chitope The Best robots.txt File and thinks it’s Geeky
- Eduardo M. The Best robots.txt File and thinks it’s Geeky & LOL
- Michael C. thinks The Best robots.txt File is Geeky
- Almost Chicken The Best robots.txt File and thinks it’s Geeky & Cute
- Lazar Taxon The Best robots.txt File and thinks it’s LOL & Geeky
- Ken T. thinks The Best robots.txt File is Geeky
- The Internet's Very Own Veronica Belmont The Best robots.txt File and thinks it’s Geeky
- Chris Willenbrock The Best robots.txt File and thinks it’s Geeky & LOL
-
Allow: /terminating/connor/sarah
- Chris S. The Best robots.txt File and thinks it’s Cute & Geeky
- JP thinks The Best robots.txt File is WTF
- Michael A. thinks The Best robots.txt File is Geeky
- Anna P. thinks The Best robots.txt File is LOL
- M. Dushko thinks The Best robots.txt File is LOL & Geeky
- Flora C. thinks The Best robots.txt File is LOL
- pax The Best robots.txt File and thinks it’s Geeky
- lec thinks The Best robots.txt File is LOL & Geeky
- Adam Hyland thinks The Best robots.txt File is LOL
-
Isaac Asimov would be proud.
- Donkeyrock The Best robots.txt File
- Kristian S. The Best robots.txt File and thinks it’s LOL & Geeky
- citizenjane thinks The Best robots.txt File is LOL
- Amy A. thinks The Best robots.txt File is Geeky
- keeprunning The Best robots.txt File
-
god this is good
- Joshua Fjelstad thinks The Best robots.txt File is Geeky
- Jaan K. The Best robots.txt File and thinks it’s LOL
-
what is your prime directive!
- cybergata The Best robots.txt File and thinks it’s Geeky & LOL
- annam6 The Best robots.txt File and thinks it’s OMG & Geeky
-
?
- Andrew G. thinks The Best robots.txt File is Geeky
- phojoe thinks The Best robots.txt File is Geeky
- tomrankin thinks The Best robots.txt File is Geeky & LOL
- Bwgan The Best robots.txt File
- BlueInkOgre The Best robots.txt File and thinks it’s Geeky & LOL
- turtlefeed The Best robots.txt File
