# /robots.txt file for http://www.louisiana.edu/ # comments to webmaster@louisiana.edu # DATE: 2005.Mar.08 # AUTHOR: Duke Hillard (dxh0844@louisiana.edu) # We're using an unofficial standard from 1996 to control robot behavior. # As the standard is unoffical, some robots disregard it. Some robots are # capable of recognizing a variety of extended commands that are proposed # by themselves or by a consensus of unoffical standard bearers. As there # are a number of these in play, the time and energy invested in learning # each and every ruleset (moving targets, anyway) is currently too great. # As a result, we follow the original standard because it appears to have # the widest acceptance. Currently, three sites summarize in a useful way # the origins and evolution of robots.txt. They are: # (1) http://www.robotstxt.org/wc/robots.html # (2) http://www.searchtools.com/robots/ # (3) http://www.google.com/remove.html # Even though our first User-agent field declares that we wish all robots # to adhere to our directives, the robots in the subsequent User-agent # fields are very polite and want a specific invitation to index our site. User-agent: * User-agent: htdig User-agent: NetMechanic Disallow: /favicon.ico # favorites icon for IE users Disallow: /footer.txt # internal use Disallow: /footer.html # internal use Disallow: /trademark.html # outdated file Disallow: /cgi-bin/ # Common Gateway Interface Disallow: /Computer/ # outdated directory Disallow: /Departments/ # outdated directory Disallow: /error/ # error messages Disallow: /Graphics/ # contain no text Disallow: /iticse99/ # temporary directory Disallow: /Organizations/ # outdated directory Disallow: /WebEvent/ # configuration files Disallow: /Web/ # dynamic reports Disallow: /SACS/ # we don't want this subdirectory to be searchable # Hypothetically, robots are case-insensitive, so we disallow these just once. # For a variety of reasons, we wish to index only HTML and PHP. Disallow: .ai Disallow: .aif Disallow: .aiff Disallow: .arc Disallow: .asp Disallow: .au Disallow: .avi Disallow: .bmp Disallow: .cfm Disallow: .cgi Disallow: .class Disallow: .csh Disallow: .css Disallow: .doc Disallow: .dot Disallow: .eps Disallow: .exe Disallow: .gif Disallow: .gz Disallow: .hqx Disallow: .ico Disallow: .ief Disallow: .inf Disallow: .jar Disallow: .jpeg Disallow: .jpg Disallow: .js Disallow: .jsp Disallow: .lwp Disallow: .maf Disallow: .mam Disallow: .mcw Disallow: .mdb Disallow: .mid Disallow: .mov Disallow: .mpeg Disallow: .mpg Disallow: .mp3 Disallow: .mp4 Disallow: .pdf Disallow: .pl Disallow: .png Disallow: .ppt Disallow: .ps Disallow: .psd Disallow: .pwd Disallow: .qt Disallow: .ram Disallow: .rm Disallow: .rtf Disallow: .sea Disallow: .sh Disallow: .sit Disallow: .svg Disallow: .swf Disallow: .txt Disallow: .wav Disallow: .wma Disallow: .wmv Disallow: .wpd Disallow: .wps Disallow: .wvx Disallow: .xbm Disallow: .xls Disallow: .zip