# Disallow Access to spiders/crawlers so individual # files pertinent to courses & resources are not # spidered and directly listed in search results page # This allows all directories containing the .html files # to be spidered but not local files linked in those # .html files #disallow offline browsers and bad bots User-agent: HTTrack Disallow: / User-agent: HTTrack/3.0x Disallow: / User-agent: HTTrack Website Copier/3.0x Disallow: / User-agent: Teleport Disallow: / User-agent: Teleport+Pro/1.29 Disallow: / User-agent: Teleport+Pro/1.29.1820 Disallow: / #Disallow all access to datasheets & source code directly User-agent: * # Root Site # images Disallow: /images/ Disallow: /customerrors/ #Added because Google not seeing below disallows Disallow: /Lecture/ Disallow: /StudentPresentations/ Disallow: /negele/ Disallow: /AVRLibrary/ Disallow: /Tutorial/ Disallow: /DataSheets/ Disallow: /calculators/ # Ericks Personal Site Disallow: /elo/EE-Work/ # ME601 Site # course lectures Disallow: /me601/Lectures/ # Student Presentations Disallow: /me601/StudentPresentations/ # Lab Supported Projects Site # Directory with thesis Disallow: /SupportedProjects/negele/ # Lab Resources Site # source code library Disallow: /labresources/AVRLibrary/ # Atmel tutorials Disallow: /labresources/Tutorial/ # datasheets Disallow: /labresources/DataSheets/ # Calculators Disallow: /labresources/calculators/ # images Disallow: /labresources/images/