Jump to content

Welcome to Geeks to Go - Register now for FREE

Geeks To Go is a helpful hub, where thousands of volunteer geeks quickly serve friendly answers and support. Check out the forums and get free advice from the experts. Register now to gain access to all of our features, it's FREE and only takes one minute. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more.

Create Account How it Works
Photo

How to get ROBOTS.TXT to disallow most of my many subdirectories?

robots.txt nofollow

  • Please log in to reply

#1
ally1205

ally1205

    Member

  • Member
  • PipPip
  • 12 posts

Happy new year!

 

Say I wanted my Robots.txt file to allow all files in my web site's root directory but disallow all of my (many) sub-directories except one called 'ABC'. Would the following work?

---------------------------------------

User-agent:*

disallow: /*/*

allow: /ABC/

---------------------------------------

 

 

TIA...


Edited by ally1205, 01 January 2016 - 10:48 AM.

  • 0

Advertisements


#2
RKinner

RKinner

    Malware Expert

  • Expert
  • 17,350 posts
  • MVP

Not that familiar with the program but most programs will see the disallow everything first and then not bother with the Allow so I would at least reverse the last two as they do on:

 

 https://support.goog...f_topic=6061961

 

You might be interested in this test program:

 

https://support.goog...r/6062598?hl=en


  • 0






Similar Topics

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

As Featured On:

Microsoft Yahoo BBC MSN PC Magazine Washington Post HP