Using robot.txt is it possible to restrict robot access for (specific) query string (parameter) values?
使用robot.txt是否可以限制(特定)查询字符串(参数)值的机器人访问?
ie
http://www.url.com/default.aspx #allow
http://www.url.com/default.aspx?id=6 #allow
http://www.url.com/default.aspx?id=7 #disallow
2 个解决方案
#1
User-agent: *
Disallow: /default.aspx?id=7 # disallow
Disallow: /default.aspx?id=9 # disallow
Disallow: /default.aspx?id=33 # disallow
etc...
You only need to specify the url's that are disallowed. Everything else is allowed by default.
您只需要指定不允许的网址。默认情况下允许其他所有内容。
#2
Can just the query variable defined such as
可以只定义查询变量,如
Disallow: /default.aspx?id=*
or better still
还是更好
Disallow: /?id=
#1
User-agent: *
Disallow: /default.aspx?id=7 # disallow
Disallow: /default.aspx?id=9 # disallow
Disallow: /default.aspx?id=33 # disallow
etc...
You only need to specify the url's that are disallowed. Everything else is allowed by default.
您只需要指定不允许的网址。默认情况下允许其他所有内容。
#2
Can just the query variable defined such as
可以只定义查询变量,如
Disallow: /default.aspx?id=*
or better still
还是更好
Disallow: /?id=