Changeset 1612 for trunk/swish_website/etc/spider.config

Timestamp:

02/02/05 17:46:11 (4 years ago)

Author:

whmoseley

Message:

Don't want to index the archives twice...

Files:

• trunk/swish_website/etc/spider.config (modified) (1 diff)

trunk/swish_website/etc/spider.config

@@ -35 +35 @@
-
-            return 0 if $path =~ /\.(?:gif|jpeg|.png|.gz)$/i;
+
+            return 0 if $path =~ m!/archive/!;  # don't index the list archive again
-
-            # Don't follow any links above the base_url