Changeset 2158 for libswish3/trunk/src/xapian/swish_xapian.cpp

Timestamp:

09/19/08 07:16:36 (4 months ago)

Author:

karpet

Message:

yank words.c in favor of tokenizer.c -- benchmarking shows tokenizer.c is 20% faster for ascii and about the same for utf-8

Files:

• libswish3/trunk/src/xapian/swish_xapian.cpp (modified) (1 diff)

libswish3/trunk/src/xapian/swish_xapian.cpp

@@ -310 +310 @@
-        swish_debug_docinfo(parser_data->docinfo);
-    }
-WORD
-wordlist(parser_data->wordlist
+TOKEN
+token_list(parser_data->token_iterator
-    }
-    if (SWISH_DEBUG & SWISH_DEBUG_NAMEDBUFFER) {