tokenizer.c

	Rev	Chgset	Date	Author	Log Message
(edit)	@2176	[2176]	09/22/08 22:57:38	karpet	all tests passing, all (known) leaks fixed
(edit)	@2175	[2175]	09/22/08 10:08:24	karpet	clean up a few compiler warnings
(edit)	@2171	[2171]	09/22/08 00:08:00	karpet	get rid of the circular reference in TokenList/Token?; make swish_init() a …
(edit)	@2169	[2169]	09/21/08 22:28:58	karpet	nevermind with cpts.
(edit)	@2166	[2166]	09/21/08 21:59:57	karpet	store codepoints instead of start_byte (since we no longer need the …
(edit)	@2165	[2165]	09/20/08 21:44:21	karpet	simply the ->value by pointing directly into the internal buffer, and …
(edit)	@2162	[2162]	09/20/08 15:37:55	karpet	refactor to simplify signatures for TokenIterator?. Now a TI always creates …
(edit)	@2160	[2160]	09/20/08 01:05:24	karpet	ref_cnt checking
(edit)	@2155	[2155]	09/18/08 23:51:41	karpet	avoid xmlStrncat because (a) it fails under linux and (b) the realloc is …
(edit)	@2153	[2153]	07/31/08 23:10:16	karpet	fix the bump_word feature so that ->pos for Word or Token reflects …
(edit)	@2148	[2148]	07/21/08 23:51:24	karpet	change top-level tokenizer functions to use same signature so that we can …
(add)	@2142	[2142]	05/05/08 23:30:12	karpet	add some tokenizer tests and (doh!) include tokenizer.c

Note: See TracRevisionLog for help on using the revision log.