root/swish_website/src/devel/index.html

[% META
    title = "Development Information"
    id    = "development"
    author = '$Author$'
%]


<h1>Development</h1>

<p>
The current stable release is [% link_to_page('download', swish.current_version ) %].
</p>

<p>
Swish-e is continually under development.
This page contains a laundry list of requested features planned for a future
Swish-e release. To request new features, bug fixes, or (best of all) 
to submit code patches, send e-mail to the

[% link_to_page('discuss' , 'Swish-e mailing list' ) %].
</p>

<h3>Daily Builds</h3>

<p>
Swish-e source is available for anonymous public download from
the [% link_to_page('cvs', 'swish-e subversion server' ) %].
</p>

<p>
The daring and adventurous can download the
<a href="[% site.url.latest_snapshot %]">daily build snapshot</a> from
the [% link_to_page('daily') %] page.
This is <strong>not an official release</strong> of Swish-e,
rather the current development version.
There is no guarantee that these packages run.
Please do not use this code in production.
</p>

<p>
For Windows development binary (pre-compiled) snapshots, please
see <a href="http://www.webaugur.com/wares/files/swish-e/daily/">http://www.webaugur.com/wares/files/swish-e/daily/</a>.  

<br />
The most current Windows <strong>development</strong> version is
<a href="http://www.webaugur.com/wares/files/swish-e/daily/swish-latest.exe">here</a>.
</p>

<p>
Questions regarding daily development builds,
or about using Swish-e in general, should be directed to
the [% link_to_page('discuss', 'Swish-e mailing list' ) %].
</p>


<h3>Features planned for 2.6</h3>

<p>
<ul>
<li>Remove expat and other older parsers. Libxml2 will be default (only) parser.</li>
<li>Remove -S http method.</li>
<li>Documentation overhaul.</li>

</ul>
</p>

<h3>Features planned for 3.0</h3>

<p>
Swish-e 3.0 (abbreviated Swish3) will be a complete overhaul of the code.
You can <a href="http://dev.swish-e.org/wiki/swish3">track development progress here</a>.
Major feature improvements will include:

<dl>
 <dt>Unicode support</dt>
 <dd>Unicode is the <a href='http://www.unicode.org/unicode/faq/'>international standard 
 for character encodings</a>. Swish3 will implement
 support for the <a href='http://www.cl.cam.ac.uk/~mgk25/unicode.html'>UTF-8</a>
 <a href='http://czyborra.com/utf/'>character encoding</a>,
 which should handle all major languages in the world (UTF-8 handles up to 
 2,147,483,648 unique characters).
 The Swish-e developers need input from non-English language experts. 
 Please contribute to the discussion at the
  
  [% link_to_page('discuss' , 'Swish-e mailing list' ) %].
  
 Some significant known issues include:
 <p />
 <dl>
  <dt>lowercase vs. UPPERCASE</dt>
  <dd>Version 2.x uses <tt>tolower()</tt> to lowercase all characters
  before searching and indexing. Should the same approach be used for UTF-8? Will this have
  significant impact on usability for non-English languages? 
  </dd>
  <dt>Wildcards</dt>
  <dd>Version 2.x uses an internal table to support wildcard searching with <tt>*</tt>.
  The table assumes 8-bit (non-Unicode) character encoding. That approach will likely need
  to be re-thought for multibyte encodings like UTF-8.
  </dd>
  <dt>Tokenizing</dt>
  <dd>Version 2.x uses 5 different configuration options to control how a 
  'word' (token) is defined. The basic assumption is that a word is defined by which characters it
  <i>includes</i>. That assumption is based on a manageable character set of 256 characters.
  However, the sheer size of UTF-8 makes that system unworkable. Instead, some kind of
  regular expression library will likely be used.
  </dd>
  
  <dt>Stemming</dt><dd>The stemmers used will need full international support.
  </dd>
  <dt>Configuration format</dt>
  <dd>Since Swish-e depends on a configuration file for StopWords, Character
  definitions, etc., the parsing of the configuration file must support UTF-8 as well.
  The current idea is to switch to XML-style configuration files and use Libxml2 to parse
  them.
  </dd>
 </dl>
 
 </dd>

 <dt>Incremental indexing</dt>
 <dd>Swish3 will support true incremental indexing. This will allow for document records
 to be modified, added and deleted in an existing index. This feature may or may not build
 on the version 2.x experimental btree/incremental feature.
 </dd>
 
 <dt>Scaling</dt>
 <dd>Swish3 will reliably scale to larger (multimillion) document collections.
 </dd>
 
 <dt>Indexing API</dt>
 <dd>Swish3 will include an indexing API in addition to the current searching API.</dd>
 
 <dt>Streamlined feature set</dt>
 <dd>Swish3 will not contain several features in the current version:
 <ul>
  <li>Expat parsers</li>
  <li><tt>-S http</tt> indexing method and related configuration options</li>
  <li>Older stemmers</li>
  <li>Current native index format</li>
 </ul>
 </dd>
 
 <dt>Alternate index backends</dt>
 <dd>Swish3 will offer alternate index backends using available open source libraries,
 such as <a href='http://xapian.org/'>Xapian</a>, 
 <a href='http://hyperestraier.sourceforge.net/'>HyperEstraier</a>,
 <a href='http://incubator.apache.org/lucene4c/'>Lucene</a>, or 
 <a href='http://www.lemurproject.org/'>Lemur</a>.
 </dd>
 
</dl>
</p>


<hr />

<h3>The Players</h3>

<p>
You can't tell the players without a program. And we wouldn't have a program without
all these players! All these folks have made key contributions to Swish-e:
If you are not listed here, and you should be, <a href="mailto:roy.tennant@ucop.edu">drop a line</a>.


<h4><i>On the Field</i></h4>

<DL>

<DT><b>Bill Moseley</b><DT>
<dd>
The person leading the charge.  Rewrote much of the documentation and bundled it with the distribution
(you now know who to complain to), added the "prog" document source feature,
added Expat and libxml2 parsers, redesigned properties, and added many new and exciting features.
</dd>

<dt><b>Jose Manuel Ruiz</b></dt>

<dd>
Jose added phrase searching and has made huge contributions toward speed and memory
usage improvements.  He added result sorting, improved metanames and properties, merging, and searching.
Swish is the powerful program it is today because of Jose.  And there's more coming!
</dd>

<dt><B>David Norris</B></dt>

<dd>
David has provided ports to all flavors of Windows, as well as a Swish-e interface script written in PHP3.
The windows version is now bundled with a self installer, making instalation just a click away.
</dd>


<dt><b>Peter Karman</b></dt>
<dd>Peter added improvements to the ranking code and a new website design. His main role is creating more
work for Bill.</dd>

<dt><b>Roy Tennant</b></dt>

<dd>Roy was the one who originally rescued SWISH when Kevin Hughes, the
original author, was no longer supporting it. He has remained active in
the effort since the beginning, but can't code in C to save his life,
and therefore must remain content with web site support and other such
minor tasks.</dd>

</dl>

<H4><i>Hall of Fame</i></H4>


<dl>


<dt><b>Bill Meier</b></dt>
<dd>
Bill improved the ranking code, and provided much help in memory optimizations and indexing speed.
</dd>

<dt><B>Rainer Scherg</B></dt>
<dd>
Rainer has worked on Swish-e for many years.  Rainer added Swish-e's filters providing ways to index many
document types.  Rainer also added the powerful "-x" feature to easily control Swish-e's output.
<dd>


<dt><b>Giulia Hill</b></dt>

<dd>Giulia was the first programmer to tackle upgrading SWISH to
Swish-e, back when it was a project of the UC Berkeley Library.
Without her, we would not have gotten out of the starting gate.</dd>

<dt><b>Ron Klatchko</b></dt> <dd>Ron added the crawling capability to
Swish-e, subsequently enhanced by others.</dd>

<dt><b>Kirk Hastings</b></dt>

<dd>Kirk programmed a neat Perl-based tool, called "AutoSwish" that
allowed anyone to easily set up and maintain indexes from a web page.
Unfortunately, this program is no longer a part of the release due to
security issues.</dd>

<dt><b>Bas Meijer</b></dt>

<dd>
Bas has been an active member of the Swish-e team since 1999 providing code enhancements and user support.  
He converted Swish-e's build process to the GNU Auto Configure script and ported Swish-e to a number of
platforms.  Bas has also provided add-on scripts to the Swish-e user community.
</dd>


<dt><B>Marc Gaulin</B></dt>
<dd>
Marc added code to support the document properties and stemming features, among other things.
</dd>

<dt><B>Warren Jones</B></dt>

<dt><B><A HREF="http://is.rice.edu/~riddle/">Prentiss Riddle</A></B>, <A 
HREF="http://www.rice.edu/">Rice University</A></dt>
<dd>
The source of a number of
SWISH bug fixes that were implemented in the first Swish-e release
</dd>

<dt><B>Mark Seiden</B></dt>

</dl>


</p>

<P>
We owe a debt of gratitude to <a href="http://kevcom.com/"><B>Kevin 
Hughes</B></a>, without whom there would be no SWISH, and definitely no 
Swish-e. His dedication to building useful tools and making them widely 
available should be an inspiration to us all.

</p>