Subject: webster

From: [email-masked] (Dave Curry)
Date: Fri, 26 Dec 86 22:43:17 EST

Well, I finally got a chance to do some hacking on webster.  I completely
rewrote the stuff that deals with the index file; the crap where we read
in 1.2MB bit the dust.  Now it uses dbm files and such, and takes up about
100 kbytes in memory instead of 1300.

Here's the list of things I did that I sent to people here.  If you want the
new stuff, you can get it through anonymous ftp from ee.ecn.purdue.edu.  Go
for pub/webster.tar (make sure you use image mode on the ftp).

Also, I forget if I told you, the new files from SRI-NIC have been fixed and
all the words actually seem to be there.  I strongly recommend you FTP them,
or else send me a tape and I'll put them on it for you (it takes about 4 or
5 hours to FTP the files, even late at night).

- --Dave

- --------------
Well, webster is much better now.  I'm extremely pleased with myself (and
happy, since I just happened to write code the first time that was easy to
massively change this time).

Anyway, here's a summary of changes.  Some of you may not care, but read on
anyway, what the hell.

	1. The new dictionary has been installed.  The previous copy
	   had several holes in it due to damaged files on SRI-NIC's
	   end.  The new copy does not have these holes.

	   Significance: all the words that start with "conv" are now
	   actually in the dictionary.

	2. Webster now uses your personal erase, line kill, and so on
	   characters instead of hardwiring ^H, ^X, and so on.

	   Significance: not much, but someone ragged about it today so
	   I fixed it while I was in there.

	3. Websterd no longer reads in a 1.2 megabyte index file when
	   you connect to it.  It instead uses a dbm-based index for
	   simple lookups, and uses a disk-based word search for
	   wildcard lookups.
	   
	   Significance: the first word is now defined right away
	   instead of several seconds into things; simple lookups take
	   less time in general; wildcard lookups (and Tenex word
	   completion) take slightly longer, but the time is still
	   more than acceptable; webster doesn't pound the crap out of
	   the machine when people run it.

	4. The bug in which the last word in a dictionary file (there
	   are 220 dictionary files) could not be defined, and instead
	   the response was just a blank line, has been fixed.

	   Significance: you people who have a fetish for the word
	   "fylfot" can stop bothering me now. :-)

	5. The "bug" (actually a "fix" for another bug) which caused
	   the server to constantly go away has (I think) been fixed.

	   Significance: you potato heads (bimbies, for you PUCC
	   people) who use this thing to do crossword puzzles can quit
	   calling me now. :-)

	6. The "bug" in which you can only use a numbered response once
	   instead of looking at all words (1, 2, 3, etc.) has not been
	   fixed.  It would be real hard given the design of the
	   program, and I don't think it's useful enough to bother
	   with.

	7. Thanks to Rich Kulawiec, we now have a list of how all the
	   words in the dictionary (all 59,963 of them) should be
	   hyphenated.  Our current theory is that the "syllables"
	   indicated in the definitions (and that often look wrong)
	   are actually indications of the acceptable places to split
	   a word between lines.

There are several files available from the dictionary on the ee.ecn.purdue.edu
machine in /m/webster/misc.  Included are the pronunciation guides, a list of
all the suffixes, a list of all the prefixes, the hyphenation guide, a list
of all the palindromes in the dictionary, and so on.  If you have an ee.ecn
account feel free to examine these.  If you don't, and really want the file,
send me mail.

Enjoy.  Please send any bug reports to me.

- --Dave