yabbawhap - Y and AP compression filters yabba applies Y compression to its input; unyabba decompresses the result. whap applies AP compression to its input; unwhap decompresses the result. whap and unwhap run at about the same speed as UNIX compress and uncompress, which use LZW coding; yabba and unyabba are two to three times slower. AP and Y compression are typically 10-20% more effective than LZW compression in the same amount of memory. Y coding, unlike LZW coding and AP coding, is unpatented. It should be possible to use these programs on any reasonable C platform, though they were originally designed on a BSD UNIX system. yabbawhap version 1.00, March 19, 1991. Placed into the public domain by Daniel J. Bernstein. Thanks to the gamma testers for their comments, criticism, and code: Eirik Fuller Marc Andreessen Loren J. Rittle Jean-loup Gailly Terje Malmedal Lennart Augustsson John C. Schultz Dave Gudeman Colin Plumb Ozan Yigit Benno Tietz Alexios Zavras Hans Henrik Eriksen Frank Wales Paul A. Houle Graham Thoal Robert Kelley Organization of README: 1. Files 2. Requirements 3. How to configure 4. How to compile 5. How to install 6. TODO list 1. Files: BLURB advertisement CHANGES list of changes since first distributed version README this file FORMLETTER form letter to send to the author PATENTS some notes on compression patents QUESTIONS questions and answers about yabbawhap FILES file list sysconf script to test for certain system features checkconf.c tool to ease Makefile configuration Makefile compilation commands try script to test yabba and unyabba and compare to compress tryap script to test whap and unwhap and compare to compress tryapy combination of try and tryap INSTALL script to install the programs yabba.1 man page for yabba and whap unyabba.1 man page for unyabba and unwhap huptrie.h huptrie library bitout.h bit output library percent.h percent library, computes 100a/b without overflow texts.h various messages printed by the programs yw.c main code for yabba and whap unwhap.c main code for unwhap unyabba.c main code for unyabba bitout.c bit output code percent.c percent code texts.c texts code ycoding.4b Y Coding, paper draft 4b ycoding.uu yabba'd, uuencoded version of ycoding.4b 2. Requirements You should be able to adapt yabbawhap to practically any C platform (with 8-bit characters). However, this package was designed on a UNIX system. The compressors do not take file names; they only act as filters. The support files are also oriented towards UNIX. yabbawhap has been reported to work on the following systems: Sun 4/280, SunOS 4.0.3 (1.00) Sun 4/280, SunOS 4.0.3 (0.98) Sun 4/280, SunOS 4.0.3, gcc (0.98) Sun 4/330, SunOS 4.0.3 (0.95) Sun 4/330, SunOS 4.0.3c (0.98) SPARC?, SunOS 4.1 (0.95) Sun 3/?, SunOS 4.1 (0.95) Sun 3/160, SunOS 4.1 (1.00) Sun 3/480, SunOS 4.1 (0.98) SPARCStation SLC, SunOS 4.1 (0.95) Sun 3/50, SunOS 4.1.1, gcc (0.98) Sun 3/60, SunOS 4.1.1, gcc (0.98) Sun 3/60, SunOS 4.1.1, gcc 1.39 (0.95) SparcStation 2, SunOS 4.1.1 (0.98) DECStation 5400, Ultrix 4.1 (1.00) DECStation 5000/200, Ultrix 4.1 (0.95) DECStation 5000/200, Ultrix 4.1, gcc (0.98) DECStation 5000/200, Ultrix 4.1 (0.98) may need -Olimit 2500 to compile with DEC's ANSI C compiler V2.10 DECStation 3100, Ultrix 4.0 (0.95) DECStation 3100, Ultrix 4.0 (0.98) DECSystem 5820, Ultrix 4.1 (0.98) DECSystem 5820, Ultrix 4.1 (1.00) VAX 11/780, BSD 4.3 (0.95) VAX ?, BSD 4.3, gcc 1.39 (0.95) Tek 4316, Utek 4.1, gcc 1.39 (0.95) need -DBRAINDAMAGED Sequent S811, Dynix 3.0.17 (0.95) Sequent Symmetry, Dynix 3.0.17.9 (0.95) Sequent ?, BSD 4.2? (0.95) Apollo DN3500, DOMAIN/OS SR10.2, cc -A cpu,3000 -W0,-opt,2 (1.00) Convex C1-XP, Convex UNIX 9.0, cc -pcc (1.00) need -pcc for the new compiler; -UPTRS faster than -DPTRS IBM RS/6000, AIX 3.1 (0.95) HP 9000s300, HP/UX 7.0 (0.98) -O rather than -O2, optimizer appears buggy anyway NeXT, NeXT Mach 1.0 (0.98) Astronautics ZS, ZSUnix 1.2 (1.00) Amiga, AmigaOS 1.3.2 (0.95) (Under gcc, always use -O -fstrength-reduce in place of -O2 in CCOPTS. Don't even bother with -finline-functions or the other function-call optimizations.) If your machine isn't in this list, and you get the programs working, *please* send a note to me at brnstnd@nyu.edu on the Internet describing what you had to do to make the programs compile. (Of course, please also let me know if you have trouble, or if you have comments, questions, or suggestions.) I'd rather be flooded with reports and be able to compile a more comprehensive list than have no feedback because everyone assumes someone else has talked to me first. You can use FORMLETTER if you want. Thanks for being a good sport. 3. How to configure First, run the sysconf shell script. It will try to figure out whether you have bzero(), memset(), and a certain putchar() bug, and will modify Makefile accordingly. Next, make checkconf. Then run checkconf to see a few facts about your current configuration. You can give it options, like -DNODEMAX=21000, and it will instantly show you how that change will affect the size of the programs if you add it to CCOPTS in the Makefile. It will also make sure that various constraints are met. checkconf -H shows a help screen. Next, read through the option descriptions in the Makefile, or print out a copy and peruse it at your leisure. You can configure yabba, unyabba, whap, and unwhap in several different ways to change compression size, speed, and power. No single configuration is right for every job. Finally, armed with checkconf and the option descriptions, decide how you want to configure the programs. Change Makefile appropriately, and remake checkconf just in case you want to experiment with changes later. If you want to get through configuration as quickly as possible, run % ./sysconf and press return when it asks whether it should make and run checkconf. But don't complain to me about teething trouble if you haven't read through all of README and Makefile, as well as the checkconf output. Two big caveats: 1. ZEROFILLED should be off on practically any non-UNIX operating system. 2. If you increase NODEMAX, you probably want to set -DNODENUM=65533 unless you know your recipients have compiled with the same high value of NODEMAX. 4. How to compile Just make. You'll get yabba and unyabba. If your optimizer dies, first try defining -DOPTCANT5 in the Makefile. If that doesn't work, try -DOPTCANT2. If that doesn't work, try -DOPTCANT1, or lower the optimization level. To test the programs, run the ``try'' shell script with a filename argument. % ./try yw.shar, for instance. try will give you times and results for yabba, unyabba, compress, and uncompress on the file. (Note that it does not test for nonexistent files or symbolic links.) If you want to compile whap and unwhap, beware! AP coding is patented. (Then again, LZW coding is patented, and people use compress all the time.) You should understand the information in PATENTS first. Then if you're curious to see how well AP coding can do, make AP. You can then run the tryap and tryapy shell scripts the same way as try. Another test you can run is to uudecode ycoding.uu and unyabba -m9999 the result, ycoding.Y. You should get a perfect copy of ycoding.4b. 5. How to install By default, yabba and unyabba are installed in /usr/local/bin; whap and unwhap are installed in /usr/local/bin; yabba.1 and unyabba.1 are installed in /usr/man/man1. If you want to change these defaults, edit INSTALL. Then run it from a root shell; it will check every action with you before proceeding. 6. TODO list -E errs -f filterfile (Please don't bug me about having yabba take a filename argument until you've seen what -f does.) -F flexible -m? report size? (tnx dg) make no-header and random-bits independent? better RESET defaults? put compression in function? will happen with -f rename as fwhap, fyabba, funwhap, fyabba? no use array trie for top level or two?