DOCUMENTATION and NOTES about the BARTON interface program: ========================================================== (by Ping Huang [pshuang@athena.mit.edu], June/July 1991) ------------------------------------------------------------------------ Tracing how the program gets data from the GEAC: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ** main() calls Interface::Service (and terminates when it returns). ** Interface::Service calls Library::read (through crnt_library, which is set to LBarton) in a forever-while loop which watches for the quit_now variable to get set and executes a return when it does. ** Library::read calls Telnet::read (through the variable tn), and maintains ohist for purposes of matching logon sequences from the library system itself. ** Telnet::read calls TtyForkExec::read through the variable tfex while implementing a watch for timing out on the connection. It maintains a history (tno) to watch and see if the telnet process itself returns any messages (such as "Unknown host" or "Connection refused", etc.) ** TtyForkExec::read calls the UNIX system "read" routine to read from a file descriptor opened as a "/dev/pty"+unique device ID characters. The TtyForkExec object's constructor had spun off a telnet process through UNIX's "exec", but rather than passing it the standard file handles, creates new /dev/tty and /dev/pty pairs and gives it those handles. ** UNIX telnet program gets characters from BARTON.MIT.EDU and places them into and expects local input from what it thinks are STDIN/STDOUT. ** character is funneled from GEAC through various hardware connections (Cisco box and other stuff?) to BARTON.MIT.EDU machine. ** The GEAC system emits a character to be displayed. Tracing how the program subsequently gets data to the user's terminal: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ** Interface::Service calls User::write to process character. ** User::write in turn calls pTerm->write since we are connected to Barton and not some other libary; pTerm=&TermVTbart, which is of type VT100_barton, which by a typedef is VT100_from_VT100barton. ** Method VT100_from_VT100barton::write checks for control codes using VT100emulator::filter_escapeCodes, which it inherited; if a control sequence (note that GEAC only sends control sequences to reposition the cursor, never newlines or carriage returns) is found by the filter, ::write translates them into curses calls; otherwise, it checks for the token "SEND" and replaces it with "ENTER" if found; it also compresses the screen by one line (for non-standard VT100 emulators like Procomm with only 24 writeable lines available). It calls userWindow->addch to do simple output. Currently I've modified it to force a refresh of the screen whenever there are more than FORCE_REFRESH (3) occurences of cursor repositions which are POSSIBLY_AT_BEGIN_OF_TEXT_LINE. ** When asked to refresh, curses updates the terminal screen from internal buffers to the terminal display, using TERMPCAP entries to figure out how to position cursor, etc. Tracing how the program gets user input to the GEAC: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ** Interface::Service calls User::read. ** User::read calls UNIX system routine "read" to get a character from STDIN, file mode of which has been set to non-blocking (no wait). I have modified this to accept input only when on the correct line for Barton, and throw input away otherwise. User::read modifies c, declared as a char in Interface::Service. ** Interface::Service calls Interface::parse_cmd with the character c. Interface::parse_cmd screens the input and captures (does not send onto GEAC) special backslashed commands like "\Q" or "\?". If it is a command, ::parse_cmd processes it accordingly, otherwise it sets isCmd, declared as a boolean in Interface::Service. ** Interface::Service calls crnt_library::write with c, calling Interface::working_message if c is a newline. crnt_library is hardwared to be LBarton, of class Barton, which inherits from class Library. Library::write translates keystrokes (like ESC, BS, NL, CR, etc.) according to constants established in the constructor Barton::Barton, and calls tn->write with the translated string. ** tn is a Telnet object. Telnet::write does a Telnet::time_check along the way, passing the string to tfex->write. (It calls Telnet::wrote, which does nothing.) ** tfex is a TtyForkExec object. TtyForkExec::write calls UNIX system routine "write" to send the string to our_filedescriptor, which points to "/dev/?ty??" magic files set up by constructors. ------------------------------------------------------------------------ What objects where (and some useful components therein): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **** [vt.cc, vt.h] VT100emulator: inherits from nothing ::userWindow (of class CursesWindow) ::filter_escapeCodes(int) ::write (overloaded) ::flush() ::check_if_refresh_needed() VT100_from_VT100barton: inherits from VT100emulator ::write(char*buf,int nchars) VT100_filter: inherits from VT100emulator, is unused by rest of program **** [user.cc, user.h] User: inherits from ReadWriteableThing ::userWindow (of class CursesWindow -- note this one isn't used) ::os (of class ostream -- hardly used at all) ::User (constructor) ::read (char*buf, int nchars) ::write (overloaded) ::flush() ::working_message ::redraw **** [interface.cc, interface.h] Library_Collection: inherits from nothing, not really utilized Parse: inherits from nothing, not really utilized Interface: inherits from Library_Collection ::*usr (of class User) ::show_menu() ::WaitForInput() ::WaitForUserKey() ::Use_Library(Library&,int) ::Close_Library() ::~Interface (destructor) ::Lbarton (of class Barton) (and other libraries) ::*crnt_libary (of class Library) ::show_welcome() ::attempt_refresh() ::Service(User&) ::Interface() **** [lib-barton.cc] Barton: inherits from Library ::Barton (constructor) ------------------------------------------------------------------------ Comments: (a) speed could probably be much improved (because the CPU overhead on LIBRARY.MIT.EDU would be reduced) if some of the procedure calls are modified to take strings which are more than one character long. Note that at least ohist (a History object) used by Library::LoggingOn may need to be increased in size. Also, inlining some of the functions may also help, although they would make the executable bigger which may force more swapping. (b) Running the same binary on LIBRARY (MicroVax II) versus on BEEBLEBROX (VAXstation 3100) showed a three-to-one difference (specifically, it took 21 seconds for the process on LIBRARY to finish repainting the screen from hitting ENTER to getting the prompt back on a database searching screen, whereas BEEBLEBROX only took 7). This would seem to indicate that GEAC and the Cisco boxes are not the narrowest bottlenecks, but the LIBRARY machine itself, especially when multiple users are making queries. (c) currently telnetting into LIBRARY.MIT.EDU leaves processes sitting around, and require that the machine be taken down every once in a while so that the process tables do not fill up completely. This might be fixed. It is also not clear what is the eventual fate of tty/pty files used to communicate with the child telnet process. At least one user has complained of getting pty/tty errors (couldn't get pair), and I had managed to duplicate this problem also on occasion. (d) there seem to be some serious problems with the dirtiness of the code since it was originally written to support many different library systems and has since then been extensively hacked for Barton. Switching between vt100/hardcopy seems to be a problem, for instance. A full re-write from scratch would be helpful. (e) occasionally on startup there is an error message "resize: unknown character received, exiting"; whence this is from needs to be tracked down; it seems to occur more often with funny geometries of terminals. (f) major changes summary: interface program now eliminates input unless the cursor is on the line on which GEAC accepts input, incrementally updates the screen every few lines so the user doesn't have to wait for a screen's worth of search results before seeing anything happen. ------------------------------------------------------------------------ Summary of specific modifications made to source: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [mostly in chronological order, but this is not guaranteed] ** [user.cc, user.h] corrected misspelling of "receiving" (as "recieving"). Occurences of this change are not noted in the code as being intuitively obvious. Change made for cosmetic reasons. ** [user.cc] User::read method, when pTerm==TermVTbart, will now throw away any input unless curses reports that the cursor is on line BARTON_INPUT_LINE (currently #defined as 21). Used to accept input always. Helps to prevent user input overrun should user get impatient waiting for the screen to refresh. Advantage is that user will not inadvertently tell GEAC to display the next twenty screens of search results; disadvantage is that user cannot type ahead, which the user could do to a small extent before (but because GEAC apparently also throws away some keystrokes intermittently when it is not ready for input, loss of utility is very small). ** [user.cc] User::working_message method now displays a more user-friendly prompt as well as use curses to make the message standout on the user's terminal. For cosmetic purposes for the end-user. ** [interface.cc] boolean variable InteractWithLibrary has been moved external to the Interface class, whereas it used to be private. This makes it possible for my hack to User::read to access this variable to allow input when not interacting with the library. ** [interface.cc] Interface::show_welcome now requires that you hit and will not accept just any key. Created simple WaitForEnter routine, modified show_help and show_menu to require exit. ** [interface.cc] Interface::show_help will now, like show_menu, zero the boolean InteractWithLibrary, thus telling my User::read hack to accept input no matter where the cursor ends up. ** [interface.h] added a null ("{}") definition of the method Close_all_Libraries in class Library_Collection, whereas it used to be just a declaration with no body. Strictly to make the linker happy. In making some of my other changes, it appears that g++ all of a sudden woke up to the fact that it had no definition for this method whatsoever. It's rather bizarre; while this method never does get called, it is referred to indirectly because the destructor for class Interface calls a procedure which calls it. ** [vt.cc] manifold intended changes to VT100emulator::write, all of which have been commented out since the method gets overrode in the specific case of Barton {growl}. ** [vt.cc] changed VT100_from_VT100barton::write so that when it processes character reposition codes from GEAC, it keeps count of the ones which reposition the cursor near the beginning of the line, and calls VT100emulator::check_if_refresh_needed. The hack to count only the ones near the beginning of line was because GEAC sends repositions for practically *EVERYTHING* and without the hack curse often refreshed the screen in the middle of a line, which didn't look very pretty. ** [vt.h] provide a null definition ("{}") for VT100emulator::flush. Whyfor is explained elsewhere (make compiler/linker happy). ** [vt.h] added declaration for new method check_if_refresh_needed for class VT100emulator. ** [Help/Welcome.txt] updated the contents of this file to read what /mit/mitlibs/info/welcome.barton read as of 6-19-91. ** [user.cc] changed User::flush to wait a little bit longer after a waiting message before clearing everything out. ** [vt.cc] added method VT100emulator::check_if_refresh_needed, keeps internal counter on how many times it's been called, forces userWindow->refresh every REFRESH_COUNT time it is invoked. ** [interface.cc] added calls to Interface::attempt_refresh() after calls to show_menu() and show_help() in Interface::parse_cmd. ** [interface.cc] implemented the "\r" option for the user to request a refresh of the screen by the library system. ** [interface.cc] added lots of "return 1" to Interface::parse_cmd where a valid command came through to tell ::Service code that the character shouldn't be passed through to the library. ** [interface.cc] updated text displayed in show_menu and show_help. ** [more.cc] modified More::run to call More::display upon invocation so that I can be sure it is doing something at all. The More class definitely seems poorly implemented and needs replacement. ** [interface.cc] show_help no longer uses More. In fact, More is not used by any portion of the program now. ** [interface.cc] several lines rearranged in Interface::Use_Library so that User::receivingBarton gets set before User::Person_is_VT100 gets called to change pTerm to the correct setting. ** [interface.cc] changed show_menu and show_help code force the user to return to interaction with the library before issuing another command (more reliable parsing of commands without rewriting the whole thing). ** [interface.cc, vt.h, user.h] changed instances of CursesWindow to ExtendedCursesWindow, which is declared in new source file savescr.h, which is now #include'd into the above files. ** [interface.cc] changed Interface::attempt_refresh to use the new capability of ExtendedCursesWindow to save and restore screens. ** [messages.cc] changed cant_refresh_message text. ** [lib-barton.cc] put in a reminder in the Barton-specific help about the special commands to get GEAC to display help screens, and reformatted the rest of the information. Will *NOT* be implmenting the actual functions keys through curses -- not worth it. ** [vt.cc] modified VT100emulator::write & VT100_from_VT100barton::write to act on VT100-SGR commands (i.e. boldfacing), modification to stay only if people like it, since the "bolding" is hard to read on xterms, being by default "reverse-video". ** [interface.cc] changed parse_cmd to turn off highlighting whenever it encounters a backslash command (so that help and menu is not highlighted), aesthetic change related to the enablement of c_SGR. ** [interface.cc] used a History object to add to Interface::parse_cmd so that the user can type "END" to quit out of the program. ** [interface.cc] changed Interface::quit to indicate to user that he has selected to quit. ** [user.cc] modified User::read to permit reads when the cursor is below the prompt line but is at the leftmost column -- this change was made so that the user can hit enter to continue on some of GEAC's help screens which leave the cursor on the next line. ** [user.cc] reposition the working message to be always right below the prompt line rather than relative to the bottom of the screen ** [library.cc] changed ohist to be a larger History (from 10 to 32). ** [interface.cc] modified User_Library to refresh screen correctly (i.e. do *NOT* used the CursesWindow saved screen). ** [interface.cc] added cleanup to remove user's command from the saved CursesWindow screen in ::parse_cmd. ** [user.cc] modified User::write -- now all writes go through the pTerm variable, whereas it used to go straight to curses if not Barton. ** [user.cc] modified Person_is_VT100 to make more sense; eliminated code which refers to dont_process_output, which is never used. ** [user.cc] added a VT100emulator in object User so that Person_is_VT100 can switch back to a clean VT100emulator if necessary. ** [interface.cc] Interface::Service now requests data from the library 32 bytes at a time (should reduce CPU overhead). Notes on possible limitations to requesting more data at a time: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Interface::Service uses scratchbuffer[1000] Library::read adds no limitations (uses scratchbuffer as passed) ohist in Library object was created with History(10) tno in Telnet object was created with History(100), usage in Telnet::red suggests keeping size below 100-55=45 ** [library.cc, lib-barton.cc] modified almost all default messages (clarity). ** [messages.cc, message.h] changed update_messages to be more general by taking the filename and variable to change as parameters, and to return the updated or original message. ** [interface.cc] modified call to update_message in Interface::Service. ** [lib-barton.cc] Barton::Barton now calls update_message to update the customized Barton help screen so the libraries can change it themselves. ** [lib-barton.cc, Help/helpscreen.barton] broke out Barton_Help_message into an include file for consistency with other variable which is being used with update_message (welcome_message), now calls update_message in method Barton::Barton. ** [interface.cc, Help/welcome.barton] renamed Help/Welcome.txt to welcome.barton so that the filename would be the same as when it was found on /mit/mitlibs (consistency). ** [Makefile] added new dependencies due to creation of savescr.h ** [other.h] removed redundant program_quit declaration. ** [interface.cc] Use_Library now clears the screen. ** [user.cc] modified User::working_message and User::flush so that they only do their thing when Barton is selected with VT100. ** [user.cc] modified User::Person_is_VT100 to make sure the terminal chosen always has the same userWindow as User does. ** [user.h, user.cc] created status_Person_is_VT100 to record the last call to Person_is_VT100. ** [interface.cc] switching to VT100 or hardcopy is now preceded by checks to see if the switch would be redundant. ** [hardcopy.cc] modified Hardcopy_from_VT100::insert to only printf a newline and to swallow carraige returns (although it still keeps internal track whenever a carriage return comes through. ** [lib-melvyl.cc] changed exactly what terminal emulations to request MELVYL for. ** [vt.cc] VT100emulator::write now returns number of characters written, as VT100_from_VT100barton::write does. It uniformly writes a single NL no matter what EOL combination is sent (CR, NL, CR+NL, NL+CR). It scroll the screen using curses when necessary. ** [interface.cc] Modify attempt_refresh to not display message at startup ** [interface.cc] Add parse_cmd to WaitForEnter and added variable dont_save_screen to track when not to save and restore screens (i.e. don't do it when a command was issued from the menu/help screens). ** [vt.cc] VT100emulator::write now does a refresh whenever it receives a "punctuation" character from the library (easier than timer-based). -------- cut here -------- ** [DO?] Find out what is wrong with Harvard ** [DO?] also, warn users who have huge xterms that this may cause problems and that they may wish to use the dash version instead if they are running X. ** [DO?] what about the "setup_X library" interface? ** [DO?] optimize the History object (using memcpy and optimizing Match?) ** [DO?] inline all methods which are called to move data from the GEAC to the screen and from the keyboard to the GEAC. ** [DO?] check the code which would handle user input when there is *NO* library open at all (i.e. after closing Barton after startup). ** [DO?] pty/tty acquisition?