From penninfo@nisc2.upenn.edu Tue Jun 29 11:21:03 1993 Received-Date: Tue, 29 Jun 93 11:21:02 EDT Received: from NISC2.UPENN.EDU by noc4.dccs.upenn.edu id AA02445; Tue, 29 Jun 93 11:21:00 -0400 Return-Path: Received: by nisc2.upenn.edu id AA17865; Tue, 29 Jun 93 11:20:59 -0400 Date: Tue, 29 Jun 93 11:20:59 -0400 From: penninfo@nisc2.upenn.edu (PennInfo Client) Posted-Date: Tue, 29 Jun 93 11:20:59 -0400 Message-Id: <9306291520.AA17865@nisc2.upenn.edu> To: gschade@saul.cis.upenn.edu Subject: PennInfo Document: Techinfo Protocol Status: OR TechInfo Protocol 8/3/92 S. Thorne Revised 4/14/93 A. Radhakrishnan This document outlines the protocol to the TechInfo Server. Much of this is in flux, and therefore this should only be considered accurate as of this date. I. GENERAL INFORMATION What is Techinfo? ----------------- TechInfo is a public information service, that MIT uses as its campus-wide information system (CWIS). TechInfo is accessible from virtually every workstation on campus that has communications capability. Information available on TechInfo includes job listings, events calendars, the campus newspapers, course schedules and descriptions, publications, and weather reports. TechInfo provides - A single electronic tool to find public information. - "Around the clock" availability of information. - The ability to update information more frequently and more cheaply than paper versions. - The ability to access archived information, for instance, back issues of news papers. - Ability to publish information that is not available elsewhere, e.g., stock answers. TechInfo helps departments and Institute offices extend their services and reach wider audiences. For example, members of the community with disabilities have easier access to information via TechInfo. TechInfo is also available to people outside of the Institute. Browsing through TechInfo is a great way for new students and new employees to learn about the Institute even before they arrive. MIT offers TechInfo as a free service both to browsers and providers of information. TechInfo's success depends on the providers of information who are responsible for providing a broad range of up-to-date information and advertising to their customers that this information is available. To help the providers, TechInfo can produce reports that show all documents a provider has in the system, the last time a document was updated, and the number of times a document has been read (TechInfo does not attempt to identify who is reading what document.) A provider of information (PI) is responsible to keep information current and to publich new information in TechInfo. A PI can do this via an existing client application. For example, using the Macintosh, a PI can create new folders and documents, re-order menu items, modify or publish new documents, remove information, link (copy without duplicating a document) other documents to a PI owned folder and more. All that is needed is for a PI to have access to a client application and to have an account, which is given out by the TechInfo administrators. Techinfo's Features ------------------- Four types of objects for information retrieval: 1. Menus 2. Text documents 3. GIF images (Macintosh and X Window System clients only) 4. Access to an on-line directory based on UIUC CSO protocol (Macintosh client only). Five ways to search for information: 1. Browse through the menus. 2. Search for words and "keywords". 3. Find a likely starting point and use the "outline" or "path" commands to search down or up. Items (folders, documents, images) in TechInfo may appear in more than one place (folder). 4. Do Full Text Searching 5. Search by node identification or last modified date. These five browsing features extend the versatility of TechInfo. Browsing through menus helps you learn how TechInfo is organized and how the information is related. Searching via keywords lets you quickly find documents that are related to each other. The Provider Mode is one of the most important features of TechInfo. At MIT, all information in TechInfo is the responsibility of the owner (provider) of the information. How TechInfo Works ------------------ TechInfo has three separate parts: clients, servers, and the protocol that connects them, as shown in the diagram. The client supports the user interface and communicates with the server via the TCP/IP protocol. This client/server architecture relies on TCP/IP to transport the information over a network. ------------------------------ | Client | Server | ------------------------------ | TechInfo Protocol | ------------------------------ | TCP/IP | ------------------------------ At MIT, the server runs on a DECstation 5000 running Ultrix 4.2. TechInfo has also been compiled to run on other flavors of UNIX. TechInfo stores documents on NFS, AFS or local UNIX file systems as plain ascii text. The server is stateless for browsers of TechInfo. The server maintains state only when a client is in provider mode. The server retrieves and stores documents requested or sent by the client. The server updates a web (database) of information, used to maintain the structure of information and give the user information about objects in the web. This information includes the owner of an object, the last time it was modified, the tokens used for indexing, the display title, file location, and type of object (text, image). The web is built into the server and does not rely on any commercial database packages. Native clients let you have access to TechInfo from your workstation. This shortens the learning curve and allows interaction with other applications on that platform. At present, TechInfo supports the following clients: X Window System UNIX (character screen display) Macintosh DOS (character screen display) CMS A VMS (character screen display) client also exists but is not currently used at MIT. Other access methods of TechInfo are anonymous dialup and telnet. For example, if you have access to the Internet, you can access TechInfo by sending the command: telnet techinfo.mit.edu II. PROTOCOL The TechInfo protocol connects the client and the server. Getting Connected to the server ------------------------------- You can telnet to the TechInfo server. The standard Techinfo server port is 9000. The TechInfo server sends a banner (or message) back when a connection is first made. Issuing Commands to the server ------------------------------ Commands to the TechInfo server consist of one character followed by a colon with any number of parameters. Parameters are delimited by a ":". All command lines must end with a Carriage Return and a Line Feed (CRLF) or a Line Feed (LF). Each line of returned information is ended with a CRLF or LF. The end of the returned information (since it could be any number of lines) is signified by a "." on a line by itself (this line is also terminated with a CRLF). Therefore when you receive the sequence of LF.CRLF the server has finished responding and is waiting for its next command. All commands issued to the server should receive some sort of response. Return messages are strings made up of an error code, and a text message delimited with a ":". Each error code is numbered. A zero error code means that the last command was successful. Here is a list of the current messages and error codes for the TechInfo server: 0:OK 1:You are not authorized. 2:Incorrect username/password. 3:The server is busy with another provider. 4:You must first remove children. 5:Could not find the nodes to reorder. 6:Could not open this file for writing. 7:Not a document. 8:Could not write web. 9:Could not find a node. 10:The file path is incorrect. 11:Item already exists. 12:Could not locate the source. 13:Server did not understand the request. 14:This version not found. 15:Could not find server file. 16:The requested file cannot be accessed. 17:The requested file is a directory. 18:The destination file is a directory. 19:This is a public node. 20:Unknown output format type. 21:This function has been disabled. 22:Text search failed. 23:More nodes to go, continue. 24:Kerberos Error. For historical reasons, errors 7:, 16:, and 17: do not pop up with the numbers preceding the messages. You may also receive a 100: or a 101: message. A 101: precedes a message (such as the Banner Message) meant to give useful information. The client treats this as a message of the day. A 100: is a warning that the server is about to shut down soon, and it gives the user the opportunity to quit. Text Files ---------- Text files or documents must be standard 7 bit ascii in no more than 80 character rows. This is only a client enforced standard. Files are sent back and forth with only a LF (Line Feed) as an end of line character. Terms and abbreviations to note ------------------------------- 1) The term "node.id" refers to a server generated integer number that serves to identify the node. The node.id will stick for the life of the node. 2) The term "base.node.info" is a string of the following format: :::::<Source>:<Locker>:<Path> <node.id> is the node.id <Flags> is a long integer which should be set. (see flag description below) <Date> is an integer - a date in the Unix internal date format divided by 86400. The rest of the arguments are text fields. (see Restrictions) Here is an example of a <base.node.info> string: 23796:0:7618:protocol:Techinfo Protocol:admin:ti_data:/mit/ti_data/admin/ti.protocol 3) The term "links.string" is a string of node.ids seperated by commas. 4) The term "nodelist" is a list of nodes. The first line returned shows the number of items to expect, the nodes come seperated by a LF, in the format of: <level #>:<base.node.info> The Techinfo server may close any connections which have been inactive for some number of minutes. Clients should be able to reconnect. Restrictions ------------ Because colons are used as delimiters, it should be noted that colons (:) are not allowed in text fields. Commands -------- Angle brackets <> mean to substitute approriate value for this parameter. Square brackets [] mean that the parameter is optional. "0:OK" is the successful message returned. The commands are described in the format Command Name Syntax Description and Result <no arguments> means just that. Commands which do not need authorization: ----------------------------------------- Send node info. s:<node.id> This command sends node information from the server to the client. It returns: <base.node.info>:<links.string of parents>:<links.string of children> Send Text. t:<node.id>:<starting byte>:<max bytes> This command sends text from the server to the client. The <starting byte> for the beginning of the document is usually 0, but you can start at any byte you wish. You can use this command to read the text found under a particular node.id. The t: command returns an initial line of "<number> Total Characters:<number> sent: This document was last modified on <date>." It is followed by a CRLF and the text of the document from the <starting byte> for the <max bytes> Get version. v:[client]:[version #] If v: is used with no arguments it returns the current client version number as listed in the version number control file. (For the format of such a file, please see "Formats of Various Control Files") If arguments are specified it checks the client and version # against the one found in the control file. If they match, then the server returns it back. If it is not found, it returns a 14: error message, and this indicates that the client may be out of date. If it is found, but with a message, it returns the message to the client. Get list of servers. m:<no arguments> This command returns a 1st line with the number of Techinfo servers followed by the server list as in the server file. (For the format of such a file please see "Format of Various Control Files" below) Get source info. n:<node.id> OR n:<source> This command is useful if you have a node.id or a source and you are interested in knowing the complete source information of it. n: will return a line from the source control file (For the format of such a file, please see "Formats of Various Control Files"). Outline. w:2:<node.id>:<level> This command traverses the web below the level you are currently on. It will return a nodelist. Path. w:1:<node.id>:<level> This command traverses the web above the level you are currently on. It will return a nodelist. Quit. q:<no arguments> The Quit command closes the connection to the server. Help. H:<no arguments> The help command returns an a nodelist of the TechInfo Help menu preceded by the number of nodes returned. Choose nodelist format. O:<number> The capital O command allows you to choose the format in which you want nodelists to appear on the screen. If the <number> field is 1, then the nodelist is returned with the flag fields set to 0. If the <number> field is 2, then the nodelist returned will have the flag fields set. If the <number> field is 3, then the nodelist will be returned in a menu format. The following four commands work similarly in that they are searching commands. If no node.id is provided then they search the entire web. If a node.id is provided, they search below the node.id provided. Search Keywords. b:<string>:[optional node.id] The b: command will search for the <string> given in all the keyword fields of the nodelists in the web. If a node.id is provided it will search below that node.id in the web. It will initially return the number of nodes found followed by a nodelist of all nodes that contain the <string> given as a keyword. Find new nodes. I:<starting node.id>:<mm>:<dd>:<yy> This command finds all text nodes in the web modified since the date given. Use 0 for the <starting node.id> if you want to search the entire web. If a node.id is provided, it will search below that node.id in the web. It will initially return the number of nodes found followed by a nodelist of all text nodes modified since the given date. Return Nodes by Source. K:<source>:[optional node.id] This command will find all the nodes that belong to a particular source. It will initially return the number of nodes found followed by a nodelist of all the nodes that belong to that source. Perform Full Text Search. J:<string>:[optional node.id] This command will perform a full text search for the <string> provided. It will initially return the number of nodes found and a nodelist of nodes that contain the <string> being searched for. Commands which require Provider authorization: ---------------------------------------------- Provider Mode. p:<userid>:<password> This command is used to begin a provider session. It returns the default source for this user as found in the source file (For the format of such a file please see "Format of Various Control Files" below). Kerberized Provider Mode. P:<length of ktext>:<ktext> This command begins a provider session using kerberos. The <length of ktext> is in bytes. End Provider Mode. c:<no arguments> This command ends a provider session. The server will write out to the web if changes were made. Add a node. a:<base.node.info> This adds a new node to the web. The date field is filled in by the server with the date. It returns the node.id assigned. Replace a node. r:<base.node.info> This command replaces the base node information of a node, but leaves the links unchanged. The date field is filled in with the date. You must be the authorized source of the node. Delete a node. x:<node.id> Delete removes all references to the node. It will not allow a node with children to be removed. You must be the authorized source of the node. It also removes the file where possible. Link nodes. l:<node.id of the parent node>: <links.string of children nodes> Each node must be linked to the web. This command creates a link between the parnet nodes and the children. You must be authorized as the source of the parent node. Unlink node. u:<node.id of the parent node>:<node.id of child node> This command destroys a link. There is no checking done for orphans. You must be authorized as the source of the parent node. Reorder child links. g:<node id of parent>:<node id of position to displace>:<node id of child to move> This command tries to take one node.id and move it TO the position of another. It displaces all the children down to make room atthis position. You must be authorized as the source of the parent node. Reorder child links. j:<node id of parent>:<node id of position to displace>:<node id of child to move> This command tries to take one node.id and move it AFTER the position of another. It displaces all the children down to make room at this position. You must be authorized as the source of the parent node. Send a file. f:<node id> This command sends a file to the server for storage. You must be the authorized source of the node. It returns an "0:OK" and then expects to get the file with every line ending with a LF and a final line of just a "." and CRLF. Change source info. A:<source.info.line> This command is used to change the source information in the source.info.line. You must input the data for the <source.info.line> in the same format of sources found in the source file (For the format of such a file please see "Format of Various Control Files" below). Commands which require Administrator authorization: -------------------------------------------------- Begin an admin session. z:<password> This command begins the session for the administrator. Reload the web. i:<no.arguments> This command reloads the web from a disk. Saves the web. y:<no.arguments> The y: command saves the web to a disk. Current Connections. B:<no.arguments> This command is useful for the administrator in that it shows the current connections to the server. Server Shutdown. C:<number> This command shuts down the server, saves the web, does not accept connections, and shuts down when connections have been inactive for <number> of minutes. This command will return a 100: error message to the client so when a person tries to use TechInfo, he will be asked to close the connection. Change Banner Message. D:<new banner message> This command changes the Banner Message or message of the day on the TechInfo Server. To change it back to the default type D:<no arguments>. The client receives a 101: error message indicating that a message of the day is present. Set Last Modified Dates. E:<arbitrary node number>:<arbitrary node number> This command stats all files except those flagged and set the last modified date. Format of Various Control files ------------------------------- Here is the format of 5 control files mentioned in this document. msgs <msg#>:<text message> versions <machinetype>:<version#>:[message] servers 0:<port>:<server.id number>:<name>:<email>:<title of server>:<hostname>:<comment>:<pips> sources <source>:<long source name>:<contact>:<phone>:<email> providers <source>:<username>:<password> Command definition list in full -------------------------------- This command list can be found in /afs/net.mit.edu/dev/project/techinfodev/src/srv_ti/network.h #define T_ADDNODE 'a' /* add */ #define T_FIND 'b' /* fnd, find keyword*/ #define T_ENDPROVIDER 'c' /* epv, */ #define T_GETFILE 'f' /* fil, file, getfile */ #define T_REORDER_BEFORE 'g' /* reorder the child links */ #define T_SOURCE 'h' /* test of valid source for an id */ #define T_RELOAD 'i' /* reload the web from disk */ #define T_REORDER_AFTER 'j' /* reorder the child links */ #define T_ADDLINK 'l' /* lnk, link */ #define T_GET_SERVER_INFO 'm' /* get info on other servers */ #define T_SRC_INFO 'n' /* get full source info on a node */ #define T_TRYPROVIDER 'p' /* prv, provider */ #define T_QUIT 'q' /* quit */ #define T_REPLACENODE 'r' /* rpl, replace */ #define T_SENDNODE 's' /* get, show? node? */ #define T_SENDFILE 't' /* sfl, sendfile */ #define T_RMLINK 'u' /* ulk, unlink */ #define T_VERSION 'v' /* find current version */ #define T_TRAVERSE 'w' /* trv, web, trav, traverse */ #define T_RMNODE 'x' /* del, delete */ #define T_SAVEWEB 'y' /* save the web to disk */ #define T_ADMIN 'z' /* adm, admin */ #define T_CHG_SRC_INFO 'A' /* change the info about a source format is A:<source_info_line> */ #define T_SHOW_CONN 'B' /* show the current connections, an admin cmd */ #define T_SHUTDOWN 'C' /* shutdown the server, save web , dont accept connections, and shutdown when connections left have been inactive for n minutes */ #define T_CHG_BANNER 'D' /* change the banner msg */ #define T_SET_DATES 'E' /* stat all files except those flagged and set the last modified date*/ #define T_HELP 'H' /* send the help menu */ #define T_CHGD_SINCE 'I' /* send the non-menu nodes which have changed since a certain date*/ #define T_FULL_TXT_SEARCH 'J' /* Perform full-text WAIS search */ #define T_SOURCE_SRCH 'K' /* Return all nodes of a given source*/ #define T_NODE_FORMAT 'O' /* Choose nodelist format */ #define T_KRB_AUTH 'P' /* authenticate as provider using kerberos */ flags that only the server cares about #define N_ONSTACK 0x1 /* Tag Used for infinite loop protection */ #define N_CALLEDON 0x2 /* Already been passed to a traverse func */ #define N_RESOLVED 0x4 /* Have link node-ids been resolved to ptrs? */ #define N_FAKE 0x8 /* Created by web_node to keep things happy */ #define N_HIT 0x1000 /* Internal flag for WAIS search */ flags that both client & server care about #define N_TEXT 0x10 /* This is a document node */ #define N_SYSGEN_FN 0x20 /* The file name etc should be maintained by server */ #define N_IMAGE 0x40 /* The node is an image (GIF) at this time */ #define N_DONT_STAT 0x80 /* should we stat the file to update modify date of text files*/ #define N_SERVER_NODE 0x100 /* a link to another server - reserved */ #define N_MENU 0x200 /* this represents a menu item - reserved */ #define N_BINARY 0x400 /* this represents a binary item - reserved */ #define N_DONT_INDEX 0x800 /* Don't index this node for full text search */ Trademarks ---------- UNIX is a registered trademark of AT&T Bell Laboratories. ULTRIX is a registered trademark of Digital Equipment Corporation. AFS is a registered trademark of Transarc Corporation. NFS is a trademark of Sun Microsystems, Inc. TechInfo is a trademark of the Massachusetts Institute of Technology. X Window System is a trademark of the Massachusetts Institute of Technology.