TODO list: HIGH priority * Fix LYVerifyCallback. * SECURITY: add real randomness to key generation. * Rewrite the displays for all of the LYNXCERT:/ functions so that they actually look nice. * SECURITY: the private key database has a known string at the front. Is this a problem? MEDIUM priority: * Put real error messages into the key gen phase. * I'd like to have it so that I can clean out the private key database of keys which do not have corresponding certificates, on command. * It might also be nice if when resubmitting a KEYGEN tag, it didn't generate another one. LOW priority: * I imagine that the passphrase system is a little annoying when you have a NULL password. :) I think there'd be some weird interactions if you tried to cancel a LYchange_key_pw. * Track down how the keymap is displayed (for hitting 'k'). The command for bringing up the CERT_MAN page is messed up on that page. Blah. I'm too lazy. It involves renumbering stuff. I won't deal with it for the moment. * Replace all English strings with defines in _en.h. * Track bug down with display of form after KEYGEN tag? See http://www.entrust.com/breq2.htm * Further break down LYCert.c so the code is more manageable and the functions smaller. * Do some load testing. check out the cert_hash hash performance and maybe add some SSL session caching. * How about re-evaluating the trust of certificates when a new CA cert is downloaded? * When submitting certificate requests via KEYGEN, we have the problem that after the cert is downloaded, we'll just sit there, tempting the user to do it again. jis says: Personal message 11:55:55/Fri Aug 8 1997 From: jis on BLUEBOX-144.MIT.EDU I am doing a hack and sending a "reload" http header with a different URL as the "where to go" indicator. I did this on purpose so people would naturally go back to what they were doing. If I didn't do this, netscape would leave them on the "submit for your cert page" and some people might get confused and resubmit the form over and over. Completed items: + Limit the number of failed password entries to 3. + Debug selection of user certificates. [7aug97] Not completely to my satisfaction but it'll do for now. + Fix whatever stupid bug there is in the priv key management level. [6aug97]. It's fixed, but it's a big hack! + Display some sort of useful error if the SSL connect fails 'cause they wanted a certificate that we couldn't find. [5aug97] + The certificate display information is a bit, um, verbose, shall we say. perhaps this should be toned down some. [5aug97] + Do saving of cert_hash stuff --- Hrm. wanna make sure that Lynx_User_Cert_Dir is in the lookup. [31jul97] + Ability to change the passphrase on certs. [26jul97] + Handle application/x-x509-foo-cert MIME-types. [31jul97] + Want some sort of general way of redirecting from doing something to displaying some LYNXCERT:/ page and then back. Reinvestigate and document the ways LYShowInfo and LYDownload work. + Complete KEYGEN stuff. [15jul97, 0100] + KEYGEN page != CERT_MAN page. Give it a new title. [15jul97, 0200] + Track down that weird hang on exit when -trace'ing. [brlewis] ============== Things to do not entirely project related: * Check out Netscape internal storage format for certs and private keys. (For 3.x and 4.x). Get back to danw and the 98 reg team. * Do a patch to zero out the form post data. -- see reply from tzer...whatever. Design: * Pick a place to store our files. How about ~/.lynx? Substructure? See next item. [DONE. Use .lynx/certs. Other sub-structure is unspecified] * Pick a format for our files. - What do we need to store? How about these in separate dirs? + list of accepted top-level CA certs. + personal certs and private keys. - Do any formats exist for storing the data? One of the many dbm formats maybe. Or in separate files for easy direct use by SSLeay sample apps. That seems like a good idea. - Communicator supposedly defines a public format for their keys. This could be cool. (based on PKCS 12?) Unlikely to happen soon. Punt. [TENTATIVE SOLUTION: SSLeay uses PEM encoded hashes in a directory for ceritificate authorities.] * UI ================ RANDOM NOTES ================= Navigator doesn't deal with CRLs. http://digitalid.verisign.com/info_ctr.htm Hm. In Navigator, where is the randomness coming from? How about SSLeay? (for things like RSA premaster secret; see draft sec 5.6.7.1) Why? Why isn't FREE defined in some header file? Why? Using SSL: * Make a new context: SSL_CTX_new() * Make a new SSL connect? SSL_new() https://bozo.mit.edu/cgi-bin/certcgi?login We never generate PKCS#10 objects. For handling KEYGEN requests, we use SPKACs (http://home.netscape.com/eng/security/ca-interface.html) Netscape can handle DER encoded certs, PKCS #7 cert chains and a "Netscape Certificate Sequence". See http://home.netscape.com/eng/security/downloadcert.html Ok. We need to write a callback to catch connections to sites for whom we do not have a certificate. Callback will have to bring up menu screens and all that, to decide what to do. This might be painful. Also need a callback to prompt for password for RSA private key. Also need a way to decide *which* certificate/private-key to use. Check out stuff about verification depths. I know I read about this somewhere in the docs... looks like it calls the verify callback each time it verifies, all the way up the chain. The mitca.ca file is in DER format. See http://www.psy.uq.edu.au:8080/~ftp/Crypto/certs.html for information about the intended structure of our directories. Here's what we can do.... * Place the mitca.ca file in our certs dir. * x509 -inform der -outform pem -subject -issuer < mitca.ca > mitca.pem * Get the hash. x509 -noout -hash < mitca.pem * Make the symlink. ln -s mitca.pem e44425b2.0 * Test it! echo "HEAD / HTTP/1.0\n\n" | s_client -CApath . -verify 1 -host \ bozo -port 443 lynx config information. Read by LYReadCFG.c. Globals defined in LYGlobalDefs.h and initialized to default values in LYMain.c. In LYMain.c need to declare at top, FREE in free_lynx_globals, init it in main (from #define's in ../userdefs.h). If want personal rc override, see LYrcFile.c. How can we distinguish between personal certificates, site certificates and certificate authorities? How about... instead of directories, use files. We could have one file for CAs, one for sites and one for personals. (Note, for personals, we'd need to store corresponding private keys too! We can actually use this to determine if we have a personal cert.) Might it be easier to use PKCS#7? Well, we can definitely use files for CA and site certs... How is _statusline different from _user_message?? _user_message does a single argument subst into another string with a %s in it and displays it using statusline. Seems pretty dumb. --------- HTML TAGS ------- KEYGEN tag -- implement as a funky select? A KEYGEN tag will have a NAME and a CHALLENGE attribute associated with it. We will need to submit the SPKAC with the NAME tag. This is what we need to do, step by step, as soon as the form is submitted: * Figure out what key size the user requested, and generate an RSA key-pair. Keep this in memory. * Generate the SPKAC --- Hm. ca-interface.html isn't too clear but I believe ssleay can take care of this too. * b64 encode it. * submit the form. See HTML.c. (But it doesn't appear to actually be *handled* there. Just formatted.) Also see HTMLDTD.h in the libwww section. Well, select's are just funky sorts of radio or check boxes. We'll make keygen's a funky form of select. What we actually did: * Figure out what sort of processing happens when you have an OPTION_LIST. * Add appropriate tests so that KEYGEN tags act the same as OPTION_LISTs but keep a separate type (unlike SELECT/OPTION stuff which is essentially absorbed into INPUT, I think). * Add a chal_str to the submit button so that we can track the challenge string when needed later. The key size is handled through form submission --- the keygen tag will have behaved like SELECT and have a value of 1024, 768 or 512 which we atoi. * Wrap a check around submitform (change_form_.... in LYForms.c) to use our special LYSubmitKeygenForm handler. This handler creates a temp file, generates our key for us, displays information for the user, and then allows the user to resubmit the form to the actual correct site with everything in place, as needed. ========= Lynx forms. There's a global called form_post_data which is a char *. This is tracked by the documents in mainloop, and stuck into one of the documents (e.g. newdoc). This seems to only come from things passed in at start-up and not from forms we retrieve from the web though. Information is displayed about the type of link in mainloop, around line 946. (e.g. (Form submit button) Use the right-arrow or to submit.") Form submission handled in LYMainLoop at line 2003. If we activate a SELECT link, we go to 2082 and run change_form_link on the current link. This pops up the dialog box and lets us select whatever it should be... Hm. Text fields appear to be processed in LYForms.c:form_getstr, which is called by change_form_link. Ok. So this is where we need to put the KEYGEN stuff. ========== How to handle submissions? Hrm. Well, once we have a KEYGEN tag, we don't want to submit right away anymore. Instead, we want to generate a key, do the SPKI and stick that in place of the key size and forward that data on. Well, in the end, the only thing that matters is the name=foo pairs. So, we can dump all of the data into a temporary file, with hidden tags, and then do the submission later. We're going to need some way of getting the password... maybe as a form or something with a submit URL of LYNXCERT://SET-PASS/password? That seems more than slightly dangerous! Hm. Isn't there a way to read a string from the status line? HTPromptPassword. There is a built in 120 character limit... ------ MIME TYPES ------- Hm. What's up with the two HTInit's? There's one in lynx/src and one in lynx/WWW/Library/Implementation. Hm. Well. The one in the lynx source tree appears to load stuff from the .mailcap file(s) too. We appear to use a HTPresentation to specify an external command to use to handle a MIME type. We use HTSetConversion to specify a function to handle it. HTFormat.h is a useful documentation file (in WWW) --- it even has specs! extern void HTSetConversion PARAMS(( CONST char * rep_in, CONST char * rep_out, HTConverter * converter, float quality, float secs, float secs_per_byte, long int maxbytes )); typedef HTStream * HTConverter PARAMS(( HTPresentation * pres, HTParentAnchor * anchor, HTStream * sink)); We want to use extern void HTSetSuffix PARAMS(( CONST char * suffix, CONST char * representation, CONST char * encoding, float quality)); to set default suffixes for .pem and .der files. What should they be set to? application/x-x509-user-cert? or -ca-cert? pem is a 7bit type. der is binary? .ca should definitely be application/x-x509-ca-cert. There's also the case where we get a site certificate from the SSL_connect. What should we do if we don't recognize it then? Want to have our call back put something up on screen. ------ INFO PAGE ----- /* get info from an established connection */ SSL_get_session SSL_get_certificate SSL_get_SSL_CTX SSL_get_cipher SSL_get_cipher_bits SSL_state_string SSL_state_string_long Well. That's not going to work, is it. No way to get the session we used out of a context. Too bad. So, we're going to have to store it as we retreive the document. This will require modifying the ParentAnchor class to keep around information about the cipher and peername. This involves adding the hooks to LYShowInfo, changing stuff in GridText.c to include the new abstractions and then finally changing the lower level stuff in libwww. ------ MAKING CONNECTIONS ----- Modifications to make? * HTTP.c -- before SSL_connect, set SSL_SESS_CACHE_CLIENT (session.doc) [maybe also do some session counting.] low priority. * HTTP.c -- before SSL_connect, need to find a certificate maybe. Hrm. Of course, I don't know which certificate I want to send until after The server hello finishes. (maybe I should read the 2.0 SSL spec.) For certificate verifications in 0.6.6 int verify_callback(int ok,X509 *xs,X509 *xi,int depth,int error,char *arg); void SSL_CTX_set_default_passwd_cb(SSL_CTX *ctx,int (*cb)()); #define SSL_CTX_set_client_cert_cb(ctx,cb) int callback(SSL *s,X509 **x509, EVP_PKEY **pkey); Called when a client certificate is requested but there is not one set against the SSL_CTX or the SSL. If the callback returns 1, x509 and pkey need to point to valid data. The library will free these when required so if the application wants to keep these around, increment their reference counts. If 0 is returned, no client cert is available. If -1 is returned, it is assumed that the callback needs to be called again at a later point in time. SSL_connect will return -1 and SSL_want_x509_lookup(ssl) returns true Hm. Can I restrict as to whether or not to allow connections using export ciphers? Should there be warnings? We can enable specific cipher... To specify the ciphers to use, int SSL_CTX_set_cipher_list(SSL_CTX *,char *str); ----------------- We want to have a global list of certificates --- in memory all the time? That would require loading it all at start-up (or first use). OTOH, we could store information about parameters of the certificate in the cert file and just load it when we need that particular certificate. Unfortunately, there is the small problem that users can't modify the trust parameters of shared cert files. Thus, we want each user to have a config file which declares their preferences. Well. There's a LHASH in libssl, I think. It's probably a good idea overall to use the X509_name_hash as the key since it's entirely possible (and likely) that some things will have the same names. using the whole X509_NAME_oneline would probably be a pain after a while. Sample format: conf-file ::= (parameter nl)+ parameter ::= id nl trust-param nl id ::= '[' hex * 8 '].' digit trust-parm ::= 'trust' ws* '=' ws* (ALWAYS|NEVER|ASK) hex ::= [0123456789abcdef] nl ::= [\r\n] Let's use X509_name_hash as the hashing function? Though, apps/x509.c uses X509_subject_name_hash. Ooo. how about saving. Ick. How about initial initialization? How do I find out what directory a cert came from?? Do we want to save all certificates that we have loaded in memory? Only those for which we have established some sort of concrete trust value, that is different from parents. Hm.... I'd rather not modify SSLeay if I didn't have to, so I don't really want to modify the CONF stuff to allow adding stuff, though that's really what I need. If I use my own LHASH, I'll need to have a hashing function and a comparison function. But, I'll want to convert it *from* a CONF structure to my own lhash. ------ Messages. Lynx stores all messages in LYMessages_en.h so that they may be ripped out and replaced with another language. That probably means that any time we want to stick our own messages in there so that we don't get English SSLeay errors. sigh. ------- At each step in the verification, we determine whether or not the cert we are looking at was correctly signed by its issuer. If it is, then perform according to cert_trusted(xs). If it is not valid, then just disconnect. Do we special case depth 0? Well, invalid is invalid, no matter what. If a host is UNKNOWN, we must check its issuer. We may have a chain of CAs, some of which we trust, some of which we don't. Anything in a SHARED_CERT_DIR is automatically trusted ALWAYS, though subject to local change, of course. Hm. So we have a *lot* of possibilities! Lesse... - a self signed cert for a site. (weird.) - unknown issuer -- site may be known or unknown. If unknown, ask. If known either ask or deal as pre-specified. - site cert intermediately verified -- the CA may be always/never/ask. (Unknown is taken care of above.) - site is fully verified -- the site itself may be never/always/ask. Suppose a site is verified but along the way, one of the certs is untrusted. The site specific trust should override. I don't think this works too well.... Hrm. Then, there's a complete chain with no trust values anywhere. Of course, hopefully, some of these possibilities will be eliminated when we correctly initialize stuff, as opposed to the nothingness we have now. Tomorrow, I should think about this carefully and see what cases I really need to worry about. How to handle: * UNKNOWN - ALWAYS - ALWAYS - NEVER Let's have it set so that lowest wins, since that is the finest granularity we have. Hm. You can stop at any time in the verification process if you have a negative response, but not necessarily if you have a positive one. Hm. Would it be possible to dynamically configure it to call something in arg? Hrmmmm.... looks like we should try the store data in static vars. Hrm. Also, should try to find out if /CN field is required or not? --------------- LYNXCERT need to modify LYGetFile.c (Need to add LYNXCERT_URL_TYPE somewhere.) --------------- What prompts do we get from navigator... * connect to a site for which we do not have a certifier. Display lots of information about the certificate and encryption negotiated on a separate screen? NS gives option of yes, for this session no, never yes, forever and *then* asks about warning. what about? yes, always allow connections. yes, allow but warn yes, but ask again next time. no, never allow no, but ask again next time. * connect to a site for which we have asked to be warned (either b/c of site or certifier parameter) ---------------- Lynx main start-up loop. LYMain.c:main() 1. Do basic startup. Register clean-up handlers, initialize globals, read config files, load up MIME handlers, figure out what page to load first (and other stuff). Now parse command line arguments. Then parse rc file. 2. Configure various other things based on all the information retreived previously. 3. Run possible extra set-up for an exit immediately, and then go to mainloop. LYMainLoop.c:mainloop() In this file, we keep track of curdoc and newdoc, which are private documents. From LYStructs.h: typedef struct _document { char * title; char * address; char * post_data; char * post_content_type; BOOL safe; BOOL isHEAD; char * bookmark; int link; int line; } document; While newdoc is different from curdoc. Use getfile(document *) to retrieve each document. Once retrieved, manually deep copy newdoc into curdoc. Do stuff (display related?) Handle user keyboard input (i.e. commands!) ======== getfile( document * doc ) makes a shallow copy of some of the doc stuff into DocAddress WWWDoc. ======= How should we do certificate retrieval? For CA certs, we have mime-type x-x509-ca-cert. We want to put that into a file. So this is sort of like LYNXDOWNLOAD. Let's put it into a temporary file, and then ask lynx to bring up some pages with our questions. We will always need to store it into our database. Let's finish up the cert structure stuff first then. ----------------------------- How showinfo and LYDownload work. well, showinfo doesn't have its own URL scheme. So, what we do when we get the LYK_INFO key is to call showinfo and pass it newdoc so that showinfo can create some temporary file and point the next round of MainLoop at that file. On the other hand, LYDownload does have its own URL scheme. It uses this plus some other semi-global variables to pass information around. LYdownload_options does stuff similar to showinfo in that it is passed the address of newdoc.address so that it can set the place to go next time 'round MainLoop. (and also dumps stuff to a temp file.) Hm. On the other hand, LYHandleCookies does things a little differently! It makes an HTStreamStack and does puts to that. Why? Well, it must somehow be forced to. When we get LYNXCOOKIE, why do we call LYHandleCookies? It's handled similarly to LYNXIMGMAP, LYNXKEYMAP, and lynxcgi (why lower case?). Presumably some evil backend stuff handles it. Ok. These must be invoked somehow by network input and downloads and showinfo by commands? LYNXDOWNLOAD is only seen when you've actually gone and hit 'd' to cause a download. Hm. But, OTOH, LYNXCOOKIE comes from ^K. I don't see why it couldn't have been handled via somethign similar to showinfo. ----------------------------- How we should handle LYNXCERT:/ commands. KEYGEN -- toughie. We start from a form somewhere. And that's eventually submitted. We can take the information read from that form, put it in a temporary file somewhere (as all hidden stuff) do the questions, get the passwords, tell the user all the what not, generate the key, finish figuring out what we're going to send the server, and then submit a second form. TRUST -- bring up prompt for new level. CERTIFY -- Hrm. Add a callback which displays the certification chain and goes all the way up. DISPLAY -- use X509_print? Or some sort of derivative perhaps. BIOs might actually be useful! Cool. ADD-CA -- intermediate thing for handling application/x-x509-ca-cert. Maybe we can just punt that altogether and do something similar to showinfo here. Maybe copy some ideas out of LYHandleCookies so that we can prompt the user at the statusline for some yes/no answers. ADD-HOST -- This comes about during a connect to a host for which we have no prior information (and likewise for all of its preceding issuers). We need to present some text warning the user about the risks s/he is about to take and deal with that appropriately. ------------------------ When do we save certificates? If it is an application/x-x509-ca-cert, we eventually get passed through to LYcertsave_options where we prompt the user for stuff. If it is a host cert, it will have been retrieved from the SSL handshake and is handled by the verify callback. If it is application/x-x509-user-cert, we will handle it when our HTStream which ate the cert is _free'd. We just need to prompt the user possibly for the passphrase at this point. ------------------------ Private Key interface The RSA private keys for each certificate must be kept as secure as possible. They should remain in memory for as little time as possible and be encrypted on disk. We want all of the keys to be encrypted with the same passphrase. The following interface will be used to access private keys. /** * LYchange_key_pw * * This function prompts the user for a new passphrase * and rewrites the database using this new pp as a key. * * Returns 1 on success or 0 on failure. */ int LYchange_key_pw(); /** * LYadd_key adds a key to the database. * * Returns 1 on success or 0 on failure */ int LYadd_key( RSA *rsa ); /** * LYfind_pkey_for -- * * Find the appropriate private key for the given cert. * Returns NULL if not found, otherwise a pointer to the * requisite struct. */ EVP_PKEY *LYfind_pkey_for( X509 *x ); /** * LYdelete_key_for * * Find the appropriate private key for the given cert and * remove it from the database. * Returns 1 on success or 0 on failure. */ int LYdelete_key_for( X509 *x ); The private keys will be stored in an encrypted (IDEA or DES) file. When unencrypted, the file would be a series of PEM encoded RSA keys, unencrypted. Implementation details: * BIO -- This interface will make use of a BIO_f_cipher on top of a BIO_s_fd simplify manipulation of the data. (Hm. I wonder if the lack of buffering on the _fd will be a problem.) * Adding and deleting will require full rewrites of the file. We'll use a temporary file and read/write encrypted. * open -- We can use straight open() with O_CREAT and O_EXCL. This will allow us to use a fixed filename (and write to user's local space, which is hopefully not world-readable) without worrying about conflicts. (Note, there is a bug in Linux implementation of this on NFS filesystems. This may or may not be a problem. I'm going to punt on this one right now.) * Let's use PEM_{read,write}_bio_RSAPrivateKey. ========================= Ok. Let's rewrite the verify call back. We know that it will be called from the top down. We want: * Any priority on the bottom to take precedence. This means that we'll have to return "1" a lot --- possibly continually, until we get to level 0. How should we handle different kinds of errors? * Find out semantics for: - UNABLE_TO_VERIFY_LEAF_SIGNATURE - SELF_SIGNED_CERT_IN_CHAIN - DEPTH_ZERO_SELF_SIGNED_CERT - UNABLE_TO_GET_ISSUER_CERT - UNABLE_TO_GET_ISSUER_CERT_LOCALLY * Deal with other certificate failures better.