So, to begin with the Internet is essentially a whole bunch of computers connected together by wires which they can talk over. Now, the wires that we're interested in connect to something inside the computer called a network interface. A network interface can be an ethernet card, a modem or a number of other things.
Each network interface (usually) has a single IP address associated with it. An IP address is a set of four numbers (each between 0 and 255, with some restrictions) seperated by periods that uniquely identify that address on the Internet.
Now, to make our lives complicated, a single computer may have multiple network interfaces and therefore multiple IP addresses, but, in general, a machine has a single IP address. An IP address always is a specific interface on a specific machine (at any given moment). For the moment, it is convenient to pretend that a single IP address corresponds to a single computer in cases we look at.
Now, all direct communication between computers on the net uses these IP addresses. Human beings, however, prefer slightly more mnemonic names like 'www.mit.edu'. This is called a hostname. Some people will distinguish that 'www' is the hostname and 'mit.edu' is the domain name, but really 'www.mit.edu' is the Fully Qualified Domain Name (FQDN). Essentially though, something like 'www.mit.edu' is a hostname.
When you use a hostname, it gets looked up on a machine called a name server. Bascially, you give it a hostname and it tells you an IP address. Usually, a given hostname will always give you the same IP address back. This process of getting an IP address for a hostname is called name resolution. Often, however, multiple hostnames will end up giving back the same IP address. For example, the hostnames 'www.mit.edu' and 'anxiety-closet.mit.edu' will resolve to the same IP address, and hence, the same machine.
Additionally, it is common to refer to 'hosts' rather than hostnames. These are distinctly different. Usually when one says 'host', one means individual computers or IP addresses, as there are often multiple hostnames per host.
Now, how does a the web connect into all of this? Well, a web page is defined by a URL (which may have some inlined content such as images or Java applets, which have their own URLs). Most URLs (such as http, ftp and gopher) contain a hostname.
So, one of the hardest terms to define and keep everyone happy is the
term 'web site'. Generally, a web site is a set of pages that are
meaningfully linked together. Some people would prefer to define any
URL as a 'web site', though 'web page' is easily a better term for
this. My definition of a web site is all documents with urls
beginning with a unique hostname. That is,
http://www.mit.edu/people/mkgray/ and http://www.mit.edu/madlibs are
part of the same site, but a document http://web.mit.edu/ is a
seperate site.
Glossary
The terms listed here are fairly accepted definitions. Fuzzier, less
well defined terms are defined below in the section
on my terminology.