Path: bloom-picayune.mit.edu!snorkelwacker.mit.edu!americast.com!americast.com\!americast-post Newsgroups: americast.ieee From: americast-post@AmeriCast.Com Organization: American Cybercasting Approved: americast-post@AmeriCast.com Subject: SPECIAL REPORT/MEMORY - Fast computer memories Date: Wed, 4 Nov 92 10:20:19 EST Message-ID: <1046.art.1992Nov4.102019@AmeriCast.com> SPECIAL REPORT/MEMORY - Fast computer memories Ray Ng, Sun Microsystems Inc. Designers are searching for new DRAM technologies to reduce memory access time and so unleash computer performance If the price-to-performance ratio of computer systems is to keep improving, the gap in speed between processors and memory must be closed. Processors perform at their peak only when the flow of instructions and data from memory is fast and unfaltering. An ever-flowing stream is particularly necessary to reduced- instruction-set computing (RISC) processors, which have become very popular during the last few years. A heavily pipelined RISC processor can execute an instruction every clock cycle, demanding a lot of the memory system. Both superscalar processors, with their multiple functional units, and multiprocessor machines make even greater demands on memory systems. Nor is the central processing unit (CPU) the only consumer of memory bandwidth. Computers now are expected to be easier to use and more capable than their predecessors, and some of the new capabilities will require speedier memory. Examples include the rapid display of high-resolution graphics in true color, the recognition of speech and handwriting, recording and playing back video and audio, and, ultimately, support of a virtual-reality environment. The machines may also need buffer memory for messages moving over the multigigabit-per-second networks expected in the near future. These lofty I/O ambitions all involve processing and moving large amounts of data. On top of even faster CPUs, they will strain both memory capacity and memory bandwidth. But the accepted dynamic RAM (DRAM) architectures and solutions have been pushed to their limits. A basic change in architecture seems the only way to obtain an urgently needed increase in memory speed. The need for change has struck a number of chip makers, because innovative architectures distinguish a variety of recent high-speed DRAMs, which go by such names as synchronous, cached, and Rambus DRAMs. The newcomers may be usefully surveyed fro m a system perspective, to see how they may solve design problems, particularly with regard to main memory. MEMORY LINE. Till now, in the familiar stored-program computer described by von Neumann, the processor has been connected directly to memory (as well as to input/output). From this model, a hierarchical memory system has evolved in which a little, very fast memory is placed very close to the processor and fed by lots of slower memory farther away from the processor [Fig.1]. This hierarchy, which is used in almost all computer systems today, reflects one of computer design's truisms, "fast memory is expensive and slow memory is cheap." At the first level of the hierarchy are the processor's internal registers. Access to these registers is very fast because they are on the processor chip. However, their number is limited by the available chip area, or "real estate." At the second level, between the processor and slower main memory, is a cache--a small, very fast memory. The cache is loaded with copies of those blocks of data stored in main memory that the processor is most likely to want for the operation it is currently performing. (Typically, the block size ranges from 16 to 64 bytes.) If the processor finds the data it wants in the cache (referred to as a cache hit), then to the processor it will look as if main memory is as fast as cache. But if the processor does not find what it needs in cache (a cache miss), then the block containing the missing data must be brought in from main memory, slowing down the system. Caches usually provide a speedup because they exploit a general characteristic of programs: locality in space and time. Spatial locality indicates that if a location in memory is accessed, then others nearby will probably be accessed soon; temporal locality means that if a location in memory is accessed once, then it will probably be accessed again soon. One problem with caches is that, in order to be effective, they require very fast RAMs that run at about the same speed as the processor; and while static RAMs (SRAMs) can deliver the required speed, they are expensive. Also, caches must keep track of which memory blocks are in the cache and what their state is, and therefore require a special controller and a tag memory that add complexity and take up precious board real estate. All the same, caches are popular. It is possible, too, to build systems with more than one level of caching, using on- and off-chip memory. Many modern processors have on-chip caches, for both program instructions and data, that are closer than an external cache and so faster to access. But like the number of registers, the caches have to be small because chip real estate is limited and in many systems they are supplemented with an external cache. The internal cache is referred to as first-level cache and the external as second -level. The third level of the hierarchy is main memory itself. Main memory is used to store programs and data, and as a source of input and destination for output. Typically, this memory is much larger than cache and is constructed of DRAMs, which are slower than the SRAMs but also less expensive. The fourth level of the hierarchy is mass storage. Today magnetic-disk storage is ubiquitous. It is used to implement a technique called virtual memory, which fools the processor into thinking the main memory is much larger than is the case. With virtual memory, the processor's address space is divided into blocks of fixed size, called pages. Pages are much larger than cache blocks, usually 4-8K bytes. Disk bears much the same relationship to main memory as main memory does to cache. Pages are called from disk and placed in main memory when they are needed or returned to disk when they are not. As with cache, the principle of locality is basic. To maintain order in the system, a memory management unit (MMU) keeps track of which pages are in main memory and what their status is. As with a cache miss, performance falls off whenever a page is not in main memory when needed (a page fault). The penalty is, however, higher because mechanical disks are much slower than semiconductor main memory. But disks are very cheap, in terms of cost per bit, and can store vast amounts of data; hard disks today commonly store hundreds of megabytes, and the use of magneto-optical and optical discs capable of storing gigabytes is growing. Strictly speaking, there is a fifth level of storage, for data that will not be used for an extended period of time or whose importance demands its preservation. This archival storage often consists of magnetic tape; of course, removable magnetic and optical discs are also used for long-term storage of programs and data. This storage level has no impact on run-time system operation and so will be ignored for now. MAIN EVENT. Main memory is almost always implemented using DRAMs, which in both speed and price lag behind the SRAMs generally used for cache. DRAMs use one transistor-capacitor pair, referred to as a cell, to store one bit of information, while SRAMs use a four- or six-transistor flip-flop to store each bit. Because each DRAM cell is very small, DRAMs can be made very dense; the densest DRAM now available is a 16M-bit part, while the densest SRAM is about 4M bits. The per-bit cost of SRAM, depending on its speed, is 5-10 times that of DRAM. However, because the charge leaks away from the DRAM cell's capacitor, it must be restored by periodic refreshing. Also, the act of reading a DRAM involves transferring and sensing mere dribbles of charge; since each read operation disturbs the cell contents, it, too, requires that the data read be restored. For these reasons, DRAMs are not especially fast. A DRAM is built as a square or rectangular array of cells [Fig. 2]; to read or write data, the processor sends an address to the DRAM, which typically it multiplexes, supplying first the row address, then the column address. For currently available DRAMs, the time it takes a row address to access a cell is about 40-80 ns, for a column address, about 20-40 ns, and the precharge time is about 30-50 ns. Thus the cycle time (the minimum amount of time between memory accesses by the processor) is about 110- 150 ns. In contrast, the cells in small CMOS SRAMs may be accessed every 8 ns; larger SRAMs have a longer access time of about 15 ns. To raise their operating speed, DRAMs have a special operating mode that takes advantage of their internal row-and- column structure, known as page mode. In this mode, when an entire row (or a chip page, but not to be confused with the virtual memory page) is read into the sense amplifiers, the user can keep the row active and merely change column addresses to access all the data. As long as the accesses remain in the page, the DRAM can work faster. For current DRAMs, the page-mode cycle time (or minimum time between column addresses) is about 40-50 ns [Fig. 3]. SPEEDING UP MAIN MEMORY. A main memory system has three crucial attributes: size, latency, and throughput. Size is affected by density, or the number of bits that can be packed into a given area; the higher the density, the better. Latency is how long it takes for data to be delivered after it has been requested, and is closely related to a DRAM's access time; the shorter the latency, the faster the DRAM. Throughput is a measure of how much data can be delivered in a given period of time, and is closely related to the DRAM cycle time; a higher throughput means that a DRAM delivers more data per time interval. While the density of DRAMs has been quadrupling roughly every three years, neither their access nor their cycle times have improved as rapidly. Improving the latency and throughput of main memory is the focus of attention among memory system designers. Page mode may reduce latency and increase throughput, but only if there is a lot of sequentiality in the memory reference stream; for multiprocessor systems, this is not true. Caches do a good job of isolating the processor from relatively sluggish main memory, but there is only so much a cache can do. The time it takes to service a cache miss is directly related to the main memory's latency so, with increasing processor speeds, cache misses have an ever greater impact on system performance, even when the hit-to-miss ratio is high. BANKS AND BANDWIDTH. The most common method of increasing memory system throughput is interleaving [Fig. 4]. The idea here is to use not one, but several identical arrays of memory, or banks. In its simplest form, interleaving spreads out the memory addresses so that adjacent addresses occupy adjacent banks. The number of banks used, N, is a power of 2. If an address yields a remainder of 0 when divided by N (or address modulo N = 0), then it resides in bank 0; if address modulo N = 1, then it resides in bank 1, and so forth. The net effect of interleaving is to increase the memory bandwidth, or throughput, by a factor of N; N words are read each memory cycle instead of only one. The N words are stored in a register, freeing the memory banks to process the next access. If the memory addresses accessed are mostly sequential, this form of interleaving works well; if they are not, it does poorly. Another form of interleaving, while more complex, works better for nonsequential accesses. As with the simple case, memory is divided into N modules and addresses are assigned to banks in the same way. In this case, however, the memory controller takes on the additional responsibility of scheduling accesses. It looks at each separate request and schedules it in such a way as to make maximum use of the data bus. The controller's objective is to overlap operation between memory banks as much as possible. Even though interleaving is popular and helpful, it runs into several problems. For one, filling a wider bus may require too much memory. Also, to maintain the interleave, memory has to be upgraded in increments of N. Lastly, interleaving can result in bulky, complex memory systems. While all these techniques have been used to improve systems performance, they will not be able to cope with the faster processors in the offing. A new memory architecture is needed, and in addition, the physics of the interconnections must be kept in view. Designers of truly high-speed memory systems have to treat memory paths as transmission lines; their impedance has to be carefully controlled and paths have to be correctly terminated to reduce reflections. As for the parasitic capacitance and inductance of the I/O driver/receiver, the device packaging, and the printed-circuit board's traces, they have to be minimized and their effects taken into account. High throughput demands high transmission rates. Since traditional TTL- and CMOS-level drivers and receivers cannot achieve the speeds necessary, new I/O drivers with low-voltage swings, carefully controlled slew rates, and the ability to drive a terminated bus are required. In selecting a high-speed interface, a number of related factors have to be considered: the interface should be simple, effective, economical, widely supported, and easy to use, and should conserve power. As speed increases, so, too, does the significance of schemes for distributing clock signals. Sophisticated techniques will have to be used to control clock skew--the different times of arrival of the same clock signal at scattered points in the system. Also, the delay introduced by the clock distribution network within each chip has to be controlled. Both skew and delay should be kept as low as possible. New clock architectures should also be considered. Instead of distributing a central clock to all the chips in the system (global synchronization), it may pay to ship a copy of the clock along with the data (source synchronization). MORE OPTIONS. Several new high-speed DRAMs solve some of the foregoing problems. Synchronous DRAMs resemble conventional DRAMs, and so are merely an evolutionary step in DRAM technology. They differ from early parts in that all inputs and outputs are referenced (synchronized) to the rising edge of a clock pulse. Another difference is that, when a read operation is performed, more than one word is loaded into a high-speed shift register; these words are shifted out, one word per clock cycle. As a result, synchronous DRAMs can have very high burst rates. (Note that some early synchronous DRAM designs use an internally pipelined design that operates on a single word only.) A synchronous DRAM running at 100 MHz has four times the bandwidth of page-mode DRAMs. However, access times are no faster than for conventional DRAMs. Some synchronous DRAMs do have a "wrap" feature, for servicing cache misses. They deliver a burst of perhaps four or eight cycles of data, with the addressed word appearing first, followed by the remainder of the block, and then wrap back around to the beginning of the block. A memory system built out of synchronous DRAMs has a peak (ideal) bandwidth equal to the system's clock frequency multiplied by the number of data lines in the system's bus. While the bandwidth actually delivered will, of course, be less than this, the memory bandwidth is directly proportional to the clock frequency. To derive the maximum benefit from these DRAMs, therefore, a computer must be designed to run the memory system at a high clock rate. This requires very careful design, but the high burst rate of some synchronous DRAMs makes it possible to use narrower memory paths than would be required for conventional DRAMs. At present, some vendors are sampling parts based on a synchronous DRAM architecture, and the Joint Electron Device Engineering Council (Jedec) is preparing a standard for them that will be available to any interested party. ON THEIR OWN. The cached DRAM is a proprietary development of Tokyo's Mitsubishi Corp., from which samples are available. Because a cached DRAM has a small SRAM cache inserted between its external pins and an internal DRAM, accesses that hit allocation in the cache are much faster than typical DRAM accesses. Also, the cache-fill bandwidth is very high because the bus connecting the SRAM and DRAM is very wide. In system configurations where memory is connected directly to the processor, cached DRAM may replace the external cache. However, the memory controller must perform the functions of a cache controller and maintain a set of tag RAMs. Another proprietary scheme, Rambus from Rambus Inc., Mountain View, Calif., offers a complete and radical solution to building a memory system. Its specification describes the protocol, electrical interface, clocking scheme, the register set, device packaging and pinout, and board layout. Although its peak transfer rate is 500 megabytes per second, actual memory bandwidth is less because address, control, and data are all transmitted over the same set of wires. The bandwidth that is in fact achieved depends on two factors: the amount of data transferred during each transaction, and (since the scheme involves caching) the hit rate. The amount of overhead for each transfer is fixed, so bus efficiency increases with transfer size. Assuming a 100 percent hit rate, the peak read bandwidth for the 4.5M-bit part is about 360 Mbytes/s for 64-byte transfers; for the 18M-bit part, which is faster, it is about 400 Mbytes/s. Bandwidth decreases if the hit rate or transfer size decreases. What does Rambus mean for the system designer? Because each chip is an independent entity, very few are needed to build a memory system and, because each chip has built-in decoding and hit detection logic, the memory controller can be simpler. If the hit rate is high enough, it may even be possible to connect the processor directly to memory and eliminate the cache. A special Rambus interface, containing the I/O drivers, phase- locked loop, and miscellaneous logic, is needed to interface to Rambus. Since Rambus is theoretically a cookbook solution, the designer need not worry about the electrical interface and clock distribution issues; the details have all been taken care of. But because each device does much more than an everyday DRAM, Ram bus DRAMs may cost more. Rambus is a proprietary technology licensed to manufacturers of DRAMs and application-specific ICs (ASICs). Toshiba, Fujitsu, and NEC are currently sampling Rambus DRAMS. The Rambus interface cell is available from a few ASIC vendors, and Toshiba plans to make ASIC versions of the Rambus controller available. RamLink is an attempt to take some of the work done for the Scalable Coherent Interface (SCI) and adapt it for use as a DRAM interface. RamLink will specify the protocol, required registers, and the electrical interface, which is a differential interface. While RamLink does not specify device pinouts or board layout, it is in many ways similar to Rambus; the chief difference is that RamLink is based on a ring, rather than bus, topology. The interconnections between each RAM are therefore point to point--and point-to-point connections can run faster than bused connections. The disadvantage of a ring topology is that the request and reply packets must traverse the entire ring. Each node on the ring adds some amount of delay, so the latency c an be very high: RamLink allows up to 64 nodes. It is anticipated that the RamLink specification will be an IEEE standard (P1596.4) when work is complete. No vendors currently offer a RamLink part. BOTTOM LINE. Main memory is the single most expensive item in a computer system. The current cost of DRAM is about US $30 a megabyte. A typical PC today comes with about 4M bytes expandable to perhaps 16M bytes; a typical high-end workstation comes with about 32M bytes, and can hold up to 512M bytes. DRAMs are commodity parts whose price is driven by volume. Any DRAM enhancement must offer a clear benefit at a small price difference. Any DRAM solution that can eliminate the need for a cache, yet cost little or no more than conventional DRAMs, could be of use in personal computers, low-end workstations, and other single-processor machines. ABOUT THE AUTHOR. Ray Ng is a member of the technical staff of Sun Microsystems Inc., Mountain View, Calif., where he is involved with the development of memory systems for RISC-based workstations and future computer systems. ***************************************************************** [1] In most computer systems today, the total memory consists of a hierarchy of media. Passing from the top to bottom of the hierarchy, the density of the medium (the amount of data it can store per unit area) increases, while its speed in delivering data and its cost per bit decrease. Some new dynamic RAM (DRAM) technologies aim at simplifying this hierarchy by speeding up main memory to the point where the need for a separate, external cache is moot. ***************************************************************** [2] In a typical DRAM, such as the 4M-by-4-bit Toshiba Corp. part, the actual memory array in the lower right is accessed through paired successive row and column addresses. The row address causes the data in the row to be read into the sense-amplifier I/O gate from which the column address decoder selects the 4-bit word, or nibble, to be placed in the data-out buffer. ***************************************************************** [3] In a normal DRAM read cycle, data becomes valid only after both row and column addresses have been supplied. However, when all the data needed is stored in the same row, a page mode speeds matters by changing the column address only; after each change, the newly requested data is output and remains valid until the next column address is received. ***************************************************************** [4] Interleaving divides main memory into blocks on the basis of address modules (the remainder when the address is divided by the number of banks, N, where N is a power of 2); because the banks can be accessed in overlapping fashion, throughput can be greater than for one bank by a factor of N. In the simplest scheme, all banks are activated simultaneously by a common control line, their outputs are stored in registers, and the registers' contents are consecutively multiplexed onto the system bus. If data is not stored and accessed from consecutive addresses, a more complex form of interleaving [bottom] allows each bank to be controlled separately so that the sequence in which the banks deliver data to the system bus can be tailored to optimize throughput Copyright 1992, IEEE Spectrum. For more information, send-email to American Cybercasting Corporation (usa@AmeriCast.COM)