OggSquish API Overview

This document is a practical description of a working draft for the libogg 0.98 library interface. Comments to xiphmont@mit.edu (especially from application developers!) are welcome.

Most recent update: May 8 1996


Working draft OggSquish library interface description

This file is a 'tutorial' style document describing a working draft of the OggSquish 1.0 library interface specification. It does not completely represent the specification nor describe in detail specifics of function usage. For programming purposes, it is meant to be used alongside the formal libogg specification documents.

This document is copyright (C) 1996 Xiphophorus. Xiphophorus and OggSquish are trademarks (tm) of the Xiphophorus company. All rights reserved.

This document, as well as the specification, preserved in its entirety without modification, may be freely distributed.


Interface use overview

OggSquish, strictly speaking, is a general purpose DSP engine designed specifically for use in signal compression. It is programmable via OggScript, and sample OggScript programs are distributed with the library. OggScript is not described in any detail here; that is left to OggScript specific documentation. Script programs can be passed to the library or loaded from a bitstream header.

After the engine has been properly set up by loading one or more scripts and setting any additional adjustments, such as specifying restrictions on memory usage, the engine can be used. The encapsulated engine functions on blocks of data, and passes back new data blocks after processing the input. Libogg does not perform any stream or file handling internally, but can return information to the application in a form directly useful for file or stream I/O.

Engine setup

The Ogg engine is set up in essentially two steps; the first step is to load an engine script, either from a string passed to the engine or from a bitstream header. In addition, if we are creating a new bitstream, we need to write an assembled script to form that bitstream's header.

The second step of setting up the engine is to tweak any number of settings available to the application that influence the operation of the OggSquish engine. These settings are chosen by modifying the contents of an ogg_settings structure (defined in libogg.h) and submitting it to the engine with ogg_settings_submit(). The settings may be changed at any time, but most will take effect only when resetting the engine or loading a new program script.

Several readable variables are also available to the application that allow it to properly set up its own behavior. Most notable among these are ogg_errno which reports error numbers (and ogg_strerr which points to a string version of the error). In addition, the application may query the output buffers to determine specifics about the output data (eg, during decompression, the application might want to know the output audio's sampling rate or number of channels). Output buffer information is available after a script or header is loaded.

Loading scripts from the application

Scripts may be specified by the application in either the compression or decompression cases, but a normal compression/decompression cycle specifies both the compression and decompression scripts during compression. This behavior is not enforced to allow clever uses of the Ogg engine, but the OggSquish bitstream specification states that every Ogg compatible bitstream contains a valid decompression script in the header which is normally specified when the bitstream is created as described below.

ogg_script_parse(char *program) is used to submit a compression or decompression script to the engine. An entire script need not be submitted at once, but my be submitted piecewise using multiple calls to ogg_script_parse. Once parsed, the script is then assembled by the Ogg engine, and the ogg_script_ready variable flag is set. ogg_script_use(int scriptnum) can then be called to 'burn' the program into the engine for immediate use (also setting ogg_run_ready), or calling ogg_script_makeheader(char **buffer,long *bytes) would return the script in header form for use as a complete bitstream header.

ogg_script_reset(void) will reset the parser at any time. An implicit reset is performed after ogg_script_use() and ogg_script_makeheader().

Loading scripts from a bitstream header

OggScript programs are not stored in a bitstream header as text; they are assembled and packed after parsing and stored in a header in packed form.

Submit headers to the engine by loading the beginning of a bitstream into a character array and passing it to the engine using ogg_script_loadheader(char *buffer,int bytes). The return value will be how many bytes of the input buffer was read; the next bitstream submission should begin at the first unread byte. Positioning and memory management of buffers submitted to and returned by the Ogg engine are up to the application. Submitted buffers should be allocated and freed by the application, and returned buffers should be freed by the application. The only exception to this rule are strings pointed to by global variables (such as ogg_strerr); these strings point to static strings in the Ogg engine. See Memory Management in the API spec for more details.

If the submitted buffer was long enough and the header was loaded successfully, ogg_errno will be cleared and ogg_script_ready will be set. The engine can the be burned in using ogg_script_use() or saved to a different bitstream using ogg_script_makeheader().

If the submitted buffer is not long enough, ogg_errno will be cleared and ogg_script_ready will also be cleared. In this case, ogg_script_loadheader() must be called again with another data block (the data already submitted is buffered; the next submission should be the next block of data, not a longer block starting at the original position).

The return value of ogg_script_loadheader() (the number of bytes read) will not necessarily be equal to the number of bytes even when the submitted block is not long enough; always check the return value to see where libogg expects the next block to begin!

If some error occurred during loading, ogg_errno and ogg_strerr will be set to appropriate values, and ogg_script_ready will be cleared. An error during the loading process implicitly resets the parser/loader. Any subsequent header (or script) submissions must start at the beginning.

Miscellaneous libogg engine settings

The engine settings described here are various changes the application can make to engine behavior that do not affect the bitstream. The bitstream behavior is specified entirely by OggScript programs. It is permissible to ignore these engine settings entirely; OggSquish will assume a default set and function normally without the application requesting changes.

ogg_settings_get(struct ogg_settings *s) is used to place a copy of the current engine settings into the passed structure. New settings are requested by submitting on ogg_settings structure to the engine using ogg_settings_submit(struct ogg_settings *s).

Settings are only guaranteed to take effect after a program change (ie, after a call to ogg_script_use()).

Currently, the only settings of interest are ogg_memory and ogg_precision. See a copy of the API specification for a description of the ogg_settings struct and these fields.

Engine usage

OggSquish operates on a frame-by-frame basis; the application sets up input data, calls the Ogg engine and then uses the output blocks returned by the engine. An audio stream is processed by repeating this process in a loop until termination.

The OggSquish engine is designed to be able to use different input streams simultaneously (for example, an input stream for raw audio and another for raw video). For this reason, using the engine is not as straightforward as passing a buffer into a function call and using a buffer returned from that call.

OggSquish input buffers are named; that is, the OggScript program is written to expect specific data to be available from certain buffers. Audio applications will typically only make use of the primary input buffers ogg_dsp_in, ogg_dsp_out, ogg_bit_in and ogg_bit_out (defined in both libogg.h and , by default, in the OggScript assembler). Note: Although flexible, this setup could be abused by the OggScript programmer to send data or read data from nonstandard places that applications cannot possibly expect. Buffers have intended uses that must be adhered to; the standard buffers and their uses are described in the formal specification.

If only one script (eg, decompression) is being used, then the application need not concern itself with multiscripting. However, the OggSquish interpreter in libogg is also capable of using more than one script as if multiple independent copies of the interpreter were available. The different interpreter 'banks' are selectable using ogg_run_set; care must be taken however, to be certain the applications is using the interpreter bank it intends to.

Querying buffer usage

After a program has been loaded and burned in, the application may query the engine to see which named buffers will be in use. ogg_buffer_used(int buffer_type,int buffer_number) will return nonzero if the buffer is in use [libogg.h provides defines for querying buffers by name, eg, ogg_buffer_used(ogg_dsp_in,ogg_primary)]. This feature need not be used; contents of unread active buffers will simply be discarded, and reading an inactive buffer will return a NULL.

Querying buffer information

Libogg provides query functions for determining information about input/output buffers, such as sampling rate dynamic range and interleaved channels in the case of audio (ogg_buffer_dsp). See the formal API specification for details about the buffer query functions.

Submitting data to the engine

Data is submitted to the Ogg engine by sending it to one of the Ogg I/O buffers.

Determining submission size

In some cases, it is possible to know the exact amount of data to submit to the engine; for example, if we are handling incoming packets, each of which holds a single frame for decompression, we can simply submit the exact frame to the compressed data input buffer (usually, ogg_bit_in/ogg_primary).

In most other cases, however, the application cannot determine by itself how much data to submit to the engine, for example when submitting raw data or input bitstreams in an unbroken file form. The application should query the engine using ogg_input_query(int buffer) to determine the safe minimum amount of data to submit to the buffer.

This value should be queried for each submission; submitted data that is not used in one frame will be buffered for use in the next. Submitting less than the requested amount of data (except in the case of EOF; more later) can cause a later error. Submitting more than the requested amount of data is safe, but the engine will need to allocate memory to buffer the excess; consistently submitting too much data will cause the allocation of more and more buffer space each frame. If the engine currently has enough data buffered, it could answer the submission query with a request for 0 values.

Any buffered data can be cleared from any input buffer using ogg_input_reset(); this is recommended between dealing with frames of packetized data that each store a single whole frame to avoid inadvertently losing synchronization in the case that a packet is the wrong length.

(NOTE: The submission size hint is actually set by the OggScript program and must be carefully chosen by the OggScript programmer).

Submitting raw data

Once the application has determined the amount of data to submit, a data block is sent to a named buffer with ogg_input_dsp() for signal data or ogg_input_bit() for compressed bitstream data.

The dynamic range of an Ogg bitstream is set in the OggScript program, however; submitting a 32 bit wide data array to a program explicitly using an internal resolution of 16 bits will not result in 32 bits of headroom. The data will either be scaled to fit 16 bits if 'scale' is set or truncated to fit 16 bits if scale is not set.

Submitting bitstream data

Bitstream data is always submitted in the form of a character string.

When dealing with prepacketized input data or any case where the exact blocksize is known, the exact amount of data may be submitted; otherwise submit the minimum recommended amount. If less than the minimum recommended amount is available for submission (for example, the very end of a non-packetized bitstream), it may be submitted and used; if the engine tries to read past the end of a submission, the engine will return an out of data error.

Managing buffer usage

Buffers sent to the engine in this fashion are copied and processed; the submitted buffer is not directly used by the engine after the ogg_input_xxxx() call. Buffers that are returned by ogg_output_xxx() are dynamically allocated by the engine. The application is responsible for managing and eventually freeing these buffers.

Processing a frame

Once data has been submitted to all of the necessary input buffers, the Ogg engine is called to process a single frame of data with ogg_run(void). The call will run a single pass, from beginning to end, of the controlling Ogg program, assuming there are no runtime errors.

After the call completes, the various output buffers will be ready for reading, and each input buffer's data request values will be updated. In addition, if the bitstream terminates with the processing of this frame, the ogg_run_ready flag will be cleared. (Note that when submitting the last available data to input buffers, the data may require more than one frame to completely consume; the ogg_run_ready flag should be checked to make certain that the bitstream successfully terminated).

Reading output buffers

Output buffers may be read via ogg_output_dsp() for raw data and ogg_output_bit() for bitstream data, analogous to ogg_input_dsp() and ogg_input_bit() above.

If left unread, an output buffer will be cleared during the next call to ogg_run.

Error handling

The ogg_errno variable should be checked after every Ogg library function request to assure that no errors took place. In a properly designed application, the only errors that should be possible are 'Out of memory' or a runtime error triggered by a corrupt Ogg bitstream.

(NOTE: Most advanced modern operating systems, such as SUN Solaris and Linus Torvald's Linux, will allocate pages of memory for malloc() calls even if virtual memory is exhausted; this results from a 'lazy allocation' scheme which is arguably correct behavior for an OS. In this case, memory is allocated, but when accessed results in a SIGSEGV; Ogg traps this SIGSEGV and returns it as a 'Virtual memory error'. Less advanced systems, such as MS Windows, tend to return NULL from malloc requests that fail due to insufficient memory.)

Ogg behavior after an error differs depending on the function call in which the error occurred; specific descriptions given in the formal function call description documentation. As a rule, the function calls other than ogg_run() simply abort and leave engine state unchanged.

If an error occurs during ogg_run(), the engine simply aborts processing at the point the error occurred (clearing ogg_run_ready and setting the error flag). This leaves the engine in an indeterminate state that should be reset with ogg_run_reset() or ogg_run_clear().

At the moment, there is no way to continue from the exact point of an error that occurs during ogg_run() processing. Arguably, this behavior is sufficient; the only cases in which a runtime error can occur would either make continuing impossible or result in at least some data loss:

Example libogg usage flowchart

Example audio compression:

It's implied to check ogg_errno and/or ogg_strerr after *every* function call. Not all of the above mentioned features are used below; the following 'flowchart' describes the necessary minimum usage.

Step 1: Engine setup

  1. Fetch default engine settings (ogg_settings_get)
  2. Modify setting appropriately (Manipulate ogg_settings struct)
  3. Submit settings (ogg_settings_submit)
  4. Submit compression script (ogg_script_parse) [Check ogg_script_ready to verify the whole script is parsed]
  5. 'Burn' in program (ogg_script_use) [Check ogg_run_ready to see if the script fully loaded]
  6. Submit decompression script (ogg_script_parse) [Again, check ogg_script_ready]
  7. Read decompression script as header (ogg_script_header)
  8. Send/save header out to bitstream
  9. Free header memory
  10. Query specific information about I/O buffers

Step 2: Submit data to buffers

For each input buffer (audio compression commonly uses only one):
  1. Determine submission size (if necessary) (ogg_input_query)
  2. Submit data if query nonzero (ogg_input_dsp)

Step 3:

  1. Process frame (ogg_run)
  2. Read output buffers (ogg_output_bit)
  3. Send/save output bitstream buffers
  4. If ogg_run_ready is set, loop to step 2.

Step 4:

  1. Exit, or save new header starting at step 1.


OggSQUISH, OggScript, OggLite, the Thor-and-the-Snake logo and Xiphophorus are trademarks (tm) of the Xiphophorus company. The OggSQUISH library and OggSQUISH applications by Xiphophorus are copyright (C) 1994-1996 Xiphophorus. All rights reserved.

DeskFish WWW Server / Address comments to: xiphmont@cs.titech.ac.jp