VEX "Video Extension to X" Todd Brunhoff $Header: protocol,v 3.4 89/08/02 14:47:44 toddb Exp $ | Copyright 1989, Tektronix, Inc. The spirit of this extension is to provide an X interface to the generally interesting aspects of displaying live video in windows, capturing graphics from windows and converting them to a video signal, and managing the network of connections to and from devices that may receive or produce these signals, such as video tape recorders. Of those people that have provided input, some disagree on the amount of functionality that should be provided in a server, particularly on control of video devices like VCRs. To ameliorate all parties, the protocol is designed so that the server may advertise many or few limitations on its capabilities. For example, a server may advertise that it has an input device, but that it has no control over it and looks like a cable TV channel; alternatively, it can say that it can control playback speed, direction, seeking, etc. This proposal introduces a new window type of VideoBuffer that may be used to access digitized video pixels. If the hardware cannot digitize the video signal, or if the user chooses not to use a VideoBuffer window, the video picture may be placed in an InputOutput window. A window servicing video input consists of three logical parts: the graphics which are rendered into the window; the video picture, possibly rendered onto a VideoBuffer window; and a translucent pixmap and blendmap that define the mixture of the video and graphics. If the hardware is capable, and the server expresses the capability, graphics and video may be switched (transparency) or mixed (translucence) on a pixel-by-pixel basis. Today, video input and output hardware has complex limitations and capabilities; not always do the capabilities scale in expected ways, nor do the limitations always make sense. The VEX extension can provide large amounts of information about these limitations and capabilities; and it is intended that the client, through the use of convenience routines in the Xlib implementation, will be able to easily determine what it can and can't do without the overhead of a round-trip request to the server. Terminology ----------- Active video base window (AVBW) A InputOutput window is considered an active video base window, or AVBW, when it is used as a parent of a VideoBuffer window or in a RenderVideo request. Specifically: - a visible InputOutput window without a VideoBuffer child window that is actively servicing a RenderVideo request. - a visible InputOutput window with a visible VideoBuffer child window, whether or not it is actively servicing a RenderVideo request. Active video region (AVR) See AVBW and VIR. An active video region are those pixels pairs in an AVBW that may be blended with a translucent pixmap and a blendmap. Specifically, it is either - the region of visible pixels in a VideoBuffer window, excluding the border, paired with the pixels directly beneath them in the InputOutput parent that would be visible if the VideoBuffer window was unmapped. - the region of visible pixels in an InputOutput window that is actively servicing a RenderVideo request; in other words, the VIR. Blendmap A blendmap is identical in structure to the core protocol's colormap type, and in fact, can be manipulated in the same way by most of the colormap-related requests. A blendmap is used with a translucent pixmap to determine the blended value of graphic and video pixels in an AVR. Each pixel in the translucent pixmap indexes a value in the blendmap which is then used to calculate the blended value of the video and graphic pixels. Although most video hardware has the equivalent of a one plane translucent pixmap, it has | no ability to indirect these bits through a programmable lookup table. | This corresponds to the StaticGray colormap model. More sophisticated | hardware may have more planes and a programmable lookup table akin | to a colormap. This blendmap, if it exists in the video input hardware, | matches the GrayScale colormap model. Hence, a blendmap will always be one of these two models. See translucent pixmap. Control atoms A VEX server may provide controls for various aspects of video input hardware, video output hardware and video devices. Each type of control has a unique ISO Latin-1 string interned by the server and these are provided by QueryVideo. The correspondence of strings and control types are described in a separate document, VEX Devices and Controls (not yet written). Device atoms Although each video device has a unique identifier, the identifier | does not contain any information about what kind of device it is. In VEX, unique ISO Latin-1 strings are interned by the server for each device type and these are provided by QueryVideo. The correspondence of strings and device types are described in a separate document, VEX Devices and Controls (not yet written). Full-motion video The display of video on the screen at a rate at least as fast as that defined by the encoded video signal. For example NTSC is defined as 30 frames per second and 60 fields per second. Still video A single, static frame acquired from a video signal. Translucent pixmap Most video input hardware is able to multiplex video and graphic pixels on a pixel by pixel basis. Although the actual hardware to accomplish this varies, it can be thought of as an additional plane in the frame buffer, a 1 bit signifying 100% video and a 0 bit signifying 100% graphic, or vice versa. More sophisticated hardware may (effectively) have two or more planes that describe intermediate mixtures of the video and graphic pixels, and in this form, it is also known as Alpha Channel. In VEX, the mixture of a graphic window with a video picture is defined by a pixmap referred to as a translucent pixmap, or just translucent. Each pixel in the pixmap corresponds with one pixel in the graphic window and one pixel in the video picture. The translucent pixel value is used to index a value in a blendmap which in turn is what is used to calculate the visible pixel. Because of hardware variations in the implementation of the translucent, VEX represents it as a write-only pixmap, updated only by a single request, UpdateTranslucent. See Blendmap. VideoBuffer VideoBuffer is a new window type in addition to the core | protocol's InputOutput and InputOnly. Its semantics are identical to that of an InputOutput window with possible restrictions on its visual type and that of its parent. In addition, a VideoBuffer's | parent window is always an InputOutput window which carries | with it an associated translucent pixmap and a blendmap. | Depending on the capability of the hardware, a video | signal may be rendered by analog circuitry directly onto the screen, or by digital circuitry into the workstation frame buffer, or even into a separate frame buffer or pixmap. When the hardware has access to digitized pixels, they can be obtained after specifying | a VideoBuffer window as the destination for a RenderVideo request. Video capture See video output. Video device An external device for producing, receiving or routing a video signal, regardless of the video signal format. For example, a | videodisc player or VCR. It is identified by a VIDEODEVICE id. Video digitization A process that converts a video image from an encoded signal into a digital form having a supported depth and format. Video input Video input in general refers to an incoming signal that is decoded | or digitized to produce a picture on the workstation display. In this document, video input refers to the digitizing or decoding hardware, and it is identified by a VIDEOIN id. Logically, a client will connect an output device, such as a video disk player, to a video input using the ChangeConnectivity request, and then use RenderVideo to direct which drawable the picture will appear in. Video input region (VIR) This is defined as the subset of pixels in an AVR that is actually being updated by a RenderVideo request, whether by actually digitizing pixels or by analog means. Video output Video output in general refers to signals produced by encoding workstation graphics. In this document, video output refers to the encoding hardware, and it is identified by a VIDEOOUT id. Logically, a client will connect an input device, such as a video tape recorder, to a video output using the ChangeConnectivity request, and then use CaptureGraphics to direct which drawable the graphics will be captured from. Video output region (VOR) This is defined as the set of pixels on the screen or in a pixmap that are being encoded by a CaptureGraphics request, whether it is accomplished by reading pixels from a framebuffer or by analog conversion. Types ----- VIDEOIN: 32-bit value (top 3 bits guaranteed to be zero) VIDEOOUT: 32-bit value (top 3 bits guaranteed to be zero) VIDEODEVICE: 32-bit value (top 3 bits guaranteed to be zero) VIDEOIO: VIDEOIN or VIDEOUT VIDEOID: VIDEOIN, VIDEOOUT or VIDEODEVICE When supplied to a request, VIDEOIN, VIDEOOUT, VIDEODEVICE must be one supplied by the QueryVideo request, else Match error. CONNECTION: [ source, destination: VIDEOID old-state, new-state: CARD32 ] The CONNECTION type is used by the QueryVideo request to describe a transition on the directed graph of adjacency-matrices (returned by QueryVideo) when the given source and destination are connected. The old-state and new-state element are indices into the list of adjacency-matrices. The two matrices they index describe the connectivity among video inputs, outputs, and devices before and after the source and destination are connected. CONTROL: [ name: ATOM id: VIDEOID event: BOOL format: { 8, 16, 32 } length: CARD32 ] The control structure is used to represent the capabilities of the "knobs" and "dials" on various video hardware. It is returned by the QueryVideo request. The name in a control is a control atom interned by the extension when the server starts. The id element identifies which VIDEOID the control affects. The event boolean is a hint as to whether requesting an event for the control would be meaningful. For example, some controls can be adjusted instantaneously, and others may take some large amount of time. The latter kind of control will be marked true, although events may be requested on all controls. The format and length precisely describe the structure expected in a ChangeVideoControls request. The length is the number of bytes expected in the value element of a SETTING structure whose id and name match those in the control. The format describes the arrangement of data cells so the server may swap them properly before interpreting them. The meaning of the data and the appropriate names for controls are described in another document, VEX Devices and Controls (not yet written). SETTING: [ event: BOOL name: ATOM id: VIDEOID value: LISTofINT8 or LISTofINT16 or LISTofINT32 ] This type is used to change the setting of a named control associated with a named VIDEOID. Only the ChangeVideoControls and QueryVideoControls requests utilize it. The event field controls whether a VideoControl event is generated on the named window when the control setting has actually been accomplished. The control to be changed is named by the atom and is associated with the device named by the id. The value must have a length in bytes equal to the length advertised in the corresponding CONTROL structure provided by QueryVideo, else Length error. FRACTION: [ numerator, denominator: INT32 ] FRACTIONRANGE: [ base, increment: FRACTION limit: CARD32 type: { Linear, Geometric } ] The fractions represented by FRACTIONRANGE depend on the type, and for each type, the members of each set are as follows: Linear: { base, base + 1*increment, base + 2*increment, ..., base + limit * increment } Geometric: { increment ** (base+0), increment ** (base+1), ..., increment ** limit } RECTANGLERANGE [ base, limit: RECTANGLE x-inc, y-inc: INT16 width-inc, height-inc: CARD16 type: { Linear, Geometric } ] The RECTANGLERANGE is ultimately used in the VIDEOGEOMETRY type below. It describes the possible geometries of sources and destinations for RenderVideo and CaptureGraphics requests. The set of rectangles described by a single RECTANGLERANGE are all permutations of four sets derived for x, y, width and height. The members of the sets are determined based on the type as follows. Linear: x = { base.x, base.x + x-inc, base.x + 2*x-inc, ..., limit.x } y = { base.y, base.y + y-inc, base.y + 2*y-inc, ..., limit.y } width = { base.width, base.width + width-inc, base.width + 2*width-inc, ..., limit.width } height = { base.height, base.height + height-inc, base.height + 2*height-inc, ..., limit.height } Geometric: x = { x-inc ** (base.x+0), x-inc ** (base.x+1), ..., x-inc ** limit.x } y = { y-inc ** (base.y+0), y-inc ** (base.y+1), ..., y-inc ** limit.y } width = { width-inc ** (base.width+0), width-inc ** (base.width+1), ..., width-inc ** limit.width } height = { height-inc ** (base.height+0), height-inc ** (base.height+1), ..., height-inc ** limit.height } For example, the RECTANGLERANGE whose value is [ base = {0, 0, 320, 240}, limit = {960, 784, 320, 240} x-inc = 16, y-inc = 1 width-inc = 0, height-inc = 0 type = Linear ] would describe a total of 47,100 {x, y, width, height} sets {0, 0, 320, 240} {16, 0, 320, 240} ... {960, 0, 320, 240} {0, 1, 320, 240} {16, 1, 320, 240} ... {960, 1, 320, 240} etc. {960, 784, 320, 240} PLACEMENT: [ frame-rate: FRACTION } source, destination: RECTANGLERANGE x-scale, y-scale: FRACTIONRANGE ] The PLACEMENT type is used in the VIDEOGEOMETRY type described below. Each placement describes the possible source and destination rectangles used in RenderVideo and CaptureGraphics requests, but restricted to the scale factors described by the x-scale and y-scale elements. For video input, the frame-rate element describes the maximum nominal frame rate that the hardware can achieve in a RenderVideo request using a source and destination rectangle derived from the other elements. For video output, the frame-rate element describes the maximum nominal rate at which frames can be encoded. It is important to stress that this is only a nominal rate and may vary due to other factors such as hardware contention. The x-scale and y-scale FRACTIONRANGE specifies the possible ratios of destination x pixels : source x pixels destination y pixels : source y pixels where the ratio is numerator (destination) divided by denominator (source). The source and destination rectangles derived must have width and height ratios described by x-scale and y-scale. For example, given a PLACEMENT of frame-rate = [ 30, 1 ] source = [ base = {0, 0, 10, 15}, limit = {639, 479, 640, 480} x-inc = 1, y-inc = 1 width-inc = 1, height-inc = 1 type = Linear ] destination = [ base = {0, 0, 320, 240}, limit = {960, 784, 320, 240} x-inc = 16, y-inc = 1 width-inc = 0, height-inc = 0 type = Linear ] x-scale = [ base = {0, 1}, increment = {2, 1} limit = 6 type = Geometric ] y-scale = [ base = {0, 1}, increment = {2, 1} limit = 5 type = Geometric ] This implies that the source rectangles may be any with the constraints that 0 <= x <= 639 0 <= y <= 479 10 <= width <= 640 15 <= height <= 480 | and the destination rectangles may be any selected from the earlier example. The combined rectangles are limited to those source-destination pairs where the fractions destination width destination height width-scale = ----------------- and height-scale = ----------------- source width source height can be reduced to one of the fractions from the respective sets width-scale = [ {1,1}, {2,1}, {4,1}, {8,1}, {16,1}, {32,1}, {64,1} ] height-scale = [ {1,1}, {2,1}, {4,1}, {8,1}, {16,1}, {32,1} ] And for this example, using any of these source-destination pairs, the hardware in this example can maintain a nominal 30 fps. VIDEOGEOMETRY: [ signal-frame-rate: FRACTION signal-field-rate: FRACTION signal-width, signal-height: CARD16 placement: LISTofPLACEMENT ] This type describes the attributes of video inputs and video outputs and is returned by the QueryVideo request. The signal-frame-rate and signal-field-rate describe the number of frames per second and fields per second in the signal format. For input, the rate describes the signal received; for output, the signal produced. For video input, the signal-width and signal-height describe the dimensions of the signal picture as if it were placed directly on the workstation screen; for video output, they describe the dimensions of a screen region whose pixels would map one-to-one to the output signal. It is important to note that the frame-buffer holding the video picture, if any, may not actually have these dimensions; but the numbers describe the extent of the source rectangle for video input, or the destination rectangle for the video output. Each placement element in the list describes its own range of values. The complete range of placement parameters is derived from the entire list, but parts of one element in the list may not be combined with another. So, for example, source and destination rectangles described in one placement may not be combined with scale factors described in another placement. RENDERMODEL: [ depth: CARD8 visual-id: VISUALID ] This type is used by QueryVideo to reference the depth and visual types used by the VEX extension to describe VideoBuffer windows, their parents, and translucent pixmaps and blendmaps. The visuals are described by the connection setup and/or the QueryVideo request. BLENDMODEL: [ render-model-index: CARD8 | blendmap: COLORMAP ] | The QueryVideo request uses this type to describe the depth, visual and default blendmap associated with each unique RENDERMODEL. The render-model-index is an index into a list of RENDERMODELs | returned by the QueryVideo request. Requests -------- QueryVideo wid: WINDOW ==> screen: CARD8 + video-depths: LISTofDEPTHS allowed-depths: LISTofRENDERMODEL vin: LISTofVIDEOIN vout: LISTofVIDEOOUT vdev: LISTofVIDEODEVICE inattr: LISTofVIDEOGEOMETRY outattr: LISTofVIDEOGEOMETRY video-input-models: LISTofBITMASK { Pixmap, Window, Buffer, Composite } video-output-models: LISTofBITMASK { Pixmap, Window, Buffer, Composite } blendmap-models: LISTofBLENDMODEL default-blendmap: COLORMAP | min-installed-maps, max-installed-maps: CARD16 device-atoms: LISTofATOM device-controls: LISTofCONTROL clip-size: LISTofRECTANLGE input-overlap: BOOL capture-overlap: BOOL io-overlap: BOOL adjacency-matrices: LISTofLISTofBOOL transitions: LISTofCONNECTION current-state: CARD32 time: TIMESTAMP Errors: Window This request returns information about video hardware for the screen associated with the specified window. The screen is returned as | the first element in the reply. | The video-depths specify what depths/visuals are unique for video input. These depths are unique from those returned in the X connection setup; the list may be null. The definition of this type is included in the core protocol's description of the connection information. Pixmaps are supported for each depth listed. The allowed-depths specify what depths/visuals are supported for use with video input, video output and blendmaps. These depths and visuals are selected from those listed in video-depths, and those provided by the X connection setup. The list is in no particular order. Depth 0 with a null visual id is always listed. The vin identifiers are a handle for the digitizing or decoding hardware where a video signal may be converted into pixels; the list may be null. For example, a list of two identifiers implies that the hardware is capable of simultaneously decoding two input signals, and depending on video-input-models, rendering them into a drawable. | The vout identifiers are a handle for the encoding hardware that may capture graphics to be directed to an external device; the list may be null. For example, a list of two identifiers implies that the hardware is capable of simultaneously capturing two areas on a screen or in a pixmap and converting them into a signal. The vdev lists the devices that provide signals to a video input or accept signals from a video output, such as playback or record. The list may be null and implies that the server has no control over any devices that may be attached to the video inputs or video outputs. The inattr list the attributes of the decoder or digitizer used by the video inputs. The entries in the list correspond one-to-one with those in vin. The outattr list the attributes of the encoder used by the video outputs. The entries in the list correspond one-to-one with those in vout. The list of video-input-models should be interpreted as a two dimensional array, column moving fastest and represents the relationship between the VIDEOIN resources and the set of depths/visuals used to create VideoBuffer windows and AVBWs. The columns are labeled left-to-right with the list of allowed depths, and the rows are labeled top-to-bottom with the list of video inputs. Each cell is a bitmask containing zero or more true bits. Window is asserted if a potential AVBW for this video input can be created using the intersecting depth/visual. If a video input has no Window entries in its row, it implies that the corresponding video input may not be rendered onto any window. Buffer is asserted if the hardware can digitize the video signal in a VideoBuffer window with the intersecting depth/visual. VideoBuffer windows for a video input must be created using a depth/visual whose | whose intersection has a Buffer bit set. If a video input has no Buffer | entries in its row, it implies that the video input may not be rendered onto a VideoBuffer window, and therefore, there are no digitized pixels available to VEX. Conversely, if Buffer is asserted at least once in a row, there is guaranteed to be at least | one Window assertion in the same row. If there are no Pixmap entries in its row, then the video input may not be rendered into a pixmap. There is guaranteed to be at least one Window or Pixmap | assertion in every row. If Composite is asserted in a row, then Window and Buffer must be | asserted somewhere in the same row, and graphic pixels and video | pixels occupy the same frame buffer. (If Composite is asserted, it | will be asserted for the entire row) In this case, if pixels are | copied from an AVBW window, the pixels from the AVR that are not | 100% graphic are undefined. Likewise, pixels copied from a | VideoBuffer window that are not 100% video are undefined. If Composite is not asserted, but Window and Buffer are specified somewhere in the row, then graphic and video pixels occupy separate frame buffers. In this case, pixels copied from an AVBW, are guaranteed to be graphic pixels, even in the AVR, unless occluded | by other windows of the same depth and visual. Pixels copied from a VideoBuffer window are guaranteed to be pixels from that window | unless occluded by other windows of the same depth and visual. For example, the following array means that an AVBW for video input #1 must be depth 12; The AVBW may have a VideoBuffer child of depth 24; and pixmaps of depth 24 can be used as a destination for a RenderVideo request. Video input #2 can only support pixmaps of depth 12. depths 12-bit 24-bit ------------------------------ | vin 1 | Window Buffer,Pixmap Video in | vin 2 | Pixmap 0 The list of video-output-models should be interpreted as a two dimensional array, column moving fastest, and represents the relationship between the VIDEOOUT resources and the set of depths/visuals that each may capture. The columns are labeled left-to-right with the list of allowed depths, and the rows are labeled top-to-bottom with the list of video outputs. Each cell is a bitmask containing zero or more true bits. For example, if the video hardware has two video outputs, one able to capture 12-bit regions from windows or buffers, and one able to capture 24-bit windows or pixmaps, then the matrix would look like: depths 12-bit 24-bit ------------------------------ | vout 1 | Buffer,Window 0 Video out | vout 2 | 0 Window,Pixmap | If Window is asserted, then a video output can capture the intersecting depth visual in an InputOutput window, including the border. If Buffer is asserted, then a VideoBuffer window with the intersecting depth visual can be captured, including the border. If Composite is asserted, then Window and Buffer must be asserted, and AVBWs will be captured after their primaries are mixed according to each associated translucent pixmap and blendmap. (If | Composite is asserted, it will be asserted for all the entire row) | If Composite is not asserted, but Window and Buffer are specified, | then capture from an InputOutput window will never include pixels | from a VideoBuffer window, and regions of the source that are | occluded by a VideoBuffer window are undefined: it is hardware | dependent. Likewise, capture from a VideoBuffer window will never | include pixels from non-VideoBuffer windows, and regions of the source | that are occluded by an InputOutput window are undefined; it is hardware | dependent. If Pixmap is asserted, then a pixmap with the | intersecting depth can be captured. Each structure in the list of blendmap-models describes the visual | and depth for the blendmap and translucent pixmap supported by the corresponding video input; it also lists the default blendmap for | the video input. Each structure in the list contains an index | into the allowed-depths list, and the nth blendmap-model corresponds to | the nth video input. This implies that there is only one blend | model per video input, even though it is possible (if not probable) | to express multiple window and buffer depths. The indexed visual | type will always be either StaticGray or GrayScale. The indexed | depth may be zero, and implies that graphics cannot be mixed with video, | although the hardware may still be able to clip a video picture at a granularity more coarse than a pixel. It also implies that a VIR displays all video, else all graphic. Most hardware will support a | single StaticGray blendmap of depth one with two colormap entries. | The default-blendmap is the one initially associated with the root window for this screen. Clients with minimal blending requirements creating InputOutput windows of the same depth as the root may want to allocate from this map by default. Min-installed-maps specifies the number of blendmaps that can be guaranteed to be installed simultaneously (with InstallBlendmap), regardless of the number of entries allocated in each map. Max-installed-maps specifies the maximum number of blendmaps that might possibly be installed simultaneously, depending on their allocations. Multiple StaticGray blendmaps with identical contents but differing in resource ID should be considered as a single blendmap for the purposes of this number. For the typical case of a single hardware blendmap, both values will be 1. The device-atoms correspond to the video outputs and are server-dependent strings interned at startup. They are intended to be a unique identification for device types and are described in a separate document, VEX Devices and Controls (not yet written). The device-controls describe the controls, if any, for each VIDEOID. The order of the list will, in general, match the vin, vout and vdev, although the number of controls for any one particular VIDEOID is arbitrary. The list may be null. The clip-size list corresponds to the list of video inputs, and describes the smallest rectangle that can be used to clip a VideoBuffer window partly occluded by siblings or children. If either the width or the height are 0, then the entire window must be displayed and cannot be clipped. This is meant as advisory for clients and window managers as they choose window placement. If the input-overlap element is false, then VIRs may not overlap; true otherwise. This element is guaranteed to be false if the number of video inputs that can be displayed in a window is zero or one. The capture-overlap element is false if the hardware does not allow VORs to overlap; true otherwise. This element is guaranteed to be false if the number of screen regions that can be captured and directed to video output is zero or one. The io-overlap element is false if the hardware does not allow a VIR and a VOR to overlap; true otherwise. Note that io-overlap may be true, but video-output-models determines if a VIR may be captured or not. In all cases, if clip-size, input-overlap, or io-overlap constraints are violated, the content of the violated regions is undefined; if capture-overlap or io-overlap is violated, then the output signal for violated regions are undefined and is hardware dependent; no errors are returned to the offending request, but a VideoViolation event may be generated. In addition, other constraints that VEX cannot express may be violated, and a VideoViolation event may be generated. Requests that may violate constraints are RenderVideo, CaptureGraphics, MapWindow, UnmapWindow, MapSubwindows, UnmapSubwindows, ConfigureWindow, CirculateWindow, DestroyWindow, DestroySubwindows, ReparentWindow. The adjacency-matrices should be interpreted as a list of two dimensional arrays, all the same size, each with column moving fastest. Each array represents a possible set of connections among video input, output and devices. The columns in each individual matrix represent sources, and are labeled left-to-right with the video devices followed by the video outputs; the rows represent destinations, and are labeled top-to-bottom with the video devices followed by the video inputs. This implies that no video input may be a source, and no video | output may be a destination. The cells in an array are true if | the intersecting source and destination are connected as such; false, otherwise. The list of arrays is an unordered list of nodes on an directed graph. The transitions list provides the edges to the graph. Each entry describes a single transition between two nodes given the connection of a source and destination. The nodes are adjacency matrices in the list, indexed by the old-state and new-state elements of the CONNECTION type. Each edge described is bi-directional; i.e., disconnecting the same source and destination while in the new-state will transition back to old-state. Also, other connections or disconnections may be implied by a single transition, and these edges will also appear in the transitions list. For example, if there are four matrices and four transitions returned by QueryVideo srcs srcs dev dev out dev dev out #0 1 2 1 #2 1 2 1 -------------- -------------- dev1 | F T F dev1 | F T F dst dev2 | F F F dst dev2 | F F F in1 | F F F in1 | T F F srcs srcs dev dev out dev dev out #1 1 2 1 #3 1 2 1 -------------- -------------- dev1 | F T F dev1 | F T F dst dev2 | F F T dst dev2 | F F T in1 | F F F in1 | T F F { out1, dev2, 0, 1 } { dev1, in1, 0, 2 } { dev1, in1, 1, 3 } { out1, dev2, 2, 3 } This states that device 1 and 2 are always connected as destination and source, respectively; output 1 may be connected to device 2 at any time; device 1 may be connected to input 1 at any time. The current-state element describes the present state of connection; i.e. which matrix in the adjacency-matrices represents the current connectivity. VideoConnectivityState events are guaranteed to be delivered following each change caused by a | ChangeConnectivity request. The time element is used to prevent race conditions between the delivery of a VideoChange event and the use of some other VEX requests. The value of the timestamp is constant until there is a change in the list of video devices, and is then replaces with the current server time. ChangeConnectivity time: TIMESTAMP old-state, new-state: CARD32 Errors: Match, Value The time specified must be later than the time returned by the last VideoChange event, else Value error. The state of the network transitions to new-state if the current state index is old-state. If the current state is not old-state, then no transition is made and a Match error is returned. However, if the connectivity of the network is already in new-state, no error is returned and the current connectivity is maintained. Moving from old-state to new-state may require several transitions along the directed graph of adjacency-matrices. CreateColormap < this is a core protocol request > This request is used to create blendmaps for the VEX extension. Identical to the core protocol's colormap, blendmaps are created with CreateColormap using a visualtype appropriate for blending. These visuals are described by the QueryVideo request, and will be selected from the color models GrayScale and StaticGray. All other core protocol colormap requests can operate on the blendmap (FreeColormap, CopyColormapAndFree, InstallColormap, UninstallColormap, ListInstalledColormaps, AllocColor, AllocNamedColor, AllocColorCells, AllocColorPlanes, FreeColors, StoreColors, StoreNamedColor, QueryColors and LookupColor). However, since the created colormap may use a visual and depth only applicable to the blending hardware, it may be impossible to create a window that has a visual and depth that match. To affect the blending of video and graphics, InstallBlendmap, UninstallBlendmap and ListInstalledBlendmaps must be used. CreateWindow < this is a core protocol request > If the VEX extension is present, this request will accept a new window type: VideoBuffer. It creates a window whose semantics are identical to an InputOutput window with one exception and two additions. The exception is that although a VideoBuffer window may be reparented, the new parent must have an acceptable visual and depth, as advertised by the QueryVideo request. The additions are that the VideoBuffer window may hold digitized video pixels and it can be blended with its InputOutput parent. InputOutput windows effectively have two new attributes, a translucent pixmap and a blendmap. The blendmap and the translucent define the blending of pixels in an AVR. When an InputOutput window is used in a RenderVideo request, then its | blendmap and translucent are used. When a VideoBuffer window is viewable, the InputOutput parent window's blendmap and translucent are used. When an InputOutput window is created, the blendmap is set to the default blendmap for the window's visual and depth; and in effect, a translucent pixmap is allocated whose extent is precisely the same as the window, whose origin is coincident with the window's, whose depth is the same as the blendmap, and whose pixel values are | initialized to 0. Note that only an AVBW's blendmap and translucent | are ever used, so the server may never actually allocate any space for | translucents and blendmaps in non-AVBWs. The translucent pixmap and blendmap define the mixture of graphic and video pixels on a one-to-one basis. The translucent equally affects the visible red, green and blue primaries for a pixel. The pixmap corresponds one-to-one with each pixel in the window with their origins coincident. The state of each pixel's r, g and b in the AVR are calculated as follows: dvalue = alpha * gvalue + (1-alpha) * vvalue where bmap[ tpixel ] alpha = -------------- 65535 | and dvalue is the displayed (visible) primary gvalue is the graphic primary from the InputOutput window vvalue is the video primary from the VideoBuffer child if present, otherwise the video picture itself tpixel is the pixel in the translucent pixmap bmap is the array of red protocol-intensities in | blendmap for the InputOutput window | When an InputOutput window becomes an AVBW, and a pixel in the AVR changes from zero graphic to non-zero graphic, and the graphic | (AVBW) pixel has not been retained by the hardware or the server, | then the following happens: the graphic pixel is treated as if a | ClearArea had been performed on the pixel. If possible, expose | events for all exposed pixels in the window will be coalesced. | When an InputOutput window becomes an AVBW, and there is a VideoBuffer child, and a pixel in the AVR changes from zero video to non-zero video, and the video pixel (in the VideoBuffer window) has | not been retained by the hardware or the server, then the following | happens: the VideoBuffer pixel is treated as if | a ClearArea had been performed on the pixel. | If possible, expose events for all exposed pixels in the | window will be coalesced. If a RenderVideo request is active, digitization for that pixel will begin immediately. Graphics may be drawn on a video buffer, but if the window or buffer is servicing an active RenderVideo request, the results are undefined. If for some reason the translucent pixmap contents are lost, | the affected regions will be filled with 0 and expose events will | be generated on the window (even if the same regions in the window are | not damaged). If the window is resized, the translucent pixmap is | also (effectively) resized, and bits in the translucent follow the | BitGravity of the window, if possible, or they will be filled with | 0 and expose events generated. | ChangeBlendmap win: WINDOW bmap: COLORMAP Errors: Window, Colormap, Match The named window's blendmap is changed to bmap. The window must be of type InputOutput, else Match error. The blendmap must have the same visual type as the window, else Match error. RenderVideo source: VIDEOIN destination: DRAWABLE src-x, src-y, dest-x, dest-y: CARD16 src-width, src-height, dest-width, dest-height: CARD16 full-motion: BOOL priority: CARD8 subwindow-mode: { ClipByChildren, IncludeInferiors } Errors: Drawable, Match, Value The destination must be a valid drawable id, else Drawable error. The specified region of the source video signal is decoded and directed to the destination. The visible portion of the destination rectangle becomes a VIR. In general, it is intended that when an InputOutput window is the + destination, the pixels of the video picture are inaccessible and + are magically placed "over" the (graphic) window without modifying + its pixels. When the destination is a VideoBuffer window, the + intent is the same as above, except that the video pixels become + accessible. However, not all hardware supports this concept and + pixels may migrate between domains, become intermixed, and + sometimes undefined because of a change in depth or visual. This + asymmetry is defined where possible and is described here and in + the description for QueryVideo. + + If the drawable is an InputOutput window, then Window must be asserted in the video-input-models returned by QueryVideo for the intersecting visual/depth of the window and the specified video input, else Match error; the window must not have a VideoBuffer child, else Match error. If the destination is a VideoBuffer window, then Buffer must be asserted in the video-input-models for the intersecting visual/depth of the window and the specified video input, else Match error; the parent window must have Window asserted in its intersecting visual/depth for the same row, else Match error. If the destination is a pixmap, then Pixmap must be asserted in the video-input-models for the intersecting depth of the pixmap and the specified video input, else Match error. If the window is visible, then the visible AVR pixels are displayed according to the associated translucent pixmap and current installed blendmap. If the destination is an unmapped VideoBuffer window, then the state and appearance of the parent window's visible pixels will not be affected, but the VideoBuffer window's pixels may be rendered, depending on the value of backing-store for the window. The src-x, src-y, src-width and src-height describe the extent of the source rectangle in the video picture. The dest-x, dest-y, dest-width and dest-height describe the rectangle in the destination drawable to receive the video picture. The source video picture will be scaled by dest-width dest-height x scale = --------- y scale = ----------- src-width src-height The source and destination rectangles and scaling factors must match one of those provided in a single PLACEMENT by the QueryVideo request for the specified video input source, else Value error. If | the destination rectangle is valid, but some or all of it is | outside the extent of the destination drawable, the result is | clipped to a granularity advertised by QueryVideo, and a | VideoViolation event may be generated. If the source rectangle is | valid, but some or all of it is outside the extent of the video | picture, the result in the corresponding region of the destination | is undefined: it is hardware dependent. | The full-motion element determines whether the video input signal is continually updated or not. If it is true, then the server will try to achieve the maximum nominal rate advertised by the QueryVideo request for the specified scale factor and placement. If it is False, then a single frame will be rendered. It is just as if the RenderVideo request was immediately followed with a StopVideo request specifying the same drawable. However, a full-motion element of False is an aid to the server to avoid extra digitization. Priority is a value (0 to 100 inclusive, else Value error), representing the percentage of video bandwidth that the server should try to preserve. A value of 0 tells the server that graphic output to the screen should have top priority and that the video decoding may occasionally miss some incoming or outgoing frames. A value of 100 gives video decoding top priority meaning that the graphics performance may be degraded. It is advisory only and does not guarantee anything. Subwindow-mode controls the treatment of the window's children. For ClipByChildren, the destination is clipped by all viewable InputOutput children. If IncludeInferiors is specified, then viewable inferiors of the destination that overlap the VIR are | updated with pixels from the video picture if the following are | true: | - Composite is asserted in the video-input-models for the video input source. | - The destination is an InputOutput window; or the destination is a VideoBuffer window and the destination and its InputOutput parent have a matching depth/visual. - The destination and its inferiors have a matching visual/depth. Otherwise, the result in the inferiors is undefined. | If the destination window is clipped by siblings or children, a + VideoViolation event may be generated if an overlap or clipping + constraint is violated; see QueryVideo. + + A RenderVideo request using a window W already allocated to an earlier RenderVideo request has the same effect as issuing a StopVideo request with arguments W and Render, followed by the new | RenderVideo request. Similarly, a RenderVideo request using a | video input V already allocated to an earlier RenderVideo request | has the same effect as issuing a StopVideo request with arguments | W' and Render, where W' is the window from the earlier request, | followed by the new RenderVideo request. In either case, a | VideoOverride event may be generated. | A RenderVideo request to a pixmap or an unmapped VideoBuffer window calculates the value of the pixel entirely upon the value of the video pixel; no translucent or blendmap is involved. When a VideoBuffer window is used as a destination, clients + that do not need access to digitized pixels should use an InputOutput + window as a destination, because this is the most portable. + There is no guarantee that the signal is actually digitized, even if a VideoBuffer window is specified. However, if a VideoBuffer window is used as the destination, then CopyArea, CopyPlane and GetImage requests with the VideoBuffer window as the source are guaranteed to return a consistent frame with respect to adjacent fields in the video signal. As detailed above, note that some | pixels may be undefined if Composite is asserted in the | video-input-models for the video input source. | If the specified window changes size while the request is | still active, the destination x and y the original request will be adjusted to follow the BitGravity of the destination, but the width, height and scale are not adjusted. If the resulting rectangle is beyond the extent of the destination, the result is clipped appropriately; a VideoViolation event may be generated. UpdateTranslucent source, destination: DRAWABLE | gc: GCONTEXT | src-x, src-y: INT16 width, height: CARD16 dst-x, dst-y: INT16 Errors: Window, Pixmap, Match This request updates a rectangle of a translucent pixmap associated | with window, or will obtain the pixels in a translucent. The | semantics are very similar to that of CopyArea, with a restriction | on the source and destination type combination. If destination is an | InputOutput window, then the translucent associated with the window | is updated, and the source drawable must be PIXMAP, else | Match error. If the destination is PIXMAP, then it receives the | contents of the translucent pixmap associated with the source, and | the source must be an InputOutput window, else Match error. The | depth of the source and destination must have the same root the | depth, else Match error (note that the depth of the translucent | pixmap is used for comparison, not the depth of the associated | InputOutput window). The src-x and src-y coordinates are relative | to the source's origin. The dst-x and dst-y are relative to the | destination's origin, each pair specifying the upper left corner of | the rectangle. | If regions of the source rectangle are obscured and have not been | retained in backing store or if regions outside the boundaries of | the source drawable are specified, then those regions are not | copied, but the following occurs on all corresponding destination | regions that are either visible or are retained in backing-store. | If graphics-exposures in gc is True, then GraphicsExposure events | for all corresponding destination regions are generated. | If graphics-exposures is True but no GraphicsExposure events are | generated, then a NoExposure event is generated. | GC components: function, plane-mask, subwindow-mode, | graphics-exposures, clip-x-origin, clip-y-origin, clip-mask | FillTranslucentRectangle | drawable: WINDOW | gc: GCONTEXT | rectangles: LISTofRECTANGLE | Errors: WINDOW, GContext, Match | This request fills the specified rectangles in the translucent | pixmap associated with the named window, as if a PolyFillRectangle | were specified with the same list of rectangles. | CaptureGraphics source: DRAWABLE destination: VIDEOOUT src-x, src-y, dest-x, dest-y: CARD16 src-width, src-height, dest-width, dest-height: CARD16 full-motion: BOOL priority: CARD8 subwindow-mode: { ClipByChildren, IncludeInferiors } Errors: Drawable, Match, Value The source must be a valid drawable id, else Drawable error. The specified region of the source is encoded into a video signal by the destination encoder. If the drawable is an InputOutput window, then Window must be asserted in the video-output-models returned by QueryVideo for the intersecting visual/depth of the window and the specified video output, else Match error. If the source is a VideoBuffer window, then Buffer must be asserted in the video-output-models for the intersecting visual/depth of the window and the specified video output, else Match error. If the source is a pixmap, then Pixmap must be asserted in the video-output-models for the intersecting depth of the pixmap and the specified video output, else Match error. The src-x, src-y, src-width and src-height describe the extent of the source rectangle of the region captured. The dest-x, dest-y, dest-width and dest-height describe the rectangle in the destination video picture to receive the graphics. The source graphics region will be scaled by dest-width dest-height x scale = --------- y scale = ----------- src-width src-height The scaling factors applied must match one of those provided by the QueryVideo request for the specified video output destination, else Value error. If the destination rectangle is valid, but some or | all of it is outside the extent of the video picture, those pixels | are clipped in the output video picture. If the source rectangle | is valid, but some or all of it is beyond the extent of the source | drawable, the result in the encoded video signal is undefined: it | is hardware dependent. | The source-x and source-y are coordinates in the source drawable | that will be made coincident with the upper left corner of the destination video picture. If the source rectangle intersects | with a region beyond the extent of the screen, the results in | the output picture is undefined: it is hardware dependent. | The full-motion element determines whether the video output signal is constantly updated or not. If it is true, then the server will try to achieve the maximum rate advertised by the QueryVideo request for the specified scale factor and placement. If it is False, then a single frame will be captured. It is just as if the CaptureGraphics request was immediately followed with a StopVideo request specifying the same drawable; however, a full-motion element of False is an aid to the server to avoid extra encoding. Priority is a value (0 to 100 inclusive, else Value error), representing the percentage of video bandwidth that the server should try to preserve. A value of 0 tells the server that graphic output to the screen should have top priority and that the video capture process may occasionally miss some incoming or outgoing frames. A value of 100 gives video capture top priority meaning that the graphics performance may be degraded. It is advisory only and does not guarantee anything. Subwindow-mode controls the capture of the window's children. If | ClipByChildren is specified, the source is clipped by all viewable | children only if Composite is not asserted in the | video-output-models for the video output destination. If Composite | is asserted, subwindow-mode will be effectively IncludeInferiors, | regardless of its value. | | If IncludeInferiors is specified for the subwindow-mode, then | each visible region of an inferior of the source that overlaps | the VIR may captured, but only if one or more of the following is true: | - The inferior is the same depth, visual and window type as | the source window. | - Composite is asserted in the video-output-models for the video output destination. | Otherwise, the source rectangle is clipped. | If the source rectangle is clipped by siblings or children, the + appearance of the clipped regions in the encoded picture are + undefined: it is hardware dependent. + + A CaptureGraphics request using a window W already allocated to an earlier CaptureGraphics request has the same effect as issuing a StopVideo request with arguments W and Capture, followed by the new | CaptureGraphics request. Similarly, a CaptureGraphics request | using a video output V already allocated to an earlier | CaptureGraphics request has the same effect as issuing a StopVideo | request with arguments W' and Capture, where W' is the window from | the earlier request, followed by the new CaptureGraphics request. | In either case, a VideoOverride event may be generated. | If the specified window changes size while the request is | still active, a VideoViolation event may be generated. The source x and y the original request will be adjusted to follow the BitGravity of the source, but the width, height and scale are not adjusted. If the resulting rectangle is beyond the extent of the source, the result in the output signal is undefined. StopVideo LISTofDRAWABLE action: { Render, Capture, RenderAndCapture } Errors: Drawable, Value The id must be a valid drawable, else Drawable error. If more than one id is in error, which one is reported is arbitrary. The video actions previously requested on any of the listed ids are stopped, even if one of the ids produces an error. If action is Render, then RenderVideo requests on the listed ids are terminated. If action is Capture, then CaptureGraphics requests on the listed ids are terminated. If action is RenderAndCapture, then both are terminated. If neither RenderVideo or CaptureGraphics are currently active on a id, no error is reported. If a RenderVideo on InputOutput window W is followed by a StopVideo + on W, and Composite is asserted in the video-input-models for the + video input source, then the visible result on the screen is + undefined, because it is hardware dependent. Under the same + circumstances, if Composite is not asserted, then the visual effect + is that all traces of the video picture will disappear, nothing is + retained. If a RenderVideo on a VideoBuffer window W is followed + by a StopVideo on W, the result in W is the pixels take on the + value of the last frame acquired: they can be manipulated by other + X requests. + + QueryVideoControls vid: VIDEOID control-list: LISTofATOMS ==> LISTofSETTING Errors: Match, Atom This request provides the current setting associated with each of the listed controls for the listed video id. The control list must be taken from the set of control names supplied by QueryVideo, else Atom error. If the vid or any of the control-list generate an error, no settings are returned. ChangeVideoControls setting: LISTofSETTING Errors: Window, Match, Atom, Value, Length This request changes the setting of each control named in each SETTING structure. The control named by the atom and associated with the named device is set according to the value of the data. If event is true, then a VideoControl event is generated on the named window when the control setting has actually been accomplished. An event may be delivered anyway if the setting failed for some reason. All valid settings take effect, even if some of the settings are invalid. If an error is returned, which setting generated the error is not specified. InstallBlendmap bmap: COLORMAP Errors: Colormap This request makes this blendmap an installed map for its screen. All AVRs associated with this blendmap immediately display their blended pixels correctly. As a side effect, additional blendmaps might be implicitly installed or uninstalled by the server. Which other blendmaps get installed or uninstalled is server-dependent except that the required list must remain installed. If bmap is not already an installed map, a ColormapNotify event | is generated on every window having bmap as an attribute. In addition, for every other blendmap that is installed or uninstalled as a result of the request, a ColormapNotify event is generated on every window having that blendmap as an attribute. At any time, there is a subset of the installed maps that are viewed as an ordered list and are called the required list. The length of the required list is at most M, where M is the min-installed-maps specified for the screen by the QueryVideo request. The required list is maintained as follows. When a blendmap is an explicit argument to InstallColormap, it is added to the head of the list; the list is truncated at the tail, if necessary, to keep the length of the list to at most M. When a blendmap is an explicit argument to UninstallBlendmap and it is in the required list, it is removed from the list. A blendmap is not added to the required list when it is installed implicitly by the server, and the server cannot implicitly uninstall a blendmap that is in the required list. Initially the default blendmap for a screen is installed (but is not in the required list). UninstallBlendmap bmap: COLORMAP Errors: Colormap If bmap is on the required list for its screen (see InstallBlendmap request), it is removed from the list. As a side effect, bmap might be uninstalled, and additional blendmaps might be implicitly installed or uninstalled. Which blendmaps get installed or uninstalled is server-dependent except that the required list must remain installed. If bmap becomes uninstalled, a ColormapNotify event is generated on every window having bmap as an attribute. In addition, for every other blendmap that is installed or uninstalled as a result of the request, a ColormapNotify event is generated on every window having that blendmap as an attribute. ListInstalledBlendmaps window: WINDOW ==> bmaps: LISTofCOLORMAP Errors: Window This request returns a list of the currently installed blendmaps for the screen of the specified window. The order of blendmaps is not significant, and there is no explicit indication of the required list (see InstallBlendmap request). RedirectVideoRequests wid: WINDOW Redirect video requests based on lack of ownership?? Events ------ VideoConnectivityState window: WINDOW new-state: CARD32 time: TIMESTAMP This event is reported unconditionally to any client that has issued a QueryVideo request on the screen belonging to the specified root window when the connectivity of video inputs, outputs and | devices changes state. The new-state element is an index into the list of adjacency-matrices returned by QueryVideo. | The time reported is the time of the last successful ChangeVideoConnectivity request. VideoChange window: WINDOW + vdev: VIDEODEVICE state: { Online, Offline } time: TIMESTAMP This event is reported unconditionally to any client that has issued a QueryVideo request on the screen belonging to the specified | root window when the named video device has changed | state. Issuing an QueryVideo request will yield a new device list and adjacency matrices. New video devices are guaranteed not to have the same id as devices that were once listed, except for devices that go off line and return later: these devices will retain their previous id. This event is guaranteed to be delivered | before any VideoConnectivityState events are delivered for the new connectivity state. The time is what would be reported by the QueryVideo request. VideoControl vid: VIDEOID name: ATOM state: { Success, Fail } time: TIMESTAMP This event is generated when requested by a ChangeVideoControls request, or if a ChangeVideoControls request failed in an unexpected way. The vid and name are those in the SETTING given to the | ChangeVideoControls request. The state is Success if the setting | was accomplished, Fail otherwise. | The time is the current server time. VideoViolation window: WINDOW vid: VIDEOIO action: ACTIONMASK where ACTIONMASK is { Scale, Placement, Clip, Overlap } state: { Success, Subset, Fail } time: TIMESTAMP This event is reported to the client selecting EXPOSURE on the window. The event is generated for the specified window and video input or output when scaling, placement, clipping and overlap constraints are violated. If state is Success, then the listed violations caused no change in the display or video output for the specified window, although some other window may be affected; If state is Subset then some subset of the unobscured regions are still being displayed or captured; if the state is Fail, then the video input is no longer being displayed, or the capture is no longer occurring. The time is the current server time. VideoSync id: WINDOW vin: VIDEOIN state: { Acquired, Lost } time: TIMESTAMP This event is reported to the client selecting EXPOSURE on the id. It is generated following a RenderVideo request with the specified id and video input, and before a StopVideo request on same, whenever the sync is lost or gained. At least one VideoSync request is guaranteed following a RenderVideo request, if selected. The time is the current server time. VideoOverride id: WINDOW vin: VIDEOIN time: TIMESTAMP This event is reported to the client selecting EXPOSURE on the id. It is generated following a RenderVideo request that results in an implicit StopVideo on the specified id. The time is the current server time. VideoRequest vid: VIDEOID client: ? request: ? ======================================================================== Received: from EXPO.LCS.MIT.EDU by decwrl.dec.com (5.54.5/4.7.34) id AA03581; Thu, 3 Aug 89 15:50:26 PDT Received: from EXPO.LCS.MIT.EDU by decwrl.dec.com (5.54.5/4.7.34) for 3d::xvideo-dec; id AA03581; Thu, 3 Aug 89 15:50:26 PDT Received: from RELAY.CS.NET by expo.lcs.mit.edu; Thu, 3 Aug 89 18:36:58 EDT Received: from tektronix.tek.com by RELAY.CS.NET id aa10038; 3 Aug 89 18:36 EDT Received: by tektronix.TEK.COM (5.51/7.1) id AA28415; Thu, 3 Aug 89 15:36:14 PDT Received: by tekirl.LABS.TEK.COM (5.51/7.0) id AA13020; Thu, 3 Aug 89 15:33:46 PDT Received: by tekcrl.LABS.TEK.COM (5.51/6.24) id AA25183; Thu, 3 Aug 89 15:34:13 PDT Message-Id: <8908032234.AA25183@tekcrl.LABS.TEK.COM> Reply-To: toddb%tekcrl.labs.tek.com@relay.cs.net