From: dcc@ATHENA.MIT.EDU
Message-Id: <8909081825.AA14110@THANATOS.MIT.EDU>
To: toddb%tekcrl.labs.tek.com@relay.cs.net
Cc: xvideo@expo.lcs.mit.edu
Subject: re: re: Introduction and basic questions
Date: Fri, 08 Sep 89 14:25:24 EDT


This message has gotten a bit long, but most of it is transcripted
from previous mail messages.

	--david

                              - - - - - - -

>> The VEX
>> protocol touches on several interesting and hard problems, many of
>> which are not strictly (or even predominantly) related to video:
>> compositing, multiple heterogeneous frame buffers, transparency,
>> translucency, overlays, multibuffering, and the interaction of
>> multiple graphical methods within an object.
>
>This extension is not for video alone, but for video hardware as
>it exists today and as I expect it to exist for some time.
>Compositing, transparency, overlays are a core part of this hardware.
>I don't know where you get the idea that VEX addresses multibuffering,
>because I certainly didn't intend to include it... perhaps you can
>point it out.  "...the interaction of multiple graphical methods
>within an object" has a nice ring to it, but doesn't point clearly
>to a part of the protocol.  Please be specific.
>

My whole introductory paragraph was meant to be general so as to
communicate my initial impressions and basic premises.  I'll defer my
discussion of how multibuffering relates to the VEX extension until
later.

                              - - - - - - -

>> I believe that VEX 
>> should attempt to avoid these issues for following reasons: (1) 
>> there doesn't seem to be a standard approach taken by hardware in 
>> supporting these capabilities, 
> 
>If there is no standard approach to something (you must be more specific), 
>then an extension must either not express the functionality or 
>express the common denominator.  As appropriate, I believe VEX does both.  
>

I'm sensitive to the "common denominator" issue; it is fundamental in
the design of standards.  In the design of the X11 protocol, and in
the design of PEX and the double buffering extensions, hardware
specific functionality was either abstracted, removed, or enumerated.
Enumeration was always a last resort because it is inevitably near
sighted and constraining, and when it did become necessary is was
abstracted as much as possible, localized, and made extensible; a good
example of this is the VisualType mechanism.  Specifics?  Read on...

                              - - - - - - -

>> The concept of translucency as proposed
>> by VEX is really a specific implementation of two way compositing...
>> So, for the time being, I'd try to skirt the compositing issue in
>> the VEX protocol by exposing device specific functionality through
>> the use of existing mechanisms and private extensions, and wait for
>> a broader compositing extension to be defined.
>
>This is an extension for today's hardware.  If and when a compositing
>extension is proposed and implemented, I'll be tickled to adapt it.
>In the mean time, since none is on the horizon, we can certainly break
>new ground here.  Portability is paramount and device specific
>functionality shoots that all to pieces.
>

Its clear to me that we disagree about what a standard should attempt
to do.  Anyway, it sounds like overlays provide the capability for
what applications commonly need and what current hardware commonly
supports.  The VisualType scheme that I invented is really a simple
overlay mechanism.  The question I'm left with is whether VEX should
define special overlay behavior, or whether this should be left to a
more general overlay extension.


                              - - - - - - -

>> I will attempt to quickly define the content of such an
>> extension:
>>
>>   The goal is to permit an application to create an image that is
>>   produced by compositing two other images (one graphics and one video)
>>   via a third image (blend).
>>
>>   One simple (no new semantics) way to achieve this is for the
>>   server to define a new VisualType: Blend.
>>
>>   An application would create a Blend window (a window of VisualType
>>   Blend) as the child of a normal graphics window and display a
>>   video image into a rectangle of the Blend window.
>
>This represents two images, not three, as you wanted.  You have no window
>for the video.
>

No I have three.  The video image (1) appears in the Active Video
Region which is displayed in the Blend window.  The blend image (2) is
drawn into the "alpha" buffer by drawing to the Blend window; it is
overlayed by the video image.  The graphic image (3) is drawn into the
frame buffer by drawing to the parent window with subwindow-mode =
IncludeInferiors.

                              - - - - - - -

>>   Graphics is
>>   drawn to the parent window using the subwindow-mode =
>>   IncludeInferiors GC attribute.  Thus the graphics image is drawn
>>   over the video image and the blend image is drawn independently.
>
>Again, you have lost the video.  The child is a blend window, not a
>video window.  Where do I put the video?  Does it just kind of float
>above the blend window.
>

No, video can be displayed in any InputOutput window according to
your description of RenderVideo.  Yes, video is displayed above the
blend window.  This is consistent with your description of
RenderVideo: "...the pixels of the video image are magically placed
"over" the (graphic) window without modifying its pixels".

                              - - - - - - -

>What you are missing is someplace to put the video.
>Granted, there are times when you don't want, or can't get, the digitized
>video bits (as VEX allows for), but there are more often times you do and can.

In this case I would create a third window with a new visual type
(Video), make it a child of the blend window, and draw
into the blend window with subwindow-mode = IncludeInferiors.

                              - - - - - - -

>Seems to me you are quite close to what VEX is proposing.


The mechanism I describe is only similar to VEX from the hardware
point of view.  VEX has created two new window attributes (Blendmap
and Transparency), extended the existing window class attribute, added
six new requests (ChangeBlendmap, UpdateTranslucient,
FillTranslucientRectangle, InstallBlendmap, UninstallBlendmap,
ListInstalledBlendmaps), and changed the semantics of two existing
requests (CreateColormap, CreateWindow).  The mechanism I propose adds
at most two new VisualTypes (Blend and Video)---the number of
VisualTypes was always expected to expand---and it relies entirely
upon existing mechanisms which continue to behave as expected.

                              - - - - - - -

>I think you have that backwards.  I think you have to demonstrate what
>cannot be done with the current design.

I agree that the onus is on me to point out deficiencies in the
current design, but your analysis of my proposal was helpful.


                              - - - - - - -


>
>> but that's not the point.  The point is that I can easily think of a
>> half dozen variations on the theme, and private extensions should
>> be defined to expose the specific capabilities of different devices.
>
>I disagree, with respect to blending, because it is a feature common
>to most video hardware.  I agree, however, with respect to a host
>of other capabilities, because they are not necessarily central to
>what a video application must accomplish; e.g. brightness and contrast.
>Eventually, most of these can be standardized through the "controls" and
>"settings" in VEX.  So I hope you will view this as if we mostly agree :-)
>

I agree that most video workstation hardware can support some form of
overlay (1 or more planes), but "alpha" buffering, as defined by VEX,
isn't common.  Blending is very common in broadcast video hardware,
but VEX is providing a mechanism to annotate a video image with
graphics (or vise versa), which really isn't the same.