From: Jason Jerald [jjerald@isl.hrl.hac.com]
Sent: Tuesday, October 05, 1999 1:36 PM
To: Maarten van Dantzich; 3d-ui@hitlab.washington.edu
Cc: Jerry Isdale; Howard E. Neely, III
Subject: Re: 3D window management



Maarten van Dantzich wrote:
> 
> Jerry Isdale said:
> > Very few 3d worlds do a decent job with text and when you try to
> > use the 2d app as a bit map, readabilty suffers greatly.
> 
> Text is definitely a problem, especially if you expect the reader to read
> longer texts (webpages) in the environment or if you're doing 3D Info Viz
> and you want very legible labeling that's not too huge. Vector fonts are
> just grotty. (technical term. ;)
Text is especially bad in our system because we go through sirius video
first then 
depending on the perspective the image gets filtered again.  Soon we are
going to bypass the
Sirius Video completely so hopefully things will clear up some.

> 
> We did a bit of work in the usability lab with text rotated around the
> vertical axis (Y), comparing a vector font against a trilinear filtered
> texture and an anisotropically filtered texture. Haven't gotten the result
> accepted for publication yet. (#include "reviewers-clueless.h" ;-)
> Admittedly it's a minor result, but it's an area that really needs some
> structural exploration...  PhD topic, anyone?
Yes I would think this would greatly clean things up.  However the
purpose of our
project currently is to interact with any 2d applications on a window3d
class.  Thus
nothing can be assumed about the application such as what type of font
is being used.

> 
> > One approach I'd like to see is using 2d hardware as direct sources for
> > bitmaps into the 3d world.  Instead of feeding a video converter, the 2d
> > frame would feed (be sampled) into texture memory
> 
> What exactly did you mean here?  That you want to preserve 2D drawing
> acceleration (GDI acceleration, under Windows) and still texture map the
> result into a 3D scene?  That's not so far-fetched. We've certainly
> discussed it here at MS as a desirable extension of the App Redirection
> architecture that we have implemented right now.
> 
> It might not even be THAT hard, since unification of GDI DCs and DX Surfaces
> is happening in Win2000. You'd do it on a single video card, 2D apps using
> the GDI paths into a DC bitmap allocated in VRAM, which you then also access
> as a texture surface. The main issue is locking/synchronization or coming up
> with a clever double-buffering scheme that doesn't break the "update region"
> optimization in the GDI painting model. Of course the changes do have to
> happen at the low system level, and it's hard to get cycles from the Win2000
> and Direct X teams for such far-out things... but not unimaginable.
We are currently using a multiple pipe SGI Onyx for this project.  Our
Sirius video
option is set to a single pipe.  We have the sirius hardware grab the
screen with user
defined settings such as resolution, size, and geometry.  This can then
be directly fed
to texture memory.  Thus it is almost free.  The problem is this only
works on a single pipe.
Our Cabana uses two pipes (4 channels - 3 walls, 1 floor) and the
application must be run on
the third pipe.  So the Sirius video is instead fed to shared memory,
resized to an appropriate
opengl size (sirius video has only a few size options, opengl needs to
be a power of 2), then
copied to each pipe's texture memory.  So in conclusion the Sirius
hardware auctually is more
trouble than it helps when using the Cabana.  We will be bypassing the
sirius video completely
in the near future and have the application draw to shared memory
instead of to the screen.


> 
> But not having 2D drawing accelaration is not the bottleneck right now. Even
> on a single processor P-II 400, we can run a bunch of live apps, redirect
> their drawing into memory bitmaps, blt the result into a texture, and
> achieve 20 fps or so. (1024x768x16bpp on NVidia TNT2, DX6, low polygon
> count.) Actually, that's 20fps update of the 3D scene with only partial
> update of some of the app windows on each frame;I haven't run any stress
> tests to see how much 2D drawing we can handle--but regular interactive app
> usage and even animation on web pages works fine.
As you can tell from the above comment our system is definitely not
currently
optimized due to the redundant use of the Sirius video.  We had to put
the memory
copying onto different processers to reach an acceptable rate.  Since
the 3d rendering
of the world is on its process we can get very fast render rates (60Hz)
with simple
worlds.  However the update rate of the 2d application is slower at a
fairly consistent
value of around 10 fps.  There also is some lag but it is useable as far
as interacting
with the application goes.  The problem is it takes a huge amount of
resources and the
resolution is very limited.  The sirius video currently uses 720x486 
(although it can
go up to 768x576 - haven't got this to work yet).  An entire screen such
as 1920x1200
can be grabbed but it is converted down to 720x486 which makes anything
but very large
text unreadable.


It might be worth for the
> HitLab folks to stop by for a demo some time.
I will be at the HitLab next week for the Virtual Worlds Consortium
meeting.  Perhaps it would
be possible to see a demo of the Task Gallery?

> 
> > Let me put three wicked fast 3d accelerator boards in my PC and put it up
> > on three large (flat) screens).
> 
> Yow. Are you ordering quad processor machines, too? :)
We hopefully will be creating a win2000 based CAVE like environment in
the near future.
This should help with many of the problems we have currently, but there
is also some
challenging issues such as multiple graphics cards and syncing these
together.

Jason


-- 
Jason Jerald                    Human-Centered Systems Department
mailto:jjerald@hrl.com          Information Sciences Laboratory
(310) 317-5622                  HRL Laboratories
http://home.earthlink.net/~jasonjerald