From: owner-3dui@hitl.washington.edu on behalf of Doug Bowman
[bowman@vt.edu]
Sent: Wednesday, October 03, 2001 11:55 AM
To: 3dui List
Subject: RE: Comparison of HMD and Dome

> Generalizing the tasks performed in VE is exactly what I am doing. Your
> "testbed evaluation" paper is really useful.
>
> There is one point I got confused. I noticed that the model of the testbed
> (also other previous research) was created based on tasks. For instance,
> for selection and manipulation, a boxes scene was created; for travel
> , a town type scene was created. But in our model, we did all test by
> using only one model. The VE is a room with a set of 15 colored balls,
> cubes, torus, cylinders, and paramids (5 of each type, 3 of each color)
> along one wall and 15 matching platforms along the opposite wall. The task
> requires subjects to move the 15 balls on the left side the room over to
> the matching 15 platforms on the right side of the room. The subject has
> to go through a maze type walls, while avoiding the walls.  We measure
> 6-DOF manipulation performance, task finishing time, sickness, and so on.
> Notice that subjects have to navigation to the object, pick up and then
> navigation, then drop. Two constant speeds are allowed.
>
> Does it matter we measure them together? My program did record all points
> (in each loop) subject traveled to. Should I seperate the task performace
> and discuss them?

Jian,

If you're looking for generalizable results, my personal feeling is
that it's best to separate the tasks and control all the outside factors
as much as possible. That way, when you do your analysis you can find
out statistically which factor is responsible for any changes in
performance.

There are some other people on the list who disagree with me (would Jeff
like to chime in here?).

When I created the testbeds for travel and selection/manipulation, my
aim was to create experimental environments that could be re-used. I
envisioned people designing new interaction techniques and testing them
in these same testbeds, then comparing the results to my results. However,
there's no reason why they shouldn't be used to compare different displays
as well (I did all my work on HMDs). Perhaps this would be a good test
of the generality of those experiments?

--Doug
--
Doug A. Bowman, Ph.D. (540) 231-2058
Assistant Professor   (540) 231-6075 (fax)
Computer Science      bowman@vt.edu
Virginia Tech       www.cs.vt.edu/~bowman/