Data Collection Techniques
The initial development began with an examination of existing interfaces. Open-ended interviews were held with the clients to get a sense of elements that they wish to retain and functionality they want to add.
In preparation for the prototyping plan, additional meetings were held where the existing requirements specification was reviewed. The primary point of dissatisfaction was a desire for the validation requirements to be more focused on demonstrable user interface tests. The requirements were accordingly updated.
For the discussion of the prototype, interface drawings were coupled with a narrative description of a typical program usage. The clients offered comments on elements that were either unclear to them or which the developers didn't understand sufficiently. Potential scenarios for user testing were discussed, but no definitive decisions were made.
Prototype Description
This interface interface is divided into two primary subinterfaces to support the operator in controlling both the overarching disaster response as well as teloperation of specific vehicles. Certain elements of the interface will remain consistent between the two views and they are illustrated in Figure 1:
- Menu Bar — This is an interface common to GUI programs providing access to all program functions through a set of hierarchical menus.
- Manipulation or Orchestration Interface — The contents of this portion of the interface will change depend on the interface the user is accessing and they are discussed below.
- Robot Status — Certain information about the vehicles is present on the screen at all times. This includes certain information common to all robots such as time left in field (battery or gas level). It also includes certain information specific to particular types of robots. This area also serves through the use of color to indicate groupings of robots and allows alerting the operator of situations requiring attention.
- Available Overlays — The interface will permit the operator to blend together different images to increase situational awareness while manipulating a specific robot. This area allows the selection of which overlays are included in the overlay. Available information might include robot position, chemical sensing heatmaps or aerial coverage maps.
- Interface Specific Status Information — Information about the specific interface being shown at the current time.
The orchestration interface provides a high level overview of the entire response scene. It is used by the operator for three primary purposes:
- Group Definition — Associating robots together in groups
- Task Definition — Defining the tasks that need to be completed as a part of the disaster response
- Waypoint Definition — Robots will need to move around the world to complete their tasks. The operator will specify one or more waypoints to define a path for the robots to take.
The manipulation provides teleoperation access to an individual robot. It is used by the operator to perform tasks that autonomous systems are currently incapable of. The exact controls and status monitors are dependent on the specific robot being controlled.
Prototype Technologies
To best meet the client requirements, user interface testing for this project will be conducted with a partial implementation prototype. A variety of technologies will be employed in the development of the prototype:
- USARSim — For user testing, data from actual robots is not necessary. A simulation is both easier to set up and more reliably reproducible. Urban Search And Rescue Simulation (USARSim) provides a virtual environment that tracks the position and state of the robots. The program uses Unreal Tournament 2004 for physics simulation and scene rendering.
- ImageServer — USARSim provides simulated cameras positioned on robots. Unreal Tournament does not provide a simple interface for getting an image of the current scene. The image server uses low-level Windows system calls to generate a video stream for camera simulation.
-
Player — This program needs to eventually interface with a variety of different types of robots. Player is an abstraction library that provides a unified interface for querying and controlling robots as well as implementations of some established algorithms for route finding and obstacle avoidance. Player is a client/server architecture which communicate via a network and the components are:
- Player Server — The server runs on an individual robot and receives queries and control commands. Various drivers exist for controlling different pieces of hardware including drivers to return the simulated data from USARSim as though it represented actual robots.
- Player Client — The client is an application library utilized by the interface to communicate with the server.
- QT — The graphical components of the interface require a toolkit for rapid development. QT is a flexible and powerful cross-platform library for developing GUIs. It includes facilities for sophisticated composition of different types of images which is an important feature for this interface.
All of the technologies for this application were chosen for their cross-platform capabilities and ease of use. Rapid prototyping packages such as Visual Basic, and prototyping techniques such as mimicking live video with pre-recorded video will not be used for two primary reasons. First, this interface in a completed form may be needed in the near future and time may not be available to reimplement it with more powerful methods. Second, user testing is specifically to ascertain a user's efficacy in controlling a robot, so using prerecorded video is not feasible.
Figure 4 provides a structural diagram describing the connections between the different technologies and the resultant data paths. The system may potentially employ up to three separate computers. Although it may be possible to instantiate each sub-system on the same machine, computational complexity may prohibit this deployment strategy. Currently, the only platform limited technology is the Image Server which has Windows-specific requirements.
The Player libraries and and QT will run on the majority of POSIX-based operating systems, including the Cygwin environment on Windows. Since a client-specified requirement states that the application must run under Windows, the interface application, ImageServer, and USARSim will be running under Windows during user testing.
Prototype Limitations
This protopype is being developed on an extremely abbreviated timeline. Due to these time constraints elements of the ideal interface will have to be remain unimplemented. The elements that will need to be implemented for user testing are:
- Orchestration Interface:
- Display Map — Show a 2D map of the world being explored
- Waypoint Definition — Permit the user to specify a path for the robots to follow
- Robot Map Position — Display the position of the robot in the world
- Orchestration Status Display — Display salient information about the state of the world including defined tasks and robot groups
- Manipulation Interface:
- Robot Video — Show video from the robot in the manipulation interface
- Manipulation Status Display — Display information about the operating characteristics of a particular vehicle
- Robot Manipulation — Allow teleoperation of individual vehicles
- Common Elements:
- Robots Status — Show cursory information about the state of all the robots controlled by the interface
- Ground Robots — Place ground-based robots in the interface
Elements that would ideally be a part of this interface, but which won't be implemented for time reasons:
- Task Definition — Specify information about the tasks that need to be completed
- Group Creation — Create collections of associated robots
- Overlay Selection — Allow toggling of information pertinent to the world or to the camera view
- Aerial Robots — Place aerial vehicles in the interface
- Map Scaling — Permit the user to zoom and pan the map in the orchestration view
