HMT Lab Interface Data Requirements

Go to X Location where X may be any location, a victim, a bomb, a sampling location (chemical, biological, radiological, etc.), etc.

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults
1. Task name
2. Start time — Now + 2 min
3. Interrupt ability — predefined default based upon task type
4. Priority level — predefined default based upon task type
5. Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.
6. (Possible) Robot type — may be predefined if the particular unit the interface is associated with only has one type of robot (e.g. ground or aerial).
(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
Specify what type of location the robot is to go to where the location type may be: Victim, bomb, Chemical sample, Biological sample, Radiological Sample, Nuclear Sample, rendezvous point, surveillance point, egress/ingress point, any (meaning just go to this point). — may result in automatically filling in the robot type field
1. Robot type — Default may be assigned based upon the location type since some locations will clearly be ground while others will be aerial.
Specify robot type — aerial, ground, mixed, or any
Specify the point on the map:
1. Select the map interaction button in the task specific area to enable specifying a point.
2. Select the point on the map for the robot to go to and display an appropriate icon, waypoint.
Submit task.

Alternative: Specify task type first

Select Task type — results in automatically filling in the following fields with defaults
1. Task name
2. Start time — Now + 2 min
3. Interrupt ability — predefined default based upon task type
4. Priority level — predefined default based upon task type
5. Division — defaults to last leader name’s division.
6. Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.
7. (Possible) Robot type — may be predefined if the particular unit the interface is associated with only has one type of robot (e.g. ground or aerial).
(Optional) Specify the task name
(Optional) Specify the Division

surveillance/search of an assigned area — either aerial or ground

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type (aerial surveillance) — results in automatically filling in the following fields with defaults

Task name
Start time — now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.
(Possible) Robot type — may be predefined if the particular unit the interface is associated with only has one type of robot (e.g. ground or aerial).

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
Specify surveillance vs. search task.
Specify Robot type — aerial, ground, both, or any
Specify area to be covered on map.

Select the map interaction button in the task specific area to enable specifying the area to be searched.
Select the required waypoints on the map to define the area.

Specify execution pattern — random, follow path, etc.
Specify Level of Detail (similar to NIIRS) — General level is a default.
(Optional) if the task is a search task, specify the item searching for, e.g. bomb, victims, suspicious items, etc.

If specify a point to survey and that will default the algorithms for a single point. Similarly the specification of area may also limit the available execution pattern algorithms. May need to also watch a point for things approaching a point.

Alternative: Specify task type first

See Go to Location X

Alternative: Specify coverage area first (Curtis)

Specify area to be covered on map.

Select the map interaction button in the task specific area to enable specifying the area to be searched.
Select the required waypoints on the map to define the area.

Start at step 1 and continue from above.

Option: The execution pattern may be chosen from a context menu once the user specifies the search area on the map. This could be confusing if all the other aspects are chosen in the task specification area.

Designation of the ingress/egress points

This is not truly a task since if the information has been entered by a division chief or someone higher, the information should just appear on the interface. However, this information will be communicated verbally initially and someone has to enter this information.

Option 1 (Curtis)

Select from information item à Set ingress/egress
Select ingress/egress points

Alternative: Simply reverses the steps.

Option 2

Specify — Set Ingress/Egress Point
1. This automatically enables the ability to click on the map and specify the points.
Specify point as Ingress or Egress.
Select point on the map.

Initial Detailed Victim Assessment — Ground Robots

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Specify victim location — If the victim has been discovered via the robot’s exploratory tasks, then this information is the robot’s current position, otherwise the user can specify the location on the map.
Assign a Victim ID (default value if the robot is at a victim) or assign an area in which to assess all victim that have not been assessed.
Level or type assessment
Physical contact allowed — yes or no

This task may require the robot to gather imagery and use voice to interact with the victim. If the operator is a paramedic, then the operator may speak directly with the victim via the robot’s speakers. This task is intended to be a close interaction with the victim.

Bomb Surveillance — Aerial Robots

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Specify bomb location — If the bomb has been discovered via the robot’s exploratory tasks, then this information is the robot’s current position, otherwise the user can specify the location on the map.
Specify height-above ground range — The UAV may have to reduce its height-above ground in order to obtain imagery and other sensor information. Includes Minimum and maximum values.
Specify sensor information to be provided — Default is any but may be imagery, laser scan, etc.
Specify viewing angle — top-down, 360 angled pattern, horizontal angle of attack 35-80, etc.
Persistence — yes or no
Bomb ID — could default if UAV is at the location because of discovery.
Level and type of assessment. (How does this differ from HAG, sensor, angle etc?) May be type of bomb (pipe, etc.).

If specify point to survey, then specify the viewing angle pattern to be attained, which may be specified by algorithm type or not.

Bomb Surveillance — Ground Robots

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Specify bomb location — If the bomb has been discovered via the robot’s exploratory tasks, then this information is the robot’s current position, otherwise the user can specify the location on the map.
Specify minimum/maximum distance to bomb
Specify sensor information to be provided — Default is any but may be imagery, laser scan, etc.
Specify the robots’ position (correct word?) — Stationary vs. moving (circling) the bomb.
Specify viewing angle — top-down (assuming robots exist with this capability), side view, 360 degrees, etc.
Persistence — yes or no
Bomb ID — could default if UAV is at the location because of discovery.
Level and type of assessment. (How does this differ from HAG, sensor, angle etc?) May be type of bomb (pipe, etc.).

Hazard Sampling — Aerial Robots

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
Specify area to be covered by sampling on the map.

Select the map interaction button in the task specific area to enable specifying the area to be searched.
Select the required waypoints on the map to define the area.

Specify sampling sensors — Chemical, Biological, General Air, Radiation, etc. — The following items are updated with default values based upon the type of sampling required.

Sampling pattern — random, follow path, etc.
Sampling rate/density -
Sampling duration — probably time or number of data points recorded (?)

(Optional) Specify sampling pattern — random, follow path, etc.
(Optional) Specify sampling rate/density
(Optional) Specify sampling duration — probably time or number of data points recorded (?)
(Optional) Specify decontamination to be done - checkbox

How do we specify multiple sensors or sample types?

Hazard Sampling — Ground Robots

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
Specify area to be covered by sampling on the map.

Select the map interaction button in the task specific area to enable specifying the area to be searched.
Select the required waypoints on the map to define the area.

Specify sampling sensors — Chemical, Biological, General Air, Radiation, etc. — The following items are updated with default values based upon the type of sampling required.

Sampling pattern — random, follow path, etc.
Sampling rate/density -
Sampling duration — probably time or number of data points recorded (?)

(Optional) Specify sampling pattern — random, follow path, etc.
(Optional) Specify sampling rate/density
(Optional) Specify sampling duration — probably time or number of data points recorded (?)

How do we specify multiple sensors or sample types?

Gather Samples

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

Task Specific

f. “Sample for Chemical” = Simple (default) or Advance

g. “Collect samples” = Yes or No (default)

h. “Sample for Biological” = Simple (default) or Advance

i. “Collect samples” = Yes or No (default)

j. “Sample for Radiological” = Simple (default) or Advance

k. “Collect samples” = Yes or No (default)

l. “Sample pattern” = Evenly sampled

m. “Sample density” = sparse (faster)

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify Task specific items
Select area geometry — may be a point or a region. If it is region, you need to select a pattern. How do we select regions? Polygonal regions or a list of points?

Questions:

Do we need to have the pattern and density specified by sample type?

Number of samples: either density or specify a number such as collect four samples.

Guide Ambulatory Victims to Safety

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

Task Specific

f. “Enforcement” = None, Minimal (default), Most, All

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify enforcement

9. Select coverage area geometry

10. Select destination point (Egress point, shelter in place, etc.) — possibly multiple destinations

11. Duration in minutes — Within this area (or point), continue to patrol for victims until time is up.

12. If region select search pattern, if at point select radius.

13. Select points or polygonal regions.

14. Select a message type (?) — tenacity, vigilance (i.e. persistence in delivering message or being more forceful).

Questions:

A dangerous area — make sure that there is nothing in this area, humans or non-essential robots. Can we auto generate this task, i.e. before defusing a bomb?

Do we need a task that is to guide a particular individual to a particular point or any safe place? May be a victim to safety or a responder to some other place. This case cares about a particular person. Escort/Chaperon task — Chaperon knows proper behavior for the person.

Do we need the ability to request reinforcements?

Gathering imagery

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify enforcement

9. Select Geometry — Either a Single point and viewing arch geometry or a polygon

a. If single point, Persistence?

b. If polygon, Coverage = Follow path or Cover area

10. Task Specific

a. Vertical viewing angles (default 35 degrees to 80 degrees)

b. “Level of Detail” (similar to NIIRS) = General Level

c. Active hazard (e.g. chemical leak) or potential hazard (e.g. bomb)

2. Make any changes to above pre-filled fields

Notes:

This task could be broken into two tasks Aerial Surveillance and Hazard Surveillance

Curtis’ assumption is that viewing angle will specify the ground vs aerial vehicle.

This is a catch-all task: gather imagery of bombs, objects, victims, the previous surveillance tasks, a simple persistent surveillance. Do we want a catch-all task or separate tasks?

Gross Victim Assessment

This task provides a broad, quick classification of the victims into two classes ambulatory and non-ambulatory. This is not technical triage of an individual victim.

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

Task Specific

f. Coverage = Quick, Efficient, Through, Exhaustive

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify enforcement

9. Select area geometry

Note: Sandy suggested adding an additional field to the victim search task that represents a level of detail. It could be gross or technical or it could be a continuous scale, for example 1 (very gross) to 5 (very specific).

Scene Mapping

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

Task Specific

f. Focus

1. Perimeter - - what is the perimeter of the area

2. Interior (Obstacles) — All things inside the area

3. 3D

4. Find safe passage — What is the safest route between points

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify enforcement

9. Select area geometry

Notes:

Sandy: Select points and regions. If you have points can specify a radius, so the point becomes a circle within which the task should be done. This point plus X meters radius. Behind this building and ten meters around it.

Improved Communication

Select Division (may be done when first open program or defaults to last selected division if operator is at a multi-divisional level). — Results in the list of possible tasks defaulting to a list of division specific tasks and an option to access other non-division specific tasks.
Select Task type — results in automatically filling in the following fields with defaults

Task name
Start time — Now + 2 min
Interrupt ability — predefined default based upon task type
Priority level — predefined default based upon task type
Leader name — defaults to last leader name in a single division situation. Defaults to the appropriate leader based upon the selected division when there are multiple divisions for this particular user.

Task Specific

f. Focus

1. Point to point

2. Full area coverage

(Optional) Specify the task name
(Optional) Specify the task start time
(Optional) Specify the interrupt ability.
(Optional) Specify the task priority level.
(Optional) Specify the leader name.
(Optional) Modify enforcement

9. Select area geometry

10. (Sandy) Select monitoring pattern - if there are different groups, determine who needs help. How do you decide to move between the groups?

11. (Sandy) Assistance prioritization — Defines which different groups receive priority when multiple competing groups require assistance. For example, Turn robots into communication robots to assist groups with priority 6 or 7 but not those below 5.

Notes:

Will: This could conflict with the continuous behaviors.

Want an area of high communications, so if robots dealing with a bomb, want to ensure that the communications do not go out. May specify an area that in the future will require good communication covered. Would specify a time period during which the improved communication is required so that the robots can begin to move into position to provide the communication.

Place Robots in Safe Mode

Note: This is probably not a task but is something that needs to be available elsewhere, but where?

This requires selecting a place for the robots to remain stationary, perhaps a holding area were robots wait to be deployed from within the hot zone. It may be that we need to specify a zone that robots can go to or a specific item (a building, wall or car) for robots to go behind. UAVs may be able to simply move up to a higher altitude.

Decontaminate self

Solution: Should this task be auto-generated when the robot requires leaving the hot zone?

Curtis is thinking that all the fields are fixed because you do not want a priority or allowing the task to be interruptible. Could specify areas that the robots have entered that then sets a flag requiring the robot to decontaminate.

Sandy: If we have multiple tasks don’t want to decontaminate after each task. May need a field to indicate where to decontaminate if there are multiple decontamination sites.

Should we somehow mark the robots that require decontamination? So if we specify an area that robots enter and then require decon, may want to display something on the robot to indicate required decon.

Will: If the task can be done automatically, then should do it to reduce cognitive workload. This is specifying a specific task for a specific robot, which is not currently supported by the robot.

Sandy: May want to do decontamination prior to starting a new task, so that you are not decontaminating after a task and then starting a new task that requires the robot to decontaminated again.

Decon items on map can indicated which type of decon and if the decon site is out of supplies and cannot decon, or is overloaded with individuals to decon. This information can be used to determine where to take the robot (and victims) for decon automatically. Then if the robot cannot figure out where to go, then human can make the decision.

Depending upon the number of robots, you may need to force a robot to go to an area prior to decontamination.

Continuous Tasks

Searching for victims, bombs and suspicious items

These are continuous tasks that the robots do all the time while doing another primary task.

Solution: Set in the by robot tab? Should this be part of a default or active behavior section?

Sandy: We can keep a count of items found on either the by task or by robot panels.

Curtis: Objects should automatically appear on the screen.

The Continuous tasks will not have a completion time, but we do need to be able to degrade their priority.

Specifying this task may be a general search task and then choose to search for victims, bombs, and/or suspicious tasks. Also need to specify priorities for each of these options.

Do we need to also have a continuous task of searching for contaminants?

What the robot should do if it finds something. Stop for victims but not for objects. Or notify human if see a bomb to see what it should do.

Sandy considered this a modified version of the specific search tasks, but add an additional field for the task being continuous. May only process the sensor information based on what is available and the speed at which the robot needs to move.

Questions:

How do we indicate that a task should be specified and the task panel appears?
How do we indicate that information (mouse clicks) on the map without a tie to a particular task specification are to feed a task? Perhaps use the context menu to select “create task”, which marks the location of the click on the map and the point information appears in the task specification area.

Notes:

We may want a very basic Go to X Location in addition to specific tasks that incorporate the Go to X Location task. As an example, see the Hazard Sample task below. We need to revisit this issue.
The following items may be used to specify areas on the map in relation to a task: a point, a series of points, rectangles, triangles, and circles or ellipses.
We may want some shortcut buttons for specifying tasks on the interface.
We may want to treat spatial information as a data item and specify via the context menu.
If a task is known to require map interaction, it could be that when the task is selected that the mouse flips to the task specification task and only permits task edits. This could be bad or good. The benefit of the button to enable map specification information to be entered is that it is a conscious move by the operator.

Below use cases need to be revisited later in the process.

Task: Specify that a task should be entered.

Resource hauling task

Transport food or fluids to vicitms