 |
|
|||||||||
This paper and briefing will describe the methodology for instrumenting and evaluating a distributed virtual world used for training US government (USG) personnel to prepare and plan for USG involvement in an international crises. Initiated in 1998, this study was funded by the Department of Defense (DoD) C4ISR Cooperative Research Program (CCRP) and DARPA's Advanced Simulation Technology Thrust (ASTT) program.
ThoughtLink's objective was to develop a prototype Distributed Interactive Collaboration Environment (DICE) and assess its effectiveness in an experiment with an operational user. The experiment was conducted during a three-week period in October 1999.
To assess DICE's effectiveness for training, the October 1999 training audience was divided into two groups: a control group ("No DICE"), and the DICE group which performed all its work via DICE software, with limited face-to-face meetings. The No DICE group did all its work in face-to-face meetings, as in previous PDD 56 training exercises.
Both groups were given the same curriculum and the same virtual scenario: a deteriorating situation in North Korea that leads to a coup and possible reconciliation with South Korea. Each group was tasked to create a point paper outlining US interests in Korea and a pol-mil plan describing the integrated USG approach to the crisis.
The DICE group's training differed in several ways from previous PDD 56 exercises.
Mode of delivery. The DICE group received most of its training via distributed collaborative environment, with less face-to-face training (1.5 non-consecutive days total vs. 2.5 or 3 consecutive days in previous exercises).
-
Use of role players. Role-players added realism to the exercise and helped guide the group through its distributed work, discussing the Korean situation and National Security Council guidance, and directing their work process.
-
Evolving scenario. The scenario evolved over time to increase trainees' interest and involvement in the exercise. It is also more realistic. Previous exercises were static.
-
Increased access to resources. The DICE group could access on-line documentation related to PDD 56, USG agency roles and responsibilities, and Korea.
-
Ability to improve shared awareness. During development of the short paper, the No DICE group was divided into three functional working groups: economic, security, and diplomatic. As the functional working groups met in different rooms and therefore were physically separated, group-wide interaction was kept to a minimum. In contrast, any member of the DICE group could interact with any - or all - fellow members regardless of functional group. ThoughtLink currently is analyzing resulting data to determine whether or not this resulted in greater shared awareness.
The DICE/PDD 56 web site and its collaborative environment were instrumented to log who accessed the site, which pages they visited, for how long, which collaboration tools they used, who they collaborated with, and which files they downloaded or uploaded. All of these indices will be evaluated to determine measures of interest and to assess how trainees gathered information across functional groups (economic, security, or diplomatic). DICE allowed participants to review various sections of the point paper or pol-mil plan while they were being drafted.
Measures of interest included the following. Did the DICE group take advantage of pre-training opportunities? To what extent did it use available resources, including extra background on Korea, PDD 56, and government agency capabilities? Was the work undertaken throughout the three-week course of the exercise or at the last minute?
