The Role of a Distributed Interactive Collaboration Environment (DICE) for Interagency/Military Training

 

Julia Loughran

Marcy Stahl

ThoughtLink, Inc.

2009 Cantata Court

Vienna, VA  22182

703/281-5694

loughran@thoughtlink.com

mstahl@thoughtlink.com

 

Abstract

 

This paper describes ThoughtLink’s research into the use of low-cost gaming and web-based collaboration technologies to enhance training, coordination, and communication within the interagency community, particularly training for complex contingency operations[1] (CCOs).  This research, which began in 1998, is being funded by the Department of Defense (DoD) C4ISR Cooperative Research Program (CCRP) and the Defense Advanced Research Projects Agency (DARPA). The research will culminate at the end of this year with an experiment examining how a Distributed Interactive Collaboration Environment (DICE) can be used to enhance interagency training.  An annotated briefing documenting the results of this experiment will be available at the end of the calendar year.

 

The experiment will evaluate how DICE, a collection of commercial off-the-shelf (COTS) and government off-the-shelf (GOTS) products tailored with appropriate content, can support the interagency community as they train on the development of a political-military (pol-mil) plan for a hypothetical CCO.  Although the experiment will focus on training the interagency community in the pol-mil development process, DICE might also be used as an operational planning and rehearsal environment for real-world contingencies.  The DICE approach, using collaboration tools and applying them to specific domains, can be applied to many real-world issues and problems.

 

1.      Overview

 

The focus of this research is not about the development of new collaboration technologies.  There is already a multitude of COTS and GOTS products available today.  The focus, rather, is on how these technologies can be used to enhance training, communication, and information sharing. The experiment, planned for this fall, will explore how these technologies can be applied to interagency training for CCOs and will identify the potential benefits and pitfalls of their use.

 

Section 1 of this paper describes the project’s history and the scope of the research in fiscal year (FY) 1999; Section 2 describes the technologies and components of DICE; Section 3 provides the experiment details; and Section 4 summarizes the proposed benefits of DICE as well as its potential use for training subsets of a JTF staff. 

 

1.1  FY98 Work

 

The DICE concept is an outgrowth of ThoughtLink’s FY98 task for DARPA, in which we explored the applicability of low-cost collaboration and computer game technologies to Joint Task Force[2] (JTF) training in the area of operations other than war (OOTW)[3].  The DARPA sponsor was Mr. Dell Lunceford, Program Manager for the Advanced Simulation Technology Thrust (ASTT) program.  The ASTT program does advanced research in high payoff areas for the Joint Simulation Systems (JSIMS) program and the Joint Warfare Simulation (JWARS).

 

There were 3 phases to our FY98 work: understanding today’s JTF staff training, particularly for OOTW-related training, reviewing collaboration and computer game technologies, and defining a new training environment to augment, not replace, current JTF training methods.

 

An early observation in our review of today’s JTF training was that training occurred at two ends of the training spectrum, with a void in the middle.  At one end of the spectrum are low-cost, low-tech alternatives: academic training and seminar games.  These methods are used for small groups (2-200); have a narrow focus; and are relatively static (once developed, the content doesn’t radically change with each new training audience).  At the opposite end of the spectrum is the simulation-supported command post exercise (CPX).  This method is very expensive (typically $1M+) and involves a large training audience (100-1,000) supported by almost equally large training support organization.

 

It appeared that there is utility in developing computer-based training methods to fit in the middle of the spectrum: medium-cost and medium-fidelity.  Relevant research areas included web-based gaming environments, multiple user role playing games, strategic thinking games, and synchronous and asynchronous collaboration tools, see Figure 1. 

 

The product of our FY98 work was a report – “Applying Commercial Gaming and Collaboration Technologies to JTF Staff Training” [Loughran et al., 1999].  The report proposes a new training environment to augment the current JTF staff training methods.  This new environment would use collaboration technologies, together with a team of role players and facilitators, to help multiple geographically distributed participants work together.  The report is available on our web site, http://www.thoughtlink.com.


 

Figure 1. Spectrum of JTF Training and Areas of Evaluation

 

1.2  FY99 Work

 

In FY99, DARPA, under ASTT Program Manager Mr. Larry Willis, and CCRP, under Chairman Dr. David Alberts, are funding ThoughtLink to integrate COTS and GOTS collaboration tools, add appropriate content, and employ the integrated products and content, referred to as DICE, in an experiment in the fall.  This research will be conducted with an operational user.

 

The operational users are three organizations charged with training US government officials to respond to complex contingencies under Presidential Decision Directive (PDD) 56[4].  These organizations are: the National Defense University (NDU), the National Foreign Affairs Training Center (NFATC, the training arm of the Foreign Service Institute), and the US Army War College Peacekeeping Institute (USA PKI).

 

NDU, NFATC, and the USA PKI train government officials from multiple agencies in how to develop a cohesive political-military plan for the US response to complex contingencies.  Currently, they conduct the training once a year.  In its current form, the training audience meets together for 3 days for a combination of presentations, lectures, and a seminar-based simulation.  No technology is used to support the training, outside of the computers used to display some of the presentations to the training audience. 

 

Using DICE, we propose that the training audience will meet in person for less time and do part of the training while still at their office.  The focus of the fall experiment will be to identify the benefits to this form of training.  The assessment of DICE will occur in a two-phases.  Phase 1 was a March 1999 PDD-56 training exercise, described in Section 3.2.  No collaboration technology was used for either planning or training, no gaming strategies were used to provide a time-phased scenario, and there was minimal use of role-players.  During the March training event, data were collected (both quantitative and subjective) to serve as a baseline, describing the training process without the benefit of technology.  Phase 2, to be conducted in the fall of 1999.   It will include some limited face-to-face meetings (less than occurred in Phase 1) and it will also include the use of DICE.  In Phase 2, the training audience will be instructed in the use of DICE and asked to perform a subset of the tasks from Phase 1.  Data will be collected and compared, where feasible, to the Phase 1 data.

 

In support of this effort, employees from the Naval Air Warfare Center, Training Service Directorate (NAWCTSD), in Orlando, FL, are conducting a job task analysis of the current training and helping with the definition of the measures for the fall experiment. NAWCTSD personnel supporting this task include: Dr. David Fowlkes, Ms. Adriana Martinez, LtCol Dean Marvin, Ms. Karen McBee, and Mr. Patrick O’Brien.  In addition to the support provided by NAWCTSD, subject matter experts Mr. John Howard Eisenhour and Amb. Edward Marks (ret.), Greenleaf Point, L.L.C., are contributing to the project.

 

Major milestones for the project are:

- March 1999.  Observe PDD-56 Brave Knight training exercise.

- Spring 1999.  Develop and collect requirements for DICE technology; define the PDD 56 training objectives and approach.

- Summer 1999.  Integrate COTS/GOTS products with PDD-56 training content to produce the DICE prototype.

- Fall 1999.  Use DICE in a training experiment.

- December 1999.  Document the results of the fall experiment.

 

2.  DICE Overview

 

DICE is not a specific product.  It is a concept for integrating a collection of collaboration tools, enhancing those tools with domain specific content, and employing the tools and content using distributed gaming and role playing techniques.  The choice of tools can vary depending on the operational user’s requirements.  Some organizations already use collaboration tools and may be committed to a specific set of tools, such as Lotus Notes.  Other users may have varying requirements, depending on the type of collaboration and information sharing they want to do, and their purposes for using the tools might vary (e.g., training, planning, and knowledge management).  Other requirements to be considered for each different potential user community include bandwidth availability, security issues, and cost.

 

Despite the variability that might exist across different users’ requirements, there are certain common features.  The key features of DICE follow.

 

·        It runs over a network (WAN/LAN);

·        It supports collaboration between distributed participants.  Users can share information, work together on projects, ask questions, and access outside experts or trainers;

·        It provides a persistent environment (archiving capability), meaning that documents are stored and available for retrieval and sharing amongst participants;

·        It is platform-independent, thus it can be used on multiple kinds of computers.

 

A desired feature for DICE is also a low-fidelity simulation for stimulating the training audience and for conducting what-if analyses in an operational environment.  A computer-based simulation will not be part of the fall experiment, but instead, human role players will provide the simulation.

 

One of the reasons that this experiment’s focus is on the process of applying technologies versus the technologies themselves is that, although collaboration technologies have been available for many years, knowing how to combine and apply these technologies is not widespread in the DoD or other government agencies. Just as we had to learn to incorporate e-mail into our daily work schedule, we need to explore how relatively low-cost collaboration tools and gaming techniques can be used for both operational and training applications.  The DICE experiment will identify the potential benefits and pitfalls of using collaboration technologies to support the interagency community in training and planning for CCOs.

 

Collaboration tools come in two flavors. Asynchronous collaboration occurs when people communicate with each other at different times, and synchronous collaboration is when people are collaborating at the same time.  Some examples of asynchronous collaboration tools include e-mail, newsgroups, and bulletin boards. Synchronous collaboration tools are not used as commonly as the asynchronous collaboration tools.  One example of a synchronous collaboration tool that is growing in popularity is America On-line’s real-time chat capability called Instant Messenger.  This free software will alert you whenever someone you have designated as a person of interest is on-line.

 

Other synchronous collaboration capabilities such as video teleconferencing (VTC) are used less frequently, but many tools exist and many are free of charge. For instance, Netscape’s SuiteSpot includes the Collabra Server which allows users to share information through “virtual” discussion rooms, and Microsoft has a free product called NetMeeting that supports synchronous collaboration and runs with their Internet Explorer browser.   NetMeeting is probably one of the most popular free synchronous collaboration tools. It is being used in many government projects to support synchronous collaboration.  In addition, there are many commercial web sites that provide a text chat capability for free.  The use of this capability requires that you have software to support the chat session and generally requires that you register with the site.

 

In addition to these commercial tools, there are a variety of government owned tools under development to support collaboration.  These tools are described in more detail in Section 2.2. 

 

Collaboration in general, and DICE more specifically, seems like a good fit for distributed training applications.  In PDD-56 training, the training audience includes disparate groups from many agencies, they’re short on time, and they could benefit from a reach-back capability to their offices, to provide access to experts who are not present at the training.

 

2.1  Capabilities

 

As was mentioned earlier, the actual implementation of DICE can vary depending on the user’s requirements.  In this section, we will discuss the DICE capabilities important for PDD-56 training.  Since we are still gathering requirements and evaluating the COTS/GOTS tools that might be used, these capabilities may or may not be incorporated into the fall experiment.

 

The DICE being developed for PDD-56 training will most likely be a collection of software products that operate in conjunction with an Internet connection.  Because the training audience needs to have the capability to access training materials prior to the training there will most likely be a web site established to provide a persistent workspace.  This workspace will be shared by all users and contain electronic documents, images, audio clips, and videos related to PDD-56 and the training scenario for the fall experiment.  The site can also contain the details about the training, including when and where the face-to-face portion of the training will take place.

 

This new PDD-56 web site will also include a variety of collaboration tools.  For instance, a bulletin board capability will be provided so the training audience can direct questions to the trainers or to other members of the training audience should they have a question about a particular agency. They will also have the capability to send e-mail to the trainers and/or other members of the training audience. Since e-mail is generally not captured and archived, the PDD-56 DICE application might take advantage of an e-mail archiving capability.  For instance, a tool called Hypermail captures and translates e-mail between a group of people and displays it on a web site.  These archived e-mails can be reviewed by members of the group and might contain information to help clarify something, or be used to evaluate the types of communication conducted during the training event.

 

Some of the synchronous collaboration tools that will most likely be available include VTC, audio, text chat, and shared white boards.  These collaboration tools will allow the training audience to participate from their home stations and can potentially increase understanding prior to the face-to-face portion of the training.

 

One of the requirements for this particular application of DICE is that it must be very easy to use.  One of the features that some of the COTS/GOTS products include that make them easier to use is a “room-based” navigation scheme.  With this scheme, users can navigate to different areas of the collaboration space by selecting from a 2D or 3D map display.  The 2D map may have different rooms for each of the agencies.  By selecting a particular agency room, the user has access that agency’s information, a list of people to contact for more information, a current listing of other people who are in that room at that particular time, and any documents, audio, or video that might persist in that room.  When two people are in one of these “virtual” rooms at the same time, they can have a conversation via the text chat tool or VTC if they have a sound card and digital camera on their computer.  Using this room-based navigation scheme, another “virtual” room might be related to the hypothetical scenario or there could be individual rooms for each of the functional working groups writing portions of the pol-mil plan.

 

The PDD-56 DICE training application will feature a great deal of document sharing capabilities since distributed participants will be working on different components of the pol-mil plan.  Therefore, this DICE application must include a version control capability and it must be easy for users to add documents from their computers to DICE.  In some of the COTS/GOTS products, users can “drag-and-drop” a document from their desktop to the virtual environment.  In other tools, users select a Browse button and, if using a PC environment, it opens the standard Windows file navigation window.

 

For the purposes of the fall experiment, we will need to capture data about the events that occur in DICE.  Therefore there is a requirement for tools that create user logs or that can be instrumented so we can collect the data pertinent to the analysis of the training event.  Many of the COTS/GOTS products available do not provide this capability and we will most likely have to develop the programs for gathering this data.

 

2.2  Collaboration Tools Under Consideration

 

There are a number of relatively low-cost or free COTS and GOTS products that may be used in our experiment.  We are currently mapping the training requirements to the features incorporated into these products to identify the best tool or combination of tools.  This process will be completed by the end of May.  Some of these tools feature only a synchronous or asynchronous collaboration capability while others are fully integrated products that provide the complete range of support.  A subset of the available tools is described below to provide an overview of their functionality.

 

The two most popular forms of synchronous tools (video and audio teleconferencing and text chat) are probably Microsoft’s NetMeeting and CU-See-Me by WhitePine Software Inc.  Both tools have a whiteboard and application sharing capability.  NetMeeting is available for free from Microsoft’s web site (http://www.microsoft.com) and CU-See-Me has both a free and a commercial version.  The free version is available from Cornell University where the technology was first developed under research from the National Science Foundation and the commercial version is available from WhitePine Software.  Many of the tools that have limited synchronous capability but more robust archiving capability use either NetMeeting or CU-See-Me to enrich the collaboration environment.  One of the advantages to these two products is that they are both compatible with the H.323 standard and therefore a person that was using the CU-See-Me software should be able to collaborate with someone using the NetMeeting software.

 

Some of the COTS candidates for asynchronous capability include Instinctive Technology’s eRoom, Involv’s Teamspace product, TeamWave Software Ltd.’s TeamWave workplace, and PlaceWare.  All of these products are web-based and provide a persistent workspace for multiple document types and URLs, version control for documents, a bulletin board capability, a polling capability allowing users to vote on issues, and text chat in an easy-to-use interface.  Most of these groupware products operate with the use of Internet browser software along with the individual client software associated with each product.

 

Some of the collaboration products being developed and used by the government include Mitre’s Collaborative Virtual Workstation (CVW), GTE’s Intelligent WorkStation (IWS), and SPAWAR’s Odyssey tool. Both CVW and Odyssey use the room metaphor described earlier for navigation and were developed by making enhancements to LambdaMOO, a popular and freely available multi-user domain (MUD) software originally developed at Xerox PARC. 

 

CVW supports both synchronous and asynchronous collaboration.  As you navigate through the different virtual rooms, you can see the people and objects in each room.  You can make copies of these objects and/or collaborate with someone about them.  For instance, 3 people in a room can be looking at the same document and making comments about it using the text chat tool.  Different people can have control over the document and make the agreed-upon changes.  After the document has been revised, the new document will be in this room for other users to collect and copy to their desktop, or they can post questions and/or comments for others to review at a later time.  CVW has been used extensively in the intelligence community and it is also the collaboration tool incorporated into COMPASS, the Common Operational Modeling, Planning, and Simulation System.  COMPASS is middleware used to share Global Command and Control System (GCCS) data across multiple users.

 

IWS is a web browser-based tool that has similar capabilities to CVW with a slightly different development approach and interface.  It supports both synchronous and asynchronous collaboration.  It was developed using PlaceWare, a COTS collaboration tool, as a base.  IWS is the collaboration component for the Joint Intelligence Virtual Architecture (JIVA).  The focus of JIVA is the “modernization of intelligence analytical processes and methodologies.” (http://www.mews.org/jto/jivva.html)

 

 The Space and Naval Warfare Systems Command (SPAWAR) has developed a collaboration tool called Odyssey.  It is built upon the COTS tool called Facilitate.com.  Odyssey is being used in the Adaptive Courses of Action (ACOA) Advanced Concept Technology Demonstration (ACTD).  ACOA is funded by DARPA and the Advanced Information Technical Services Joint Program Office (AITS JPO) and is intended to revolutionize the joint planning process through the use of collaboration tools and other emerging information technologies.  Odyssey, like CVW and IWS, supports synchronous and asynchronous collaboration through a variety of integrated tools.

 

All three of these products are being incorporated and supported in the Defense Information Infrastructure Common Operating Environment and are being used in fairly large experiments.

 

3.  FY99 Work and Experiment

 

3.1  PDD-56 Background

 

 ”The PDD defines “complex contingency operations” as peace operations .  . . and unless otherwise directed, this PDD does not apply to domestic disaster relief or to relatively routine or small-scale operations, nor to military operations conducted in defense of U.S. citizens, territory, or property, including counter-terrorism and hostage-rescue operations and international armed conflict.” [White House Paper on PDD-56, 1997]

 

The motivation for PDD-56 stems from problems experienced by the US government while planning and conducting international crisis intervention missions, including the Grenada invasion and Somalia.  The lack of integration between the civilian agencies and the military, but also amongst the civilian agencies, led to the realization that an integrated plan was needed.  “When identifying tasks and the resources to perform them, the absence of links between the civilian and military components of these missions led to undesirable outcomes: neglect of civil police requirements and other law and order functions, resource imbalances between humanitarian relief initiatives and military operations, and lack of attention to human rights considerations.  The uncoordinated planning produced serious differences in assumptions, concepts, policy recommendations, and plans....The separate planning processes seldom identified a complete set of strategic objectives and operational needs for these interventions.”  [Walsh and Harwood, 1998]

 

NSC interest in defining PDD-56 arose from experience planning for US and UN missions in Haiti in late 1994, “...US Atlantic Command identified a host of questions related to tasks and responsibilities that were well beyond the capability of military forces committed to the operation.  These questions from the military required fully coordinated answers from other parts of the US government.” [Walsh and Harwood, 1998]  These queries eventually led to an integrated civil-military plan, developed by the NSC with support from Pentagon planners and input from other agencies.

 

Although this PDD was signed in May 1997 and the US has participated in complex contingency operations since then, PDD-56 was invoked for the first time in April 1999, for the Kosovo crisis.  Planning for this complex CCO began in October 1998, NATO bombing of Yugoslavia began in late March, and PDD-56 was invoked in mid-April.

 

When PDD-56 is invoked, the Deputies Committee of the National Security Council creates an Executive Committee (EXCOM), composed primarily of Deputy Assistant Secretaries (DAS) from all participating agencies, to oversee day-to-day management of US participation.  The Deputies Committee tasks the development of a pol-mil plan and assigns specific responsibilities to the EXCOM officials, who are then responsible for preparing their section of an integrated pol-mil plan. The plan includes milestones, measures of success, and planning for the transition to a follow-on operation.  After the plan is written, the main elements are rehearsed by the EXCOM, to identify differences among agencies over mission objectives, timing, and resources, and to underscore accountability of agency officials.  [White House Paper on PDD-56, 1997]

 

PDD-56 charges three organizations to develop and conduct an interagency training program to be held annually.  These organizations, NDU, NFATC, and PKI, have held one training exercise per year since 1997.

 

The major features of PDD-56 to be trained are the creation of the pol-mil plan and the interagency rehearsal of the pol-mil plan.  Other training objectives include learning about other agencies: their culture, capabilities and limitations, and providing an opportunity for networking for the training audience.

 

3.2  March 1999 PDD-56 Training Exercise

 

The PDD-56 training audience is composed of three levels of US government (USG) officials: desk/action officers, directors, and DAS.  The desk officers and directors participate in functional working groups to develop a pol-mil plan, then the DAS are briefed on the plan and role-play in the EXCOM rehearsal of the plan. 

 

The March 1999 training audience included about 45 people from 16 different agencies and organizations.  They were from functional offices; no regional specialists from any agency attended the exercise.

 

The scenario was set in Poland[5] and included a broad array of troubling situations. In order to ensure that many USG agencies would have a role in responding to the crisis, these situations were numerous and somewhat exaggerated.  The major scenario elements were:

 

·        A large and sudden influx of refugees from Russia, straining Poland’s humanitarian assistance capabilities;

·        An economic slowdown due to Russia’s economic collapse;

·        The potential deterioration of Soviet-era infrastructure, including the transportation system, water and power grids, and nuclear power plants;

·        The rise of organized crime, in drugs, money laundering, and smuggling, including smuggling of fissile material.

 

The scenario also posed specific requests for aid and assistance from the Polish government that would need review by the training audience, including a review of customs procedures, police training, advice on economic restructuring of the agricultural sector, and a public information campaign.

 

The training event took place over three days: March 24-26, 1999 at NFATC in Arlington, Virginia.  All participants were co-located for the exercise.  The days were a mix of briefings, panels, discussions, and small group work. On Day 1, panels and speakers discussed experiences and lessons learned from previous CCO; the scenario was introduced as a Global News Network video; then the audience broke into agency groups to discuss their agency’s coordinated response to the emerging crisis.  On Day 2, a number of agencies gave short presentations about their capabilities.  Then the audience was divided up again, this time into six functional working groups (diplomatic, economic, environmental, health, infrastructure, security) to write the part of the pol-mil plan that describes the objectives and desired end state for each functional area. The DAS arrived on Day 3, were briefed on functional areas relevant to their agency, and then participated in an EXCOM rehearsal. 

 

3.3 Data Collection and Results

 

We had 7 observers at the exercise, from ThoughtLink, Greenleaf Point LLC, and NAWC-TSD. We sat in on all of the briefings and the agency and functional groups.  We took notes and informally interviewed some of the participants.  We also distributed questionnaires at the end of each day to the training audience.  The training audience was asked about the following topics: their work background, level of computer access and expertise, interest in using technology for future PDD-56 training, level of satisfaction with the briefings and panels on each day, knowledge of PDD-56 after the training, and the degree to which they networked with other participants.  The number of participants each day varied, decreasing each day, and not all participants on a given day filled out questionnaires.  Figure 2 shows, for each day, the number of participants in the training audience and the number who filled out surveys.  There are no data for the total number of participants on Day 3, but we estimate that number at about 25.


Figure 2.  Number of Participants and Survey Respondents by Day

 

Most participants (27 of 29) were accustomed to regularly coordinating/cooperating with other agencies in their work and had been involved in several real world events requiring interagency coordination.  On average, they were not very familiar with PDD-56: an average of 2.6 on a scale of 1-5 (1 is not familiar, 5 is very familiar).  Almost all had desktop access to e-mail and the Internet and most rated themselves as intermediate or expert in their level of computer knowledge. 

 

Overall, they learned something new during the exercise.  On Day 1, 27 of 29 said they learned something new about PDD-56. One comment from a member of the training audience was:

 

“Good experience.  I know a lot about PDD-56, but at every event, I learn something new.”

 

 Twenty-two of 24 respondents said they knew more about the interagency process after Day 2.  They had a high level of satisfaction with the briefings and panels.  Table 1. lists the training objectives and the average rating among participants on how well they felt those training objectives were met on a 1-5 scale.

 

Training Objective

Average Rating

Develop understanding for intent behind PDD-56

4.2

Develop appreciation for diversity of agencies

4.1

Learn other agencies’ capabilities and limitations

3.6

Develop personal contact with other agencies

3.6

Learn structure of pol-mil plan

3.5

Learn how other agencies function

2.8

 

Table 1.  Participant Ratings on Achievement of Training Objectives

 

The following results are of particular interest to us as we plan for the fall experiment using DICE: 

 

·        Time.  For most participants, time was short and they had difficulty getting 3 days away from the office.  In fact, only 8 of  the approximately 45 training audience members attended all three days.  In later surveys given to a subset of the audience, an exercise length of 1 to 2 days was preferred. 

·        Ratio of passive to active learning. Table 2 shows that most of Day 1 and 2 were spent listening to briefings and panels. From the observer point of view, this quickly became tiresome and the level of interest flagged.  Some comments were made about this in the training audience questionnaires.

 

“I did not find agency briefings at all useful; after two hours absorption rates fall to zero.”

 

“Would have liked longer working sections with other agencies where you really learn, within the context of this exercise, the capabilities/mandates, and limitations of the agencies.”

 

 

Day 1

Day 2

Day 3

Total time (minutes)

Time spent listening (minutes)

270

225

varied from 0 to about 45

495 to 540

Time spent working (minutes)

45

100

135

280

 

Table 2.  Allocation of Time Over Training Days

 

·        Interest in using web technology.  Most (21 of 27) wanted to use technology for distance learning for future exercises. All respondents wanted to use technology to get more information about USG agencies.  Some people wanted more detailed information about the agencies that were briefed at the exercise; some people wanted information about the agencies for which presentations were not given.  Most respondents (25 of 29) wanted to access administrative information about the training online.

·        Respondents on Day 3 reported meeting 8.9 people on average.  Some would have like more and one participant suggested incorporating a social event into future exercises.  This aspect of the training is an important benefit that must be preserved when technology is introduced into the training. 

 

“Most useful benefit was to interact with other agency personnel in thinking through a USG response to a scenario.”

 

·        Seventeen of 24 would have found it useful to have access to other groups during development of the pol-mil plan.  In the March event, this would have required someone from one group going over to the room of another group and asking about their work.  In the future, this could be done via collaboration tools.

·        Thirteen of 14 would have liked to learn what the outcome was of the pol-mil plan when applied to the crisis.  In the future, we want to explore incorporating a simulation to link participant decisions to outcomes.  This concept is discussed further in the context of JTF staff training in [Loughran et al., 1999].

·        Poor performance on the quiz. We asked six questions about the PDD itself and the majority of the training audience filling out the quiz failed to answer them correctly.  This information was contained in the PDD-56 handbooks distributed at the training event.  Although the questions covered very basic material, for example, what occurs when PDD-56 is invoked, one explanation for the poor performance might be that not much time was spent covering these areas in the actual training event.

 

Observations and participant feedback from March exercise identified the following potential improvements for future PDD-56 training events, irrespective of the use of technology:

 

·        Presence of a facilitator/trainer in each working group

·        Need for training aids, e.g., a checklist of items for different functional groups when developing their group’s response to the crisis, a glossary of terms, etc., and

·        Need for increased feedback to the training audience on their performance.

 

3.4    Plan for the fall experiment

 

We will conduct a one to three day experiment in the fall to explore the ways in which DICE can be used and where it is most useful for PDD-56 training.  This section describes the fall experiment in more detail.  Note that the experiment plans will continue to evolve during the next few months.

 

3.4.1  Assumptions for Using DICE

 

Based on our observations of the March exercise and on feedback from the training audience, we are keeping the following precepts in mind while planning the fall exercise.

 

·        Face-to-face interaction is critical for the training audience and must be maintained at some level.  This interaction promotes greater understanding of other agencies’ cultures, responsibilities, and limitations, and promotes networking among participants.  This means that the fall exercise cannot be entirely distributed.

·        Shortening the total length of the exercise is a concern of many participants.  One to two days was the preferred length, from surveys we did of a small subset of the audience after the exercise.

·        Emphasizing active learning, which corresponds to an information pull approach, is thought to be a more engaging and therefore better form of training.  The ratio of passive learning (or information push) to active learning, shown earlier in Table 3.1, was high in the March exercise. 

·        Providing feedback to the training audience is important for their learning.

·        Providing a more realistic information environment to the participants is good.  In real life, people who develop a pol-mil plan have access to a variety of information about the contingency, which might include intelligence reports, State Department cables, regional expertise within an agency, and open source information, most notably on the Internet.

 

3.4.2  Use of DICE

 

This section explores how DICE can be used to enhance the curriculum/structure of the existing PDD-56 training program.   The focus of our research is on the benefits and/or pitfalls of DICE and not on how a new curriculum might enhance the PDD-56 training.

 

The point of the fall experiment is to learn how DICE can most effectively support interagency training.  The key questions are: which parts of the training are most effective in a face-to-face setting; which parts are best done in a distributed way (to reduce the total time out of the office or to speed learning); and how much of the distributed collaborative work can be done asynchronously vs. synchronously.

 

The answers to these questions are not clear-cut.  Therefore the fall experiment will be an exploration of these new techniques and will be a learning experience for us and for the planners of PDD-56 training.  We will encourage the training audience to explore different ways of using DICE to create work products (e.g., the pol-mil plan) and for several other parts of the training.  Based on their feedback and our observations, we’ll be able to describe with greater confidence how DICE can be used to support future official PDD-56 training events.

 

DICE for Pre-training.

 

The training audience will access a DICE web site to learn the necessary background information for the training event.  Information about PDD-56, the scenario, and agency capabilities will be accessible from a web site, in advance of the exercise.  This information might be in multiple forms: text, audio, or video.  Hyperlinks will be included to agency web sites and to sites with additional country information for the scenario.  Administrative information for the training: the schedule, maps to the facility, etc., will also be posted on the web site.

 

Using the collaboration tools, the training audience will be able to interact with the exercise planners, who are subject matter experts in PDD-56, prior to the training.  The planners can answer questions about the PDD, the scenario, or the training event.

 

Given the need to supply the training audience with a certain amount of background information, along with the desire to shorten the time of at least the co-located training ( if not the total length of the exercise) using DICE for pre-training seems useful.  In previous exercises, most of this background information is provided to the training audience in real time during the exercise.  It is probably faster for them to read this information.  In addition, audience surveys showed that most of the training audience wanted additional information about agencies, including more in-depth information than was presented.