The Role of a Distributed Interactive Collaboration Environment (DICE) for Interagency/Military Training

 

Julia Loughran

Marcy Stahl

ThoughtLink, Inc.

2009 Cantata Court

Vienna, VA  22182

703/281-5694

loughran@thoughtlink.com

mstahl@thoughtlink.com

 

Abstract

 

This paper describes ThoughtLink’s research into the use of low-cost gaming and web-based collaboration technologies to enhance training, coordination, and communication within the interagency community, particularly training for complex contingency operations[1] (CCOs).  This research, which began in 1998, is being funded by the Department of Defense (DoD) C4ISR Cooperative Research Program (CCRP) and the Defense Advanced Research Projects Agency (DARPA). The research will culminate at the end of this year with an experiment examining how a Distributed Interactive Collaboration Environment (DICE) can be used to enhance interagency training.  An annotated briefing documenting the results of this experiment will be available at the end of the calendar year.

 

The experiment will evaluate how DICE, a collection of commercial off-the-shelf (COTS) and government off-the-shelf (GOTS) products tailored with appropriate content, can support the interagency community as they train on the development of a political-military (pol-mil) plan for a hypothetical CCO.  Although the experiment will focus on training the interagency community in the pol-mil development process, DICE might also be used as an operational planning and rehearsal environment for real-world contingencies.  The DICE approach, using collaboration tools and applying them to specific domains, can be applied to many real-world issues and problems.

 

1.      Overview

 

The focus of this research is not about the development of new collaboration technologies.  There is already a multitude of COTS and GOTS products available today.  The focus, rather, is on how these technologies can be used to enhance training, communication, and information sharing. The experiment, planned for this fall, will explore how these technologies can be applied to interagency training for CCOs and will identify the potential benefits and pitfalls of their use.

 

Section 1 of this paper describes the project’s history and the scope of the research in fiscal year (FY) 1999; Section 2 describes the technologies and components of DICE; Section 3 provides the experiment details; and Section 4 summarizes the proposed benefits of DICE as well as its potential use for training subsets of a JTF staff. 

 

1.1  FY98 Work

 

The DICE concept is an outgrowth of ThoughtLink’s FY98 task for DARPA, in which we explored the applicability of low-cost collaboration and computer game technologies to Joint Task Force[2] (JTF) training in the area of operations other than war (OOTW)[3].  The DARPA sponsor was Mr. Dell Lunceford, Program Manager for the Advanced Simulation Technology Thrust (ASTT) program.  The ASTT program does advanced research in high payoff areas for the Joint Simulation Systems (JSIMS) program and the Joint Warfare Simulation (JWARS).

 

There were 3 phases to our FY98 work: understanding today’s JTF staff training, particularly for OOTW-related training, reviewing collaboration and computer game technologies, and defining a new training environment to augment, not replace, current JTF training methods.

 

An early observation in our review of today’s JTF training was that training occurred at two ends of the training spectrum, with a void in the middle.  At one end of the spectrum are low-cost, low-tech alternatives: academic training and seminar games.  These methods are used for small groups (2-200); have a narrow focus; and are relatively static (once developed, the content doesn’t radically change with each new training audience).  At the opposite end of the spectrum is the simulation-supported command post exercise (CPX).  This method is very expensive (typically $1M+) and involves a large training audience (100-1,000) supported by almost equally large training support organization.

 

It appeared that there is utility in developing computer-based training methods to fit in the middle of the spectrum: medium-cost and medium-fidelity.  Relevant research areas included web-based gaming environments, multiple user role playing games, strategic thinking games, and synchronous and asynchronous collaboration tools, see Figure 1. 

 

The product of our FY98 work was a report – “Applying Commercial Gaming and Collaboration Technologies to JTF Staff Training” [Loughran et al., 1999].  The report proposes a new training environment to augment the current JTF staff training methods.  This new environment would use collaboration technologies, together with a team of role players and facilitators, to help multiple geographically distributed participants work together.  The report is available on our web site, http://www.thoughtlink.com.


 

Figure 1. Spectrum of JTF Training and Areas of Evaluation

 

1.2  FY99 Work

 

In FY99, DARPA, under ASTT Program Manager Mr. Larry Willis, and CCRP, under Chairman Dr. David Alberts, are funding ThoughtLink to integrate COTS and GOTS collaboration tools, add appropriate content, and employ the integrated products and content, referred to as DICE, in an experiment in the fall.  This research will be conducted with an operational user.

 

The operational users are three organizations charged with training US government officials to respond to complex contingencies under Presidential Decision Directive (PDD) 56[4].  These organizations are: the National Defense University (NDU), the National Foreign Affairs Training Center (NFATC, the training arm of the Foreign Service Institute), and the US Army War College Peacekeeping Institute (USA PKI).

 

NDU, NFATC, and the USA PKI train government officials from multiple agencies in how to develop a cohesive political-military plan for the US response to complex contingencies.  Currently, they conduct the training once a year.  In its current form, the training audience meets together for 3 days for a combination of presentations, lectures, and a seminar-based simulation.  No technology is used to support the training, outside of the computers used to display some of the presentations to the training audience. 

 

Using DICE, we propose that the training audience will meet in person for less time and do part of the training while still at their office.  The focus of the fall experiment will be to identify the benefits to this form of training.  The assessment of DICE will occur in a two-phases.  Phase 1 was a March 1999 PDD-56 training exercise, described in Section 3.2.  No collaboration technology was used for either planning or training, no gaming strategies were used to provide a time-phased scenario, and there was minimal use of role-players.  During the March training event, data were collected (both quantitative and subjective) to serve as a baseline, describing the training process without the benefit of technology.  Phase 2, to be conducted in the fall of 1999.   It will include some limited face-to-face meetings (less than occurred in Phase 1) and it will also include the use of DICE.  In Phase 2, the training audience will be instructed in the use of DICE and asked to perform a subset of the tasks from Phase 1.  Data will be collected and compared, where feasible, to the Phase 1 data.

 

In support of this effort, employees from the Naval Air Warfare Center, Training Service Directorate (NAWCTSD), in Orlando, FL, are conducting a job task analysis of the current training and helping with the definition of the measures for the fall experiment. NAWCTSD personnel supporting this task include: Dr. David Fowlkes, Ms. Adriana Martinez, LtCol Dean Marvin, Ms. Karen McBee, and Mr. Patrick O’Brien.  In addition to the support provided by NAWCTSD, subject matter experts Mr. John Howard Eisenhour and Amb. Edward Marks (ret.), Greenleaf Point, L.L.C., are contributing to the project.

 

Major milestones for the project are:

- March 1999.  Observe PDD-56 Brave Knight training exercise.

- Spring 1999.  Develop and collect requirements for DICE technology; define the PDD 56 training objectives and approach.

- Summer 1999.  Integrate COTS/GOTS products with PDD-56 training content to produce the DICE prototype.

- Fall 1999.  Use DICE in a training experiment.

- December 1999.  Document the results of the fall experiment.

 

2.  DICE Overview

 

DICE is not a specific product.  It is a concept for integrating a collection of collaboration tools, enhancing those tools with domain specific content, and employing the tools and content using distributed gaming and role playing techniques.  The choice of tools can vary depending on the operational user’s requirements.  Some organizations already use collaboration tools and may be committed to a specific set of tools, such as Lotus Notes.  Other users may have varying requirements, depending on the type of collaboration and information sharing they want to do, and their purposes for using the tools might vary (e.g., training, planning, and knowledge management).  Other requirements to be considered for each different potential user community include bandwidth availability, security issues, and cost.

 

Despite the variability that might exist across different users’ requirements, there are certain common features.  The key features of DICE follow.

 

·        It runs over a network (WAN/LAN);

·        It supports collaboration between distributed participants.  Users can share information, work together on projects, ask questions, and access outside experts or trainers;

·        It provides a persistent environment (archiving capability), meaning that documents are stored and available for retrieval and sharing amongst participants;

·        It is platform-independent, thus it can be used on multiple kinds of computers.

 

A desired feature for DICE is also a low-fidelity simulation for stimulating the training audience and for conducting what-if analyses in an operational environment.  A computer-based simulation will not be part of the fall experiment, but instead, human role players will provide the simulation.

 

One of the reasons that this experiment’s focus is on the process of applying technologies versus the technologies themselves is that, although collaboration technologies have been available for many years, knowing how to combine and apply these technologies is not widespread in the DoD or other government agencies. Just as we had to learn to incorporate e-mail into our daily work schedule, we need to explore how relatively low-cost collaboration tools and gaming techniques can be used for both operational and training applications.  The DICE experiment will identify the potential benefits and pitfalls of using collaboration technologies to support the interagency community in training and planning for CCOs.

 

Collaboration tools come in two flavors. Asynchronous collaboration occurs when people communicate with each other at different times, and synchronous collaboration is when people are collaborating at the same time.  Some examples of asynchronous collaboration tools include e-mail, newsgroups, and bulletin boards. Synchronous collaboration tools are not used as commonly as the asynchronous collaboration tools.  One example of a synchronous collaboration tool that is growing in popularity is America On-line’s real-time chat capability called Instant Messenger.  This free software will alert you whenever someone you have designated as a person of interest is on-line.

 

Other synchronous collaboration capabilities such as video teleconferencing (VTC) are used less frequently, but many tools exist and many are free of charge. For instance, Netscape’s SuiteSpot includes the Collabra Server which allows users to share information through “virtual” discussion rooms, and Microsoft has a free product called NetMeeting that supports synchronous collaboration and runs with their Internet Explorer browser.   NetMeeting is probably one of the most popular free synchronous collaboration tools. It is being used in many government projects to support synchronous collaboration.  In addition, there are many commercial web sites that provide a text chat capability for free.  The use of this capability requires that you have software to support the chat session and generally requires that you register with the site.

 

In addition to these commercial tools, there are a variety of government owned tools under development to support collaboration.  These tools are described in more detail in Section 2.2. 

 

Collaboration in general, and DICE more specifically, seems like a good fit for distributed training applications.  In PDD-56 training, the training audience includes disparate groups from many agencies, they’re short on time, and they could benefit from a reach-back capability to their offices, to provide access to experts who are not present at the training.

 

2.1  Capabilities

 

As was mentioned earlier, the actual implementation of DICE can vary depending on the user’s requirements.  In this section, we will discuss the DICE capabilities important for PDD-56 training.  Since we are still gathering requirements and evaluating the COTS/GOTS tools that might be used, these capabilities may or may not be incorporated into the fall experiment.

 

The DICE being developed for PDD-56 training will most likely be a collection of software products that operate in conjunction with an Internet connection.  Because the training audience needs to have the capability to access training materials prior to the training there will most likely be a web site established to provide a persistent workspace.  This workspace will be shared by all users and contain electronic documents, images, audio clips, and videos related to PDD-56 and the training scenario for the fall experiment.  The site can also contain the details about the training, including when and where the face-to-face portion of the training will take place.

 

This new PDD-56 web site will also include a variety of collaboration tools.  For instance, a bulletin board capability will be provided so the training audience can direct questions to the trainers or to other members of the training audience should they have a question about a particular agency. They will also have the capability to send e-mail to the trainers and/or other members of the training audience. Since e-mail is generally not captured and archived, the PDD-56 DICE application might take advantage of an e-mail archiving capability.  For instance, a tool called Hypermail captures and translates e-mail between a group of people and displays it on a web site.  These archived e-mails can be reviewed by members of the group and might contain information to help clarify something, or be used to evaluate the types of communication conducted during the training event.

 

Some of the synchronous collaboration tools that will most likely be available include VTC, audio, text chat, and shared white boards.  These collaboration tools will allow the training audience to participate from their home stations and can potentially increase understanding prior to the face-to-face portion of the training.

 

One of the requirements for this particular application of DICE is that it must be very easy to use.  One of the features that some of the COTS/GOTS products include that make them easier to use is a “room-based” navigation scheme.  With this scheme, users can navigate to different areas of the collaboration space by selecting from a 2D or 3D map display.  The 2D map may have different rooms for each of the agencies.  By selecting a particular agency room, the user has access that agency’s information, a list of people to contact for more information, a current listing of other people who are in that room at that particular time, and any documents, audio, or video that might persist in that room.  When two people are in one of these “virtual” rooms at the same time, they can have a conversation via the text chat tool or VTC if they have a sound card and digital camera on their computer.  Using this room-based navigation scheme, another “virtual” room might be related to the hypothetical scenario or there could be individual rooms for each of the functional working groups writing portions of the pol-mil plan.

 

The PDD-56 DICE training application will feature a great deal of document sharing capabilities since distributed participants will be working on different components of the pol-mil plan.  Therefore, this DICE application must include a version control capability and it must be easy for users to add documents from their computers to DICE.  In some of the COTS/GOTS products, users can “drag-and-drop” a document from their desktop to the virtual environment.  In other tools, users select a Browse button and, if using a PC environment, it opens the standard Windows file navigation window.

 

For the purposes of the fall experiment, we will need to capture data about the events that occur in DICE.  Therefore there is a requirement for tools that create user logs or that can be instrumented so we can collect the data pertinent to the analysis of the training event.  Many of the COTS/GOTS products available do not provide this capability and we will most likely have to develop the programs for gathering this data.

 

2.2  Collaboration Tools Under Consideration

 

There are a number of relatively low-cost or free COTS and GOTS products that may be used in our experiment.  We are currently mapping the training requirements to the features incorporated into these products to identify the best tool or combination of tools.  This process will be completed by the end of May.  Some of these tools feature only a synchronous or asynchronous collaboration capability while others are fully integrated products that provide the complete range of support.  A subset of the available tools is described below to provide an overview of their functionality.

 

The two most popular forms of synchronous tools (video and audio teleconferencing and text chat) are probably Microsoft’s NetMeeting and CU-See-Me by WhitePine Software Inc.  Both tools have a whiteboard and application sharing capability.  NetMeeting is available for free from Microsoft’s web site (http://www.microsoft.com) and CU-See-Me has both a free and a commercial version.  The free version is available from Cornell University where the technology was first developed under research from the National Science Foundation and the commercial version is available from WhitePine Software.  Many of the tools that have limited synchronous capability but more robust archiving capability use either NetMeeting or CU-See-Me to enrich the collaboration environment.  One of the advantages to these two products is that they are both compatible with the H.323 standard and therefore a person that was using the CU-See-Me software should be able to collaborate with someone using the NetMeeting software.

 

Some of the COTS candidates for asynchronous capability include Instinctive Technology’s eRoom, Involv’s Teamspace product, TeamWave Software Ltd.’s TeamWave workplace, and PlaceWare.  All of these products are web-based and provide a persistent workspace for multiple document types and URLs, version control for documents, a bulletin board capability, a polling capability allowing users to vote on issues, and text chat in an easy-to-use interface.  Most of these groupware products operate with the use of Internet browser software along with the individual client software associated with each product.

 

Some of the collaboration products being developed and used by the government include Mitre’s Collaborative Virtual Workstation (CVW), GTE’s Intelligent WorkStation (IWS), and SPAWAR’s Odyssey tool. Both CVW and Odyssey use the room metaphor described earlier for navigation and were developed by making enhancements to LambdaMOO, a popular and freely available multi-user domain (MUD) software originally developed at Xerox PARC. 

 

CVW supports both synchronous and asynchronous collaboration.  As you navigate through the different virtual rooms, you can see the people and objects in each room.  You can make copies of these objects and/or collaborate with someone about them.  For instance, 3 people in a room can be looking at the same document and making comments about it using the text chat tool.  Different people can have control over the document and make the agreed-upon changes.  After the document has been revised, the new document will be in this room for other users to collect and copy to their desktop, or they can post questions and/or comments for others to review at a later time.  CVW has been used extensively in the intelligence community and it is also the collaboration tool incorporated into COMPASS, the Common Operational Modeling, Planning, and Simulation System.  COMPASS is middleware used to share Global Command and Control System (GCCS) data across multiple users.

 

IWS is a web browser-based tool that has similar capabilities to CVW with a slightly different development approach and interface.  It supports both synchronous and asynchronous collaboration.  It was developed using PlaceWare, a COTS collaboration tool, as a base.  IWS is the collaboration component for the Joint Intelligence Virtual Architecture (JIVA).  The focus of JIVA is the “modernization of intelligence analytical processes and methodologies.” (http://www.mews.org/jto/jivva.html)

 

 The Space and Naval Warfare Systems Command (SPAWAR) has developed a collaboration tool called Odyssey.  It is built upon the COTS tool called Facilitate.com.  Odyssey is being used in the Adaptive Courses of Action (ACOA) Advanced Concept Technology Demonstration (ACTD).  ACOA is funded by DARPA and the Advanced Information Technical Services Joint Program Office (AITS JPO) and is intended to revolutionize the joint planning process through the use of collaboration tools and other emerging information technologies.  Odyssey, like CVW and IWS, supports synchronous and asynchronous collaboration through a variety of integrated tools.

 

All three of these products are being incorporated and supported in the Defense Information Infrastructure Common Operating Environment and are being used in fairly large experiments.

 

3.  FY99 Work and Experiment

 

3.1  PDD-56 Background

 

 ”The PDD defines “complex contingency operations” as peace operations .  . . and unless otherwise directed, this PDD does not apply to domestic disaster relief or to relatively routine or small-scale operations, nor to military operations conducted in defense of U.S. citizens, territory, or property, including counter-terrorism and hostage-rescue operations and international armed conflict.” [White House Paper on PDD-56, 1997]

 

The motivation for PDD-56 stems from problems experienced by the US government while planning and conducting international crisis intervention missions, including the Grenada invasion and Somalia.  The lack of integration between the civilian agencies and the military, but also amongst the civilian agencies, led to the realization that an integrated plan was needed.  “When identifying tasks and the resources to perform them, the absence of links between the civilian and military components of these missions led to undesirable outcomes: neglect of civil police requirements and other law and order functions, resource imbalances between humanitarian relief initiatives and military operations, and lack of attention to human rights considerations.  The uncoordinated planning produced serious differences in assumptions, concepts, policy recommendations, and plans....The separate planning processes seldom identified a complete set of strategic objectives and operational needs for these interventions.”  [Walsh and Harwood, 1998]

 

NSC interest in defining PDD-56 arose from experience planning for US and UN missions in Haiti in late 1994, “...US Atlantic Command identified a host of questions related to tasks and responsibilities that were well beyond the capability of military forces committed to the operation.  These questions from the military required fully coordinated answers from other parts of the US government.” [Walsh and Harwood, 1998]  These queries eventually led to an integrated civil-military plan, developed by the NSC with support from Pentagon planners and input from other agencies.

 

Although this PDD was signed in May 1997 and the US has participated in complex contingency operations since then, PDD-56 was invoked for the first time in April 1999, for the Kosovo crisis.  Planning for this complex CCO began in October 1998, NATO bombing of Yugoslavia began in late March, and PDD-56 was invoked in mid-April.

 

When PDD-56 is invoked, the Deputies Committee of the National Security Council creates an Executive Committee (EXCOM), composed primarily of Deputy Assistant Secretaries (DAS) from all participating agencies, to oversee day-to-day management of US participation.  The Deputies Committee tasks the development of a pol-mil plan and assigns specific responsibilities to the EXCOM officials, who are then responsible for preparing their section of an integrated pol-mil plan. The plan includes milestones, measures of success, and planning for the transition to a follow-on operation.  After the plan is written, the main elements are rehearsed by the EXCOM, to identify differences among agencies over mission objectives, timing, and resources, and to underscore accountability of agency officials.  [White House Paper on PDD-56, 1997]

 

PDD-56 charges three organizations to develop and conduct an interagency training program to be held annually.  These organizations, NDU, NFATC, and PKI, have held one training exercise per year since 1997.

 

The major features of PDD-56 to be trained are the creation of the pol-mil plan and the interagency rehearsal of the pol-mil plan.  Other training objectives include learning about other agencies: their culture, capabilities and limitations, and providing an opportunity for networking for the training audience.

 

3.2  March 1999 PDD-56 Training Exercise

 

The PDD-56 training audience is composed of three levels of US government (USG) officials: desk/action officers, directors, and DAS.  The desk officers and directors participate in functional working groups to develop a pol-mil plan, then the DAS are briefed on the plan and role-play in the EXCOM rehearsal of the plan. 

 

The March 1999 training audience included about 45 people from 16 different agencies and organizations.  They were from functional offices; no regional specialists from any agency attended the exercise.

 

The scenario was set in Poland[5] and included a broad array of troubling situations. In order to ensure that many USG agencies would have a role in responding to the crisis, these situations were numerous and somewhat exaggerated.  The major scenario elements were:

 

·        A large and sudden influx of refugees from Russia, straining Poland’s humanitarian assistance capabilities;

·        An economic slowdown due to Russia’s economic collapse;

·        The potential deterioration of Soviet-era infrastructure, including the transportation system, water and power grids, and nuclear power plants;

·        The rise of organized crime, in drugs, money laundering, and smuggling, including smuggling of fissile material.

 

The scenario also posed specific requests for aid and assistance from the Polish government that would need review by the training audience, including a review of customs procedures, police training, advice on economic restructuring of the agricultural sector, and a public information campaign.

 

The training event took place over three days: March 24-26, 1999 at NFATC in Arlington, Virginia.  All participants were co-located for the exercise.  The days were a mix of briefings, panels, discussions, and small group work. On Day 1, panels and speakers discussed experiences and lessons learned from previous CCO; the scenario was introduced as a Global News Network video; then the audience broke into agency groups to discuss their agency’s coordinated response to the emerging crisis.  On Day 2, a number of agencies gave short presentations about their capabilities.  Then the audience was divided up again, this time into six functional working groups (diplomatic, economic, environmental, health, infrastructure, security) to write the part of the pol-mil plan that describes the objectives and desired end state for each functional area. The DAS arrived on Day 3, were briefed on functional areas relevant to their agency, and then participated in an EXCOM rehearsal. 

 

3.3 Data Collection and Results

 

We had 7 observers at the exercise, from ThoughtLink, Greenleaf Point LLC, and NAWC-TSD. We sat in on all of the briefings and the agency and functional groups.  We took notes and informally interviewed some of the participants.  We also distributed questionnaires at the end of each day to the training audience.  The training audience was asked about the following topics: their work background, level of computer access and expertise, interest in using technology for future PDD-56 training, level of satisfaction with the briefings and panels on each day, knowledge of PDD-56 after the training, and the degree to which they networked with other participants.  The number of participants each day varied, decreasing each day, and not all participants on a given day filled out questionnaires.  Figure 2 shows, for each day, the number of participants in the training audience and the number who filled out surveys.  There are no data for the total number of participants on Day 3, but we estimate that number at about 25.


Figure 2.  Number of Participants and Survey Respondents by Day

 

Most participants (27 of 29) were accustomed to regularly coordinating/cooperating with other agencies in their work and had been involved in several real world events requiring interagency coordination.  On average, they were not very familiar with PDD-56: an average of 2.6 on a scale of 1-5 (1 is not familiar, 5 is very familiar).  Almost all had desktop access to e-mail and the Internet and most rated themselves as intermediate or expert in their level of computer knowledge. 

 

Overall, they learned something new during the exercise.  On Day 1, 27 of 29 said they learned something new about PDD-56. One comment from a member of the training audience was:

 

“Good experience.  I know a lot about PDD-56, but at every event, I learn something new.”

 

 Twenty-two of 24 respondents said they knew more about the interagency process after Day 2.  They had a high level of satisfaction with the briefings and panels.  Table 1. lists the training objectives and the average rating among participants on how well they felt those training objectives were met on a 1-5 scale.

 

Training Objective

Average Rating

Develop understanding for intent behind PDD-56

4.2

Develop appreciation for diversity of agencies

4.1

Learn other agencies’ capabilities and limitations

3.6

Develop personal contact with other agencies

3.6

Learn structure of pol-mil plan

3.5

Learn how other agencies function

2.8

 

Table 1.  Participant Ratings on Achievement of Training Objectives

 

The following results are of particular interest to us as we plan for the fall experiment using DICE: 

 

·        Time.  For most participants, time was short and they had difficulty getting 3 days away from the office.  In fact, only 8 of  the approximately 45 training audience members attended all three days.  In later surveys given to a subset of the audience, an exercise length of 1 to 2 days was preferred. 

·        Ratio of passive to active learning. Table 2 shows that most of Day 1 and 2 were spent listening to briefings and panels. From the observer point of view, this quickly became tiresome and the level of interest flagged.  Some comments were made about this in the training audience questionnaires.

 

“I did not find agency briefings at all useful; after two hours absorption rates fall to zero.”

 

“Would have liked longer working sections with other agencies where you really learn, within the context of this exercise, the capabilities/mandates, and limitations of the agencies.”

 

 

Day 1

Day 2

Day 3

Total time (minutes)

Time spent listening (minutes)

270

225

varied from 0 to about 45

495 to 540

Time spent working (minutes)

45

100

135

280

 

Table 2.  Allocation of Time Over Training Days

 

·        Interest in using web technology.  Most (21 of 27) wanted to use technology for distance learning for future exercises. All respondents wanted to use technology to get more information about USG agencies.  Some people wanted more detailed information about the agencies that were briefed at the exercise; some people wanted information about the agencies for which presentations were not given.  Most respondents (25 of 29) wanted to access administrative information about the training online.

·        Respondents on Day 3 reported meeting 8.9 people on average.  Some would have like more and one participant suggested incorporating a social event into future exercises.  This aspect of the training is an important benefit that must be preserved when technology is introduced into the training. 

 

“Most useful benefit was to interact with other agency personnel in thinking through a USG response to a scenario.”

 

·        Seventeen of 24 would have found it useful to have access to other groups during development of the pol-mil plan.  In the March event, this would have required someone from one group going over to the room of another group and asking about their work.  In the future, this could be done via collaboration tools.

·        Thirteen of 14 would have liked to learn what the outcome was of the pol-mil plan when applied to the crisis.  In the future, we want to explore incorporating a simulation to link participant decisions to outcomes.  This concept is discussed further in the context of JTF staff training in [Loughran et al., 1999].

·        Poor performance on the quiz. We asked six questions about the PDD itself and the majority of the training audience filling out the quiz failed to answer them correctly.  This information was contained in the PDD-56 handbooks distributed at the training event.  Although the questions covered very basic material, for example, what occurs when PDD-56 is invoked, one explanation for the poor performance might be that not much time was spent covering these areas in the actual training event.

 

Observations and participant feedback from March exercise identified the following potential improvements for future PDD-56 training events, irrespective of the use of technology:

 

·        Presence of a facilitator/trainer in each working group

·        Need for training aids, e.g., a checklist of items for different functional groups when developing their group’s response to the crisis, a glossary of terms, etc., and

·        Need for increased feedback to the training audience on their performance.

 

3.4    Plan for the fall experiment

 

We will conduct a one to three day experiment in the fall to explore the ways in which DICE can be used and where it is most useful for PDD-56 training.  This section describes the fall experiment in more detail.  Note that the experiment plans will continue to evolve during the next few months.

 

3.4.1  Assumptions for Using DICE

 

Based on our observations of the March exercise and on feedback from the training audience, we are keeping the following precepts in mind while planning the fall exercise.

 

·        Face-to-face interaction is critical for the training audience and must be maintained at some level.  This interaction promotes greater understanding of other agencies’ cultures, responsibilities, and limitations, and promotes networking among participants.  This means that the fall exercise cannot be entirely distributed.

·        Shortening the total length of the exercise is a concern of many participants.  One to two days was the preferred length, from surveys we did of a small subset of the audience after the exercise.

·        Emphasizing active learning, which corresponds to an information pull approach, is thought to be a more engaging and therefore better form of training.  The ratio of passive learning (or information push) to active learning, shown earlier in Table 3.1, was high in the March exercise. 

·        Providing feedback to the training audience is important for their learning.

·        Providing a more realistic information environment to the participants is good.  In real life, people who develop a pol-mil plan have access to a variety of information about the contingency, which might include intelligence reports, State Department cables, regional expertise within an agency, and open source information, most notably on the Internet.

 

3.4.2  Use of DICE

 

This section explores how DICE can be used to enhance the curriculum/structure of the existing PDD-56 training program.   The focus of our research is on the benefits and/or pitfalls of DICE and not on how a new curriculum might enhance the PDD-56 training.

 

The point of the fall experiment is to learn how DICE can most effectively support interagency training.  The key questions are: which parts of the training are most effective in a face-to-face setting; which parts are best done in a distributed way (to reduce the total time out of the office or to speed learning); and how much of the distributed collaborative work can be done asynchronously vs. synchronously.

 

The answers to these questions are not clear-cut.  Therefore the fall experiment will be an exploration of these new techniques and will be a learning experience for us and for the planners of PDD-56 training.  We will encourage the training audience to explore different ways of using DICE to create work products (e.g., the pol-mil plan) and for several other parts of the training.  Based on their feedback and our observations, we’ll be able to describe with greater confidence how DICE can be used to support future official PDD-56 training events.

 

DICE for Pre-training.

 

The training audience will access a DICE web site to learn the necessary background information for the training event.  Information about PDD-56, the scenario, and agency capabilities will be accessible from a web site, in advance of the exercise.  This information might be in multiple forms: text, audio, or video.  Hyperlinks will be included to agency web sites and to sites with additional country information for the scenario.  Administrative information for the training: the schedule, maps to the facility, etc., will also be posted on the web site.

 

Using the collaboration tools, the training audience will be able to interact with the exercise planners, who are subject matter experts in PDD-56, prior to the training.  The planners can answer questions about the PDD, the scenario, or the training event.

 

Given the need to supply the training audience with a certain amount of background information, along with the desire to shorten the time of at least the co-located training ( if not the total length of the exercise) using DICE for pre-training seems useful.  In previous exercises, most of this background information is provided to the training audience in real time during the exercise.  It is probably faster for them to read this information.  In addition, audience surveys showed that most of the training audience wanted additional information about agencies, including more in-depth information than was presented. 

 

However, in practice, there are potentially two major obstacles to pre-training.  First, the training audience members are very busy with their jobs and have almost no time for training.  Many agencies also have a culture in which training is not valued.  Once they leave their office and come to the training event, they’re safe from most work demands. If they have to do part of their training at their office, it’s quite likely that their real work will have a higher priority and that part of the training just won’t get done.  Second, the State Department employees, who are a significant block of the training audience, do not have desktop access to the Internet.  Instead, they share a computer with unclassified Internet access with many other State Department employees. When they want to go online, they have to leave their office, go to this machine, and, if someone else is already using it, get in line.

 

Use of DICE during the Pol-Mil Plan Development.

 

It is not clear how to most effectively use DICE during the development of the pol-mil plan, so we plan to set some parameters for the training audience, in order to ensure that they will use DICE, but we won’t mandate how they’ll use it.  We hope that different functional groups will use DICE in different ways (e.g., some may collaborate intensively both synchronously and asynchronously to develop their part of the pol-mil plan, others might work with minimal collaboration, relying more on asynchronous communication).  The variety of uses, and their success, will help shape how the planners structure future training events.

 

 In addition to the functional groups using DICE to develop their own sections of the pol-mil plan, other uses of DICE might include:

 

·        Reach-back to regional expertise.  When the training audience is co-located (and there will definitely be a co-located part to their training), they might find it useful to access regional experts at their agencies and DICE can be used for this. 

·        Supporting collaboration across functional groups while they’re working on separate sections of the plan.  There are areas of overlapping interest across the different functional groups.  For example, the military/security section is interested in the diplomatic section.  DICE should make it easy for functional groups to interact with each other during plan development.

·        Providing visibility into the interim sections of the plan as they are developed.  The persistent workspace in DICE will allow different participants to view any functional group’s interim version of their part of the pol-mil plan.

·        Support exercise planning.  Currently, the planners meet about once every month to coordinate their activities for the four months or so preceding the exercise.  They divide up tasks and communicate the results of their work by handing out Xeroxes at meetings.  Planners from PKI, located in Carlisle, Pennsylvania, do not attend all planning meetings due to the travel time required.  A distributed collaborative environment will provide immediate access for all planners to the most up-to-date copy of their work products (e.g., agenda, list of speakers, list of training audience members) and will reduce the number of meetings required for coordination.

·        Operational use.  Assuming that DICE can successfully be used for PDD-56 training, it is a natural extension to consider its operational use.  One problem that immediately came to light when PDD-56 was invoked for Kosovo is that most of the pol-mil plan developers had not had the training and did not know what was expected for the pol-mil plan.  An operational web site, with information about PDD-56 as well as sample plans from previous operations, would help those new to the process.  In addition, the collaboration tools could help reduce the number of face-to-face meetings and perhaps speed up overall plan development.

 

3.4.3  Experiment Design

 

The training audience for the fall experiment will most likely be a subset of the attendees at the March exercise. The total number of people will be small - between 10 and 15.  Since this experiment will be the first extensive use of technology for this training, it was decided that using a small group, for a pilot experiment, would mitigate risk, as opposed to introducing the technology into one of the annual exercises. 

 

The scenario will be a Korean scenario used in an earlier PDD-56 exercise.  The experiment will take place at NDU and will use workstations already present on the campus.

 

Given the small size and previous experience of the training audience, as well as the unstructured nature of the participants’ use of DICE, the fall experiment will not provide statistically significant results. This experiment will look at the benefits or disadvantages of using DICE and that assessment will rely on participants’ perceptions, observations by subject matter experts, and some comparison of data between the March and fall training exercises.  However, the comparison between exercises cannot be rigorous; the training events will have different training audiences, different planners, and different scenarios.

 

Earlier in the planning process, we had considered dividing the training audience into two parts and providing one half with the technology for the training while the other half would not use technology.  However, the training audience will be too small for this and we will encourage them to experiment with different ways of using DICE. We also considered measuring the participants’ knowledge before and after the training with DICE, but because the curriculum is changing at the same time technology is introduced, there would be no way to determine whether learning is due to the curriculum or the technology.

 

3.4.4 Measures and Data Collection

 

Listed below are a series of questions that will be explored in the fall experiment. We’ll repeat the surveys given to the training audience in March, asking for their subjective assessments of the training.  We’ll also repeat the March quiz on the substance of PDD-56.  In addition, the collaboration tools will need to be instrumented so that we can obtain information about whom is collaborating with whom, which tools are used, and how often are they used.

 

·        How do the fall experiment outcomes compare to the March outcomes?  Outcomes of PDD-56 training include: the participants’ level of satisfaction with the training, whether they’re learning anything new, the amount of networking they do, and the quality of the pol-mil plan they produce.  We asked the March training audience about their level of satisfaction with the panels and speakers and whether they learned anything new about PDD-56 and the interagency process.  We gave them a quiz to measure what they had learned about PDD-56 and we also asked them how much networking they had done.  We could repeat these measures for the fall and compare them to the March results. Another way of assessing “outcomes” is to have the planners subjectively evaluate the product of the fall experiment, which is the pol-mil plan. 

·        During pol-mil plan development, do students get more information from a greater variety of sources?  For the March exercise, the only information the students had was the written scenario and the video version.  In the fall experiment, the training audience will be able to access information on the Internet.

·        Does DICE provide for reach-back to office?  What’s the effect?

·        Does DICE provide the opportunity for pre-training (reviewing materials, contacting planners?)?  If so, what’s the effect?

·        Does the training audience learn more?  For example, do they create more sections of the pol-mil plan than in March or do they write the sections more thoroughly (based on our observations)?

·        Is the training audience more engaged in the learning experience?  This would be a subjective assessment they would make.

·        How does DICE change the amount of training time?  How much time is spent in pre-training, how much is spent in face-to-face activities and how much is spent in asynchronous distributed work?  In March, we measured the amount spent in passive and active learning and we will measure that again in the fall.

·        Does DICE support an evolving, dynamic scenario?  What’s the effect?

·        Do the functional groups interact more with each other during plan development?  If so, which tools do they use for this interaction?

·        What is the reliability of DICE?  How often does it fail?  For what percentage of the exercise is it working?

·        What are the training audience’s subjective assessments of DICE’s utility to training, ease of use, reliability, and applicability in their office?

·        How much are the collaboration features used each participant?: # of VTCs, # of e-mails, ...

·        What is the audience using them for?  e.g., pol-mil plan development, research, lunch plans...

·        How much is the persistent workspace used?: # of URLs visited, # of documents read, ...

·        Does DICE make it easier for the planners to:

            1) Provide training materials to the audience?

            2) Provide more materials to the audience than they have traditionally provided?

            3) Find the training audience?

            4) Attract a larger audience?

            5) Provide feedback to the audience during training?

            6) Understand what the training audience is doing during training (transparency)?

 

4.  Summary

 

Distributed collaborative environments hold great potential for interagency training.  The hypothesis is that these technologies can shorten the total training time required, provide a richer training environment,  and make it possible for participants to work collaboratively from their home offices.  The fall DICE experiment will clarify which aspects of DICE, applied to PDD-56 training, offer the highest payoff.

 

Assuming the anticipated benefits of DICE for PDD-56 training materialize, it is a natural extension to consider using the same technologies and content operationally, to help the USG agencies respond to CCOs.  Indeed, the need to communicate information about pol-mil development both quickly and easily has become apparent with the recent invocation of PDD-56 for the Kosovo crisis.

 

Another application of DICE, for which it was originally conceived in our FY98 research, is to augment current JTF staff training.  LTG Keene and others at U.S. Atlantic Command (ACOM)[6], have identified a requirement to train partial JTF staffs.  Approximately one-third of a designated JTF commander’s core staff will rotate out in a given year.  Since CPX training occurs only every 18 to 24 months, there is a need for sustainment training in between CPXs to help train the new arrivals.  Discussions are underway with the ACOM to define the requirements for partial staff training.

 

5.  References

 

[Joint Pub 3-07] Joint Pub 3-07, Joint Doctrine for Military Operations Other Than War, 16 June 1995.

 

[Loughran et al] Julia J. Loughran, Marchelle M. Stahl, John Howard Eisenhour, Thomas J. Hammel, Edward Marks, Applying Commercial Gaming Collaboration Technologies to JTF Staff Training, 1999.  http://www.thoughtlink.com

 

[Walsh and Harwood] Mark R. Walsh and Michael J. Harwood, Complex Emergencies: Under New Management, Parameters, US Army War College Quarterly, Winter 1998.

 

[White House Paper on PDD-56] The Clinton Administration’s Policy on Managing Complex Contingency Operations: Presidential Decision Directive, May 1997.  http://www.whitehouse.gove/WH/EOP/NSC/html/documents/NSCDoc2.html



[1] PDD-56 defines “complex contingency operations” as peace operations such as the peace accord implementation operation conducted by NATO in Bosnia (1995-present) and the humanitarian intervention in northern Iraq called Operation Provide Comfort (1991); and foreign humanitarian assistance operations, such as Operation Support Hope in central Africa (1994) and Operation Sea Angel in Bangladesh (1991). [White House Paper on PDD-56]

[2]  A joint task force does not exist until it is “constituted and so designated by the Secretary of Defense, a combatant commander, a subordinate unified command commander, or an existing joint task force commander.”   There are designated JTF commanders who have their own (generally service specific) staff, but the full JTF staff does not come together until there is a need. ” [Joint Doctrine Capstone and Keystone Primer, 1997, p. 17]

[3] OOTW, also known as Military Operations Other Than War (MOOTW), refers to operations short of war.  “MOOTW focus on deterring war, resolving conflict, promoting peace, and supporting civil authorities in response to domestic crises....  MOOTW may involve elements of both combat and noncombat operations in peacetime, conflict, and war situations....  [T]he purposes of conducting MOOTW may be multiple, with the relative importance or hierarchy of such purposes changing or unclear...”  [Joint Pub 3-07, 1995, pp. I-1, I-2.  emphasis in original]   Joint Pub 3-07 identifies 17 OOTW missions, e.g., Humanitarian Assistance, Peacekeeping Operations, Enforcing Exclusion Zones, and Counterdrug Operations.

 

[4] PDD-56 coordinates the US government response to complex contingencies when multiple government agencies are involved, e.g., the Haiti peacekeeping operation and foreign humanitarian assistance.  PDD-56 was most recently invoked in April of this year for the crisis in Kosovo.

 

[5] The March exercise differed from previous exercises as it was linked to a European Command JTF exercise planned for August 1999.  The pol-mil plan created in March will be used as strategic guidance for the JTF in their August exercise.    Thus the PDD-56 scenario had to be closely aligned with and act as a precursor to the August JTF training scenario.

[6] U.S. Atlantic Command is the command responsible for overseeing Joint Task Force training.