The DICE Experiment: Creating and Evaluating a Web-based Collaboration Environment for Interagency Training

Appendix D - Questionnaires

DICE Group Tests

• Pre-test
• Test 1
• Test 2
• Test 3 (Word)

No DICE Group Tests

• Pre-test (Word)
• Test 1 (Word)
• Test 2 (Word)
• Test 3 (Word)

IMPORTANT NOTES - April 15, 2000

1. The training audience was divided into two groups: DICE and No DICE, and the groups took these tests at different times. The DICE group took the pre-test, test 1, and test 2 online. The pre-test was taken any time before October 13, 1999. Test 1 was taken between October 14 and 20. Test 2 was taken between October 22 and 27.

   There were a few occasions where, because of time problems or technical problems, some members of the DICE group were unable to take the online test in a given week and were given a paper version of that test at the next face-to-face meeting.

   The DICE group received test 3 on paper, during their final face-to-face meeting on October 27.

   The No DICE group took all the tests on paper, during their face-to-face meetings as follows:
   pre-test: at the start of the 10/13 meeting
   test 1: at the end of the 10/13 meeting
   test 2: at the end of the 10/20 meeting
   test 3: at the end of the 10/27 meeting

2. In general, both groups had the same questions on each test, but the DICE group had additional questions that focused on their use of the online collaborative environment.
3. There were occasional problems with the online tests and test results for some members of the DICE group were not captured.
4. In test 2, on the scenario quiz, the correct answer to question E. ( The UN Secretary-General's report on Korea dated 10/20/00 concludes that? ) was “All of the above”. However, in the DICE group's online test, this was incorrectly typed as “None of the above”. This invalidates the DICE group's results for that one question.