mcwl logo

Australian Defence Force

Background for the Basic and Advanced Analytic Wargaming Courses from the Naval Postgraduate School in Canberra, 26 February to 7 March 2018

The Data Collection and Management Plan

Example of a Data Collection and ManagementA Data Collection and Management Plan (DCMP) is developed with issues decomposed into sub-issues, and further into questions. The decomposition may continue through essential elements of analysis (EEA) and the measures of merit (MOM) that might be used to assess them. If appropriate, a DCMP can be decomposed to a very fine granularity, usually called a data element (DE).

For seminar war games, where data collection may be limited largely to a record of the discourse between players, the MOMs would not often be quantitative measures. Data collection is more likely an assessment of qualitative views or of military judgement. Even though most seminar war games would require a far simpler DCMP than seen in many other forms of game, a DCMP is still valuable in designing a scenario that will hit on the issues that emerge from discussions with the sponsoring organization. During the conduct of the seminar war game, the DCMP provides a useful checklist to ensure that no relevant issues have been overlooked; using a DCMP should ensure that the study team will avoid reaching the end of a game only to discover that data on some important issue was never collected.

Spreadsheet formatting for a DCMPGenerally a DCMP is developed in a spreadsheet, as a convienient and familiar tool. Apart from the contents already indicated, columns can be added for many more elements.

Across a row in a DCMP spreadsheet, one should find an indication of the issue to be addressed (derived from the sponsor's ojective). If an issue is particularly complicated, it may be decomposed into a number of sub-issues. Then further to the right of the issues/sub-issues will be the essential element of analysis (EEA). And to the right of this will be a measure of merit. Note: there may be more than one EEA for each sub-issue, and more than one MOM for an EEA. For a MOM, there may be several data elements (DE), if the decomposition is to that level of details.

For example, an EEA may deal with casualties, which could result in a collection of several MOMs or DEs. It may be necessary to collect BLUE and RED casualties, and these casualties might be sub-divided by personnel and vehicles. And there could be further categories for BLUE-on-BLUE casualties or collateral casualties to the civilians population.

While the obvious purpose of a DCMP is to develop an inventory of expected data (at the levels of MOM and DE), by extending the DCMP to the right, the DCMP encourages the development of elements of the scenario -- clearly no data can be collected for an issue/sub-issue if there is not a point in the scenario where the players are confronted by it. To extend the previous example, no data will be collected on casualties unless something in the scenario creates the potential for casualties. And further, if collateral casualities to a civilian population is a concern, there has to be potential within the scenario for such casualties to occur.

Methods, Models, and Tools.

A column within the DCMP spreadsheet can be set aside for the Methods, Models, and Tools (MMT) that pertain to a particular issue, sub-issues, or essential element of analysis. If the game is a short small seminar game the only MMT used troughout could be "Facilitator's Judgement". But more elaborate MMT can also be used. For example, if a sub-issue represents some aspect of ammunition consumption, the associated MMT might be a combat model that can represent engagements and track rounds fired. Once the column associated with MMT has been completed, it can be used to manage the inventory of MMT. For example, if a combat model is a requirement within the DCMP, then the analysts and designers will have to allocate resources to ensure it will work as intended within the game struture.

Note that even if aggregated MMT amounts to a repeated use of "Facilitator's Judgement", the facilitator and the team must ensure that the facilitator has appropriate competency to support the full range of issues that may emerge during play.

Scheduling Events

The DCMP provides a mechanism to schedule certain events. There can be "Location" and "Time" indicators in the right hand columns to indicate where, during a scenario or in a MSEL (master scenario event list), an opportunity would arise to observe the issue in question. If the issue were important and there had been no opportunity to collection reactions to it, then the MSEL would have to be adjusted accordingly. Hence, the scenario design must accommodate the analysis plan.

For smaller games, the use of the DCMP for scheduling seem be of little value. However it can provide a checklist to ensure that issues that are expected at a particular point, arise at that point. If they do not, the facilitator or control cell will have to face a decision to introduce the issue at some other time, or to accept that no evidence on that issue will be captured.

For larger games, say where there are many players and they are are dispersed in a training area, or where the play extends over several days, the DCMP scheduling mechanism assists in the economical management of data collectors and observers: they will be deployed at the right time and right location.

The Extended Data Collection and Management Plan

As plans for a war game develop, the wargaming team will encounter situations where they need information. The source is frequently the sponsor or the sponsor's staff; so the responsiveness in filling these needs lies within the control of the sponsor or other leaders in the sponsor's organization. The wargaming team should use a consistent mechanism to raise and track requests for information, which are often directed to the sponsor's staff for a response, but alternatively these may have to go to outside agencies that have relevant expertise or specialized knowledge.

As indicated, some RFIs may be directed to agencies other than the sponsor's staff. For information on the opposing force, for example, the wargaming team may go directly to some intelligence organization. For cultural factors in the area of operations, responses to an RFI may be sought from embassy staff in the corresponding country or from the academic community. For weapon and sensor characteristics there may be engineering agencies that have test data on appropriate systems.

In larger wargaming projects the tracking of RFIs – where they were routed, when a response can be expected, and confirmation that a reply has fully satisfied the corresponding RFI – can be significant administrative duty. Placing a "marker" in the DCMP for each RFI ensures these requests are readily apparent. By inserting them into the DCMP opposite specific issues/sub-issues clarifies the linkage: if a specified RFI is not satisfied, then the issue against which it has been placed is unlikely to have results that will satisfy the sponsor.

As discussions with the sponsor and the sponsor's staff are taking place, the war game team is likely to encounter constraints and limitations, and be forced to make assumptions to fill in gaps. Sometimes, pending a response to an RFI, assumptions will have to be made temporarily to move the project along without having to suspend activities because an RFI response is not forthcoming.

An extended Data Collection and Management Plan is a useful mechanism to track RFIs and CLAs. Placing markers within a DCMP, typically to the right of the components identified above (EEAs, etc.), links each RFI or CLA entry to issues and sub-issues of the sponsor. Thus they provide traceability back to interests of the sponsor. It is then possible to connect a failure to get a response on an RFI or to the restrictions from a constraint or limitation back to the sponsor's expectations. This in turn can have an impact on game design and development and on the results that can be provided at the conclusion of the project. Since the RFI and CLA are specifically linked to the interests of the sponsor, these links might be used as leverage to overcome impediments to proper preparation for a game or to reduce expecations of how well the game results will cover issues where an impediment has not been overcome.

Some contraints set by the sponsor may be on the "grand scale" and affect all aspects of a wargaming project: e.g., the overall budget, the due date for delivering the final report. Other constraints may be more localized to specific issues. The sponsor might specify that a particular combat model be used to adjudicate firefights, or that certain individuals be present as players when a given topic is discussed, or that certain technologies be used by players so their effectiveness can be determined. By embedding the appropriate links in the DCMP across from the affected issues/sub-issues the localized effect of these constraints will be clear. Similarly for RFIs that have not been answered in a timely fashion (usually resulting in an assumption across the same row in the DCMP.

When including RFIs and CLAs in a DCMP only a link or some keywords should be inserted into the spreadsheet to keep its size under control. Separate lists or notebooks or databases can be kept of the necessary details.

Various Additional Extensions to the DCMP

While not necessary within a DCMP, a developing DCMP may be elaborated with various additions. For example columns can be added for the following:

• titles of doctrine or training manuals that should be consulted so observers have more contextual information on some potential event

• external agencies that might need to be consulted or that might needed to approve some activity, e.g., range control or air traffic control.

Scenario Crosswalk

MXEL Crosswalk format The Analysis Handbook also provides a mechanism to crosswalk from the issues that are the focus of a war game to scenario design -- from the Essential Areas of Analysis within the Data Collection and Management Plan (DCMP) to the Master Scenario Events List (MSEL). This is the means to do a double check that important issues for data collection will be afforded opportunities within the scenario to see how participants deal with them. Note that it is probably risky to have an important issue without some corresponding activity in the MSEL, although it could come up merely by chance. However there may be entries in the MSEL that have no corresponding issues from the DCMP; specifically, there may be items in the MSEL that provide some extra realism for the players, or that can provide some other secondary function, e.g., events to develop team cohesion or to practice a routine that may be necessary later in the scenario.

The Analysis Plan

Below is a example of the table of contents for an analysis plan showing the general format for an analysis plan. Where appropriate this should be adapted to local requirements.


Provide a summary of how the analysis will fit within the overall purpose of the study, which is derived in turn from objectives as established by the sponsor.


Sponsor's Objective

Once you have drafted a statement of the sponsor's objective, test it against the "Call Mom" criterion. If you gave your statement to "Mom" would she understand what the objective is, and its significance. Note: this objective should be consise; if it runs to more than one sentence, it may be too long.

Issue Decomposition

There are many frameworks that can be used to decompose the issues for analysis. They bring with them a natural structure, as they were designed to have one. Some familiar lists may provide templates for the current analysis (if they suit):

Extracts from task lists can expedite the decomposition process as the tasks have already been decomposed to several levels. However, some critical thinking should be applied before using an existing product like this: "Is it adaptable to our problem, or will we be re-defining (or mis- defining) our problem in atempting to adapt it to a list that is really not very relevant?"

Several other structures for hierarchical decomposition are currently popular. These may serve to initiate a decomposition of sponsor's objectives and issues:

Analytics Approach