adf logo

Australian Defence Force

Background for the Basic and Advanced Analytic Wargaming Courses from the Naval Postgraduate School in Canberra, 17 February to 6 March 2020

Introduction

The Joint Experimentation Directorate has developed procedures for joint experimentation for the Australian Defence Force. Wargaming is a tool that will be freqently applied in conducting joint experimentation. For several years wargaming practices has been taught within the Australian Department of Defence by a team from the US Naval Postgraduate School. Many of the practices and procdures from NPS have been adapted for application within joint experimentation.

While not identical to practices and procedures taught by NPS, the Australian application of NPS methods remain highly consistent with those taught by NPS. The main differences are in some of the terms that are used and in contextual aspects of working within the Australian Department of Defence and the Australian Defence Force.

Experimentation and Wargaming

The Joint Experimentation Directorate (JED) proposes four main methods for joint experimentation: seminar, seminar wargame, planning wargame, and constructed simulation. NPS practices and procedures cover the range from seminar wargames to planning wargames and many of the the associated prociples can be adapted to the remaining methods: seminars and simulation-supported experimentation.

Terminology

The principles of wargaming are well aligned between the Naval Postgraduate School (NPS) and the Joint Experimentation Directorate (JED). However there are differences in terminology. This page is intended to clarify the differences in terminology.

[NPS] Data Collection and Management Plan. The JED equivalents are called “Data Collection Management Plan” or “Data Capture Management Plan”. The DCMP as used by JED should identify the data that must be captured and how it will be captured, stored, retrieved, and manipulated during analysis. For NPS, the DCMP may be more extensive with, for example, space for constraints, limitation, and assumptions. However the DCMP template remains flexible and should be re-organized, when appropriate, for a new problem at hand.

[JED] Gaps and Opportunities / Questions (G&O/Q). NPS does not use these terms. The NPS equivalent to G&O/Q is the “sponsor's objective” and “issues” and the consequent “essential questions” (EQ). NPS methods are intended for application beyond joint experimentation, e.g., assessment of an existing operations plan or campaign plan, where the framework may not be in terms of the gaps or opportunities that are critical in analysis for future concepts or capabilities. The use of “questions” is relevant to both joint experimentation and to NPS wargaming methods. By framing wargaming objectives as questions, it can be clearer that results are adequate: the questions now have answers... or not.

[JED] Warning Order (WNGO). A warning order is a preliminary notice of an order or action which is to follow. When sufficient detail is available, it typically follows the format of the Operations Order, e.g., the five-paragraph format. When there is insufficient detail, only some elements of the five-paragraph format may be included. The purpose is to provide information to all who are likely to be involved in an upcoming operation with the necessary information to prepare for those operations. A warning order is usually used to initiate planning activity for all who may be involved in some future operation; it does not authorize anyone to take action or to commit resources (beyond the resources needed to start planning). For the purposes of joint experimentation the warning order should include as much detail as possible but as a minimum: background, timelines for activities, and relevant personnel.

Guide for Understanding and Implementing Defense Experimentation (GUIDEx)

GUIDEx was developed by The Technical Cooperation Program (TTCP). The defence scientific organizations of five nations -- the United States, Britain, Canada, Australia, and New Zealand -- provided a guidebook for conducting experimentation with military technology and concepts. The main guide book is long -- nearly 400 pages -- but is augmented, for convenience, with shorter documents: the pocketbook and the pamphlet.

Gaps and Opportunities or Questions (G&O/Q)

In the first two phases of joint experimentation there is considerable focus on stating and elaborating on gaps, opportunities, or questions associated with the upcoming experimentation activity. In "Problem and Experiment Scoping", it is expected that the sponsor (or sponsor staff) will provide some clarification and guidance on the scope of gaps, opportunities, or questions that will be addressed by the experimentation. The clarification and guidance may not be fully adequate to move on to experimentation, so a second phase will be used to refine some aspects.

Thus in the subsequent phase, "Problem Definition and Research", there is an opportunity to elaborate on the definitions of the gaps, opportunities, or questions that emerged from the preceeding phase. This second phase is used to validate the wording to ensure further analysis will be directed to provide the appropriate output.

Phases

The Naval Postgraduate School uses five phases for a wargaming project: Initiate, Design, Develop, Conduct, and Analyze and Report.

Problem and Experiment Scoping

This activity is driven by the Gaps and Opportunities or Questions (G&O/Q) being investigated. The sponsor provides clarification and guidance on the G&O/Q scope, problem owner and support commitment. Joint Experimentation Directorate applies selection criteria to ensure the G&O/Q is suitable for joint experimentation.

Apart from the more general guidance on G&O/Q from this phase, the endstate should include terms of reference for the experimentation activity. This should, as a minimum, include:

  • Introduction to the experiment
  • Aim
  • Objective (including decision to be informed)
  • Approach
  • Assumptions and Constraints
  • Authorities and Tasking
  • Roles and Responsibilities
  • Resources
  • Deliverables (including the form of the output i.e. what the sponsor needs)
  • Timeline

Problem Definition and Research

This phase is to elaborate on the G&O/Q definition. It validates the wording to ensure further analysis will be directed to provide the appropriate output.

Experiment Design

This phase focuses on deciding on methodology; strategic governance, e.g., use of Australian Capability Context Scenario(s) (ACCS); and data sets. This phase should also include the development of the project-management plan and associated subplans, e.g., risk-management plan.

Experiment Preparation

This phase consists of implementing the project-management plan, including issuing an initial Warning Order and General Instructions; data collection; modelling; administration, e.g., booking facilities; and stakeholder management.

Experiment Execution

Conduct of the experiment includes monitoring the methodology and output, to ensure the desired result is achieved.

Analysis and Reporting

Applying the prescribed analysis techniques to the data that has been captured and producing the appropriate report. Analysis will typically be a Defence Science and Technology Group activity, with reports being collaboratively developed by the Defence Science and Technology Group and the Joint Experimentation Directorate.

Roles and Responsibilities

In the Australian standard operating procedures roles and responsibilities of various components of the defence organization are explained in the context of wargaming (and the larger topic of joint experimentation).

The roles of the Director of Joint Experimentation, the service-specific organizations, and the staff of the Defence Science and Technology Group are laid out.

GUIDEx

The Technical Cooperation Programme (TTCP) prepared a guide book and associated documents on experimentation when used in support of defence activities. These consist of the guide book itself, plus a much shorter pocketbook and a pamphlet with summary tables and lists.

The 14 Principles for designing valid experiments provide a solid foundation for using the scientific method to establish cause-and-effect relationships for hypothesized military capabilities.

Designing Valid Experiments

1. Defense experiments are uniquely suited to investigate the cause-and-effect relationships underlying capability development.

2. Designing effective experiments requires an understanding of the logic of experimentation.

3. Defense experiments should be designed to meet the four validity requirements.

Integrated Analysis and Experimentation Campaigns

4. Defense experiments should be integrated into a coherent campaign of activities to maximize their utility.

5. An iterative process of problem formulation, analysis and experimentation is critical to accumulate knowledge and validity within a campaign.

6. Campaigns should be designed to integrate all three scientific methods of knowledge generation (studies, observations and experiments).

7. Multiple methods are necessary within a campaign in order to accumulate validity across the four requirements.

Considerations for Successful Experimentation

8. Human variability in defense experimentation requires additional experiment design considerations.

9. Defense experiments conducted during collective training and operational test and evaluation require additional experiment design considerations.

10. Appropriate exploitation of modeling and simulation is critical to successful experimentation.

11. An effective experimentation control regime is essential to successful experimentation.

12. A successful experiment depends upon a comprehensive data analysis and collection plan.

13. Defense experiment design must consider relevant ethical, environmental, political, multinational, and security issues.

14. Frequent communication with stakeholders is critical to successful experimentation.

Source: GUIDEx Phamplet