Support for Assessment of Assurance Cases

Built-in Assurance Case Model Constraints.

Model constraints define semantic conditions that cannot be defined in the syntactic structure of a metamodel. Since different stakeholders may have different interpretations and the underlying assumptions may be overlooked, ExplicitCase requires to document goal decompositions via strategies. Therefore, a constraint on the assurance case model enforces the existence of a strategy node whenever the user wants to connect two goals. ExplicitCase checks many more constraints to ensure the integrity of assurance cases (e.g., to prevent the creation of invalid relationships). For example, another constraint to ensure the integrity of assurance cases is that only GSN connections permitted by the GSN standard can be modeled (e.g., a context node cannot be connected to a justification node). Avoidance of circular argumentation is another built-in constraint on the semantic level.

Status Notifications

ExplicitCase offers on-the-fly checks of arbitrary complexity. We define two types of notifications: warnings and errors. Errors signal missing or erroneous information, whereas warnings indicate assurance case nodes that need to be given further consideration. The type of notifications to be get may be manually selected by the user. For example, an error is signaled when a goal is changed and the supporting solution should be reconsider (see Fig. 6). Warnings are, for instance, raised for option entities that cannot be left in the final version of the assurance case, but must be appropriately resolved (see Fig. 7).

Fig. 6 - Error reports in ExplicitCase.
Fig. 7 - Warning reports in ExplicitCase.

Feature 7: Change Impact Analysis

Throughout the operational life of any system, changing regulatory requirements, additional assurance evidence and a changing design can challenge the corresponding assurance case. In order to maintain an accurate account of the assurance of the system, all such challenges must be assessed for their impact on the original assurance argument.

Why do we need maintenance?

An assurance case consists of many inter-dependent parts: requirements, argument, evidence, design and process information. As a result, a single change to an assurance case may necessitate many other consequential changes - creating a 'ripple effect'. It is significant to recognize the importance of every challenge to an assurance case. Furthermore, the indirect impact is crucial and one of the biggest challenges. Any of these challenges imply re-certification and by extension re-generation of the assurance case of a system. The construction and maintenance of assurance case arguments is expensive and tedious, as it is mainly a manual process that requires a considerable amount of time. Therefore, offering safety engineers tool-supported re-evaluation is a big step forward.

What is the algorithm for maintenance?

The maintenance algorithm includes the handling of challenges regarding the following different argument elements.

Potential vs. actual change effect

The rules described above constitute the potential change effect and not necessarily the actual change. There is a significant difference between actual and potential change. The nodes to which the impact of the challenge in a connected GSN node propagates are called impacted nodes. The potential change includes further analysis of the possible effects on the rest of GSN nodes after one element is challenged. A safety engineer has to review all the potential challenges and decide upon them. ExplicitCase implements as a starting point, the potential change effect.

Assurance Case maintenance in ExplicitCase

The assurance case maintenance in ExplicitCase requires the participation of different entities and stakeholders (see Fig. 8). The system modeling is done by the system engineer and the GSN modeling of the assurance cases by the safety engineer. The safety engineer has also responsibilities such as hyperlinking GSN with System Models and annotating GSN assurance cases with maintainability information. ExplicitCase recognizes challenges to validity of GSN assurance cases and identifies the impact of a GSN node challenge. Finally, the safety engineer gives input to the system engineer regarding the reasons why, after a change in one system model element, other system model elements, should be reviewed.

Fig. 8 - Stakeholders in ExplicitCase.

Steps to maintenance in ExplicitCase

  1. Follow the steps in the section "Steps to specify the contained elements of a assurance case module" and build an assurance case module;
  2. Select the Solution Argument Element and right-click on it. Click 'Is Challenged';
  3. The challenged solution has changed its color to red;
  4. Right-click again on the challenged solution. Click 'Show potential change impact';
  5. The potentially impacted argument elements, by the challenged solution, have turned their color to yellow;

Support for quantitative assessment of Assurance cases

We implemented in AutoFOCUS the approached proposed by Duan et al. HASE 16, which computes the belief, disbelief and uncertainty of a GSN-argument based on the \emph{safety defeaters}. A safety defeater is anything that can reduce the confidence on the argument, such as, a software bug.

Consider the GSN-argument depicted in the figure above. It contains a main hazard which is broken down into two hazard sub-goals. Each GSN goal is annotated with the number of defeaters outruled and the total number of defeaters. In the tool, this is shown by the pair of numbers on the top left corner of GSN goals. For example, the top goal in the figure above is annotated with 15/29 denoting that 15 out of 29 safety defeaters have been outruled. Users can only enter these numbers for the leaf goals by editing their property sections, as illustrated by the figure below. Moreover, a weight is associated to all GSN nodes denoting the importance of these goal. From this data on the leaves of the GSN tree, the values of outruled and total defeaters for the remaining GSN nodes is computed by a weight sum.

Intuitively, the greater the total number of defeaters, the lower the uncertainty is. Moreover, the greater the number of outruled defeaters the greater the belief on the GSN-argument and the lower the disbelief. The exact values for belief, disbelief and uncertainty can be computed from the values of outruled and total number of defeaters. We refer to the work Duan et al. HASE 16 on how exactly these values are computed.

The belief, disbelief and uncertainty for the top most goal of GSN depicted in the figure above is shown by simply hovering the mouse over the goal as illustrated by the figure below. It is also available in the property section of the node. Moreover, the color of the numbers shown in the goal reflect the level of confidence. Red colors indicating a higher disbelief, while a green color a higher belief.