src.evaluation.vvuq_evaluation module

This module is responsible for the evaluation of the VV&UQ methodology itself.

It includes several functions that analyze different aspects. See details in their own documentations.

Contact person: Stefan Riedmaier Creation date: 20.08.2020 Python version: 3.8

src.evaluation.vvuq_evaluation.boolean_classifier(idx_dict, decision_system_estimated_da, decision_system_da, decision_model_da, qois_kpi_system_estimated_da, qois_kpi_system_da, save_path='', percentage_flag=False)

This function compares binary decision making results to their corresponding ground truth values.

It compares the decision from the nominal model with the GT and the decision from the VVUQ methodology with the GT. The binary classifier creates a confusion matrix for the comparisons.

It checks whether the GT lies within the bounds from uncertainty expansion.

It offers saving the results via pickle and in form of tables via further file formats.

Parameters:

  • idx_dict (dict) – dictionary to index the data arrays

  • decision_system_estimated_da (xr.DataArray) – array of boolean decisions from the system estimation

  • decision_system_da (xr.DataArray) – array of boolean decisions from the system

  • decision_model_da (xr.DataArray) – array of boolean decisions from the nominal model

  • qois_kpi_system_estimated_da (xr.DataArray) – array of responses from the system estimation

  • qois_kpi_system_da (xr.DataArray) – array of resonses from the system

  • save_path (str) – (optional) path where the plots shall be saved (otherwise they will be shown)

  • percentage_flag (bool) – (optional) select whether the classifier results shall be given as percentage ratios

src.evaluation.vvuq_evaluation.evaluation_area_metric(config, idx_dict, qois_kpi_model_da, save_path='')

This function calculates and visualizes the area between the regulation and the model p-box to evaluate the buffer.

Parameters:

  • config (dict) – user configuration

  • idx_dict (dict) – dictionary to index the data arrays

  • qois_kpi_model_da (xr.DataArray) – array with model kpis in the application domain

  • save_path (str) – path where the plots shall be saved (otherwise they will be shown)

Returns:

average area between regulation and pbox across all space samples