src.blocks.Metric module

This module is responsible for validation metrics.

It includes one class for the validation metrics. See details in its own documentation.

Contact person: Stefan Riedmaier Creation date: 20.04.2020 Python version: 3.8

class src.blocks.Metric.Metric(config, domain)

Bases: object

This class is responsible for validation metrics.

It includes a main method called “calculate_metric” that calls the other methods. See more details in the documentation of the calculate_metric method.

calculate_area_metrics(qois_kpi_model_da, qois_kpi_system_da)

This function handles the non-deterministic area validation metrics.

Parameters:

• qois_kpi_model_da (xr.DataArray) – pbox edges of the model

• qois_kpi_system_da (xr.DataArray) – pbox edges of the system

Returns:

probabilistic metric

Return type:

xr.DataArray

calculate_metric(qois_model_da, qois_system_da)

This function calculates the selected validation metrics.

It offers several options depending on the selected case, e.g. - area metrics for ECDFs and p-boxes - time series metrics for time signals - error metrics for scalars and time signals - etc. See more details in the documentation of the respective methods.

Parameters:

• qois_model_da (xr.DataArray) – array of responses from the simulation model

• qois_system_da (xr.DataArray) – array of responses from the system

Returns:

array of validation metrics

Return type:

xr.DataArray

iavm_get_limit_functions(qois_kpi_system_da)

This function calculates the two limit functions representing the uncertainty in the measurement cdf.

The lower and upper limit function are required for the Interval Area Validation Metric from [1]. They represent the uncertainty due to the finite number of experimental repetitions.

Literature: [1] N. Wang, W. Yao, Y. Zhao, X. Chen, X. Zhang and L. Li, „A New Interval Area Metric for Model Validation With Limited Experimental Data,“ Journal of Mechanical Design, vol. 140, no. 6, 2018.

Parameters:

qois_kpi_system_da (xr.DataArray) – array of system results

Returns:

array of left and right limit functions

Return type:

tuple(np.ndarray, np.ndarray)

static upooling_backtransform(qois_model_da, qois_system_u_da)

This function back-transforms the u values and pools them from several space samples to one.

The u-pooling theory can be found in [1, Ch. 12.8.3]. Q. He extends u-pooling to model predictions in the form of p-boxes in [2]. During the initial transformation to u values, he vertically intersects a system value with a model p-box. The min u value corresponds to the right p-box edge and the max u value to the left p-box edge, since CDFs rise. During back-transformation, he horizontally intersects both u values with both p-box edges to obtain 4 values. Then he proceeds with the min and max of the 4 data values as the new interval boundaries. The min value corresponds to the intersection of the min u value with the left p-box edge and the max value to the intersection of the max u value to the right p-box edge, since CDFs rise and p-box edges do not cross. However, this is exactly the opposite compared to the initial transformation. To implement this, we could swap the ‘left’ and ‘right’ coordinates of the ‘pbox_edges’ dimension. This basically transforms scalar (vertical) system values to wide and overly conservative intervals.

Instead, we keep the initial mapping from the transformation also during the back-transformation (w/o swapping): a) min u value from right transformation p-box edge with right back-transformation p-box edge b) max u value from left transformation p-box edge with left back-transformation p-box edge This leads to tighter intervals closer to the scalar system value. Whether a) or b) leads to the min/left or max/right interval boundaries depends on the course of the p-box.

This topic could be discussed in the future.

Literature: [1] W. L. Oberkampf and C. J. Roy, Verification and Validation in Scientific Computing, Cambridge, Cambridge University Press, 2010, ISBN: 9780511760396. [2] Q. He, Model validation based on probability boxes under mixed uncertainties. Adv Mech Eng 11(5), 2019

Parameters:

• qois_model_da (xr.DataArray) – array of responses from the simulation model

• qois_system_u_da (xr.DataArray) – array of transformed u values of the system

Returns:

array of back-transformed responses from the system

Return type:

xr.DataArray

static upooling_transform(qois_model_da, qois_system_da)

This function performs the transformation to u-values for the u-pooling method.

The u-pooling theory can be found in [1, Ch. 12.8.3]. Q. He extends it to model predictions in the form of p-boxes in [2]. During the initial transformation to u values, he vertically intersects a system value with a model p-box. The min u value corresponds to the right p-box edge and the max u value to the left p-box edge, since CDFs rise. Thus, the min/max to left/right mapping is inverted along the u/probs-dimension and will be dealt with during back-transformation.

Literature: [1] W. L. Oberkampf and C. J. Roy, Verification and Validation in Scientific Computing, Cambridge, Cambridge University Press, 2010, ISBN: 9780511760396. [2] Q. He, Model validation based on probability boxes under mixed uncertainties. Adv Mech Eng 11(5), 2019

Parameters:

• qois_model_da (xr.DataArray) – array of responses from the simulation model

• qois_system_da (xr.DataArray) – array of responses from the system

Returns:

array of transformed u values of the system

Return type:

xr.DataArray