src.variants.metrics.distributional_metrics module

This module includes distributional validation metrics.

It includes several distributional metrics. See details in their own documentations. The functions are mainly based on numpy arrays for fast vectorized operations.

Contact person: Stefan Riedmaier Creation date: 03.06.2020 Python version: 3.8

src.variants.metrics.distributional_metrics.compare_means_with_ci(y_true, y_pred, axis=-1)

This function implements the validation metric of comparing means including CIs according to Oberkampf.

The theory can be found in [1, Ch. 12.4].

Literature: [1] W. L. Oberkampf and C. J. Roy, Verification and Validation in Scientific Computing, Cambridge, Cambridge University Press, 2010, ISBN: 9780511760396.

Parameters:

  • y_true (np.ndarray) – observed outputs

  • y_pred (np.ndarray) – predicted outputs

  • axis (int) – along this axis the operation is performed

Returns:

confidence interval centered around error by comparing mean values

Return type:

np.ndarray

src.variants.metrics.distributional_metrics.ks_test(y_true, y_pred)

This function calculates the Kolmogrov-Smirnov (KS) test statistic on two samples.

Parameters:

  • y_true (np.ndarray) – observed outputs

  • y_pred (np.ndarray) – predicted outputs

Returns:

KS statistic and p-value

Return type:

np.ndarray