binaryClassMetrics¶
Purpose¶
Computes statistics to assess the quality of binary predictions and prints out a report.
Format¶
-
out =
binaryClassMetrics
(y_true, y_predict)¶ - Parameters
y_true – Nx1 vector of 0’s and 1’s, or strings that represent the true class labels.
y_predict – Nx1 vector of 0’s and 1’s, or strings that represent the predicted class labels.
- Returns
out (struct) –
An instance of a
binaryClassQuality
structure. For an instance named out, the members are:out.confusionMatrix
2x2 matrix, containing the computed confusion matrix.
out.accuracy
Scalar, range 0-1, the accuracy of the predicted labels.
out.precision
Scalar, (tp / (tp + fp)).
out.recall
Scalar, (tp / (tp + fn)).
out.fScore
Scalar, ((b^2 + 1) * tp) / ((b^2 + 1) * tp + b^2 * fn + fp) (b = 1) .
out.specificity
Scalar, (tp / (fp + tn)).
out.auc
Scalar, 0.5 * ((tp / (tp + fn) + (tp / (fp + tn)). Note: This is NOT the area under the roc curve, which requires requires predicted probabilities for its computation, rather than predicted class labels.
Example¶
Example 1: Basic use with binary labels¶
new;
library gml;
y_true = { 0, 0, 1, 0, 1, 1, 1, 0 };
y_pred = { 0, 0, 1, 0, 1, 0, 1, 0 };
call binaryClassMetrics(y_true, y_pred);
After the above code, the following report will be printed:
Confusion matrix
----------------
Class + 3 1
Class - 0 4
Accuracy 0.875
Precision 0.75
Recall 1
F-score 0.8571
Specificity 0.8
AUC 0.9
The interpretation of the confusion matrix is shown below:
Confusion matrix
------------------------
Class + (True Pos) (False Neg)
Class - (False Pos) (True Neg)
If you want to store the statistics computed by binaryClassMetrics()
, you can assign them to a binaryClassQuality
structure like this:
/*
** Continuing with y_true and y_pred created above
*/
// Declare bqs to be a binaryClassQuality structure
struct binaryClassQuality bqs;
// Compute metrics and assign to struct
bqs = binaryClassMetrics(y_true, y_pred);
// Print some members
print "Accuracy = " bqs.accuracy;
print "F-score = " bqs.fscore;
which will print the following output in addition to the standard report:
Accuracy = 0.87500000
F-score = 0.85714287
Example 2: String class labels¶
new;
library gml;
string true_label = { "cat", "cat", "dog", "cat", "dog", "dog", "dog", "cat" };
string pred_label = { "cat", "cat", "dog", "cat", "dog", "cat", "dog", "cat" };
call binaryClassMetrics(true_label, pred_label);
After the above code, the following report will be printed:
Confusion matrix
----------------
cat 4 0
dog 1 3
Accuracy 0.875
Precision 1
Recall 0.8
F-score 0.8889
Specificity 1
AUC 0.9