machine-learning Evaluation Metrics ROC curves


Example

A Receiver Operating Characteristic (ROC) curve plots the TP-rate vs. the FP-rate as a threshold on the confidence of an instance being positive is varied ROC example

Algorithm for creating an ROC curve

  1. sort test-set predictions according to confidence that each instance is positive

  2. step through sorted list from high to low confidence

    i. locate a threshold between instances with opposite classes (keeping instances with the same confidence value on the same side of threshold)

    ii. compute TPR, FPR for instances above threshold

    iii. output (FPR, TPR) coordinate