Performance evaluation & metrics

We will assess the participants' performance in two tasks:

1. classification, that is, the presence of the csRNA-Seq signal, binary classification of positive (initiation) versus negative (no initiation) genomic positions, and
2. regression (for classification positives only), quantitative prediction of initiation efficiency.

The positions with read counts of one (singletons) will be ignored in both classification and regression performance estimates. To prevent technical overfitting, only a random half of the test data labels will be actually used during the online leaderboard stage for computing the performance metrics. The complete test data will be used in the final evaluation. Performance metrics: area-under-receiver-operating-characteristic, AUROC, area-under-precision-recall-curve, AUPRC (classification); Pearson & Spearman correlation (regression). The team rankings will be based on log-ranks to aggregate scores and identify the winner and runner-up teams, as in IBIS.