Discussion

Home     Discussion Board      AI Singapore Trusted Media Challenge      ROC AUC calculation

rohan1561

ROC AUC calculation

posted in   AI Singapore Trusted Media Challenge

2021-10-10 23:28

23  comments

None

aisg_ben

2021-10-13 04:36

Reply

0
<p>May I clarify what is meant by “number of videos that were processed from the test set is lower than the number in the full test set”?</p>

ChallengeSinga

2021-10-13 17:10

Reply

0
<p>I think he meant by: suppose if the test set has 1000 videos and if the submission can process only 40 test videos</p> <p>How the AUROC values will be computed in such a case?</p>

  • None

    aisg_ben reply ChallengeSinga

    2021-10-14 00:11

    Reply

    0
    <p>May I clarify that by “the submission can process only 40 test videos”, you meant that the submission.csv output file would only have 40 probabilities?</p>

  • ChallengeSinga reply aisg_ben

    2021-10-18 22:08

    Reply

    0
    <p>Yes </p> <p>or it would contain all 1000 probabilities (filled with 0.5 or random for the unprocessed videos)?</p>

  • None

    aisg_ben reply ChallengeSinga

    2021-10-19 00:35

    Reply

    0
    <p>If some videos do not have a corresponding probability, then there would be a file length error. In other words, each video must have a probability of it being fake. The AUC score is then calculated over the entire test set.</p>