Customized Evaluation

The form on the bottom of this page allows you to customize the evaluation of the results, imposing other rules than the official ones. This is offered due to our idea of providing a profile of the tool landscape, providing a more fine-grained overview on the specific strengths and weaknesses of the contestant's tools.

RankNameScoreTotal answersWrong answers% correct

The following values can be changed:

  • Property types: Restrict evaluation to only reachability or behavioral (LTL) properties, or take both into account.
  • Problem selection: Only include certain problems for the evaluation.
  • Respect confidence: If checked, then the confidence values supplied by the contestants are taken into account. If not checked, than confidence 1 is assumed for all answers.
  • Penalty factor: By default, wrong answers are penalized by subtracting twice the confidence value from the total score. Enter another number to change this factor.
  • Double score for online problems: The score for the online problems 14 through 19 counts twice in the official evaluation. Uncheck to disable.

Property types

Problem selection
Original problems:
 Problem 1  Problem 2  Problem 3  Problem 4  Problem 5  Problem 6 
 Problem 7  Problem 8  Problem 9 

Additional problems:
 Problem 12  Problem 13 

Online problems:
 Problem 14  Problem 15  Problem 16  Problem 17  Problem 18  Problem 19 
Respect confidence

Penalty factor

Double score for online problems