Code Monkey home page Code Monkey logo

testlib4j's Introduction

testlib4j

Introduction

Testlib is the name for a library which is used to create checkers. Checkers are programs which validate the output of solutions for problems in programming competitions.

Originally, Testlib was written in Pascal. A modern implementation was done in C++ by Mike Mirzayanov, which is available on GitHub.

In this repository, an implementation in Java is made to do the same thing. This implementation was used in recent NEERC semifinals.

How to build

We use Ant due to its simplicity.

  • Run ant compile-src, or simply ant, and testlib4j.jar will be created in the deploy directory.
  • Run and compile-examples, and all the example checkers from the examples directory will be compiled and put into the deploy/examples directory.
  • Run ant clean to remove all JARs and intermediate build files.

How to implement your checker

Every checker should implement the ru.ifmo.testlib.Checker interface. There is a single method to implement, namely, Outcome test(InStream inf, InStream ouf, InStream ans), where the arguments of type ru.ifmo.testlib.InStream correspond to the input file, the output file and the answer file, correspondingly.

In this method, you do everything you normally do within a checker. For example, you can read the answer from the contestant's output file (the ouf parameter), the correct answer from the answer file (the ans parameter) and compare them for equality. You can also read the input file (the inf parameter) and validate the contestant's output against this input.

You need to return the Outcome, which is your verdict about correctness of the contestant's output. There are various types of outcomes, reflected by the Outcome.Type enum:

  • Outcome.Type.OK: everything is OK, and the contestant's output is correct.
  • Outcome.Type.PE: "Presentation Error", the output cannot be parsed.
  • Outcome.Type.WA: "Wrong Answer", the output is understandable but wrong.
  • Outcome.Type.FAIL: something is really bad; either the files are totally wrong, or things are so bad that someone needs to rewrite the checker.

When needed, you can create and return instances of Outcome directly, for instance: return new Outcome(OK, "This is a brilliant answer"). Outcomes are also Throwables, so you can also throw new Outcome(WA, "Bad guys receive no gifts today"). There is a convenient shorthand Outcome.quit(Outcome.Type, String, Object...) for immediately quitting from anywhere with the given outcome, the format string and its arguments, for instance:

throw/return Outcome.quit(WA, "Expected %d found %d", 42, 26);

If your checker consists of a validation procedure which needs to be run both for the contestant's answer and for the jury's answer, you can use the InStream.quit(Outcome.Type, String, Object...) method to achieve the same aim. The framework will ensure, however, that when you are processing the jury's answer, every error will manifest itself as a FAIL, which is what you need.

How to run your checker

Once you have compiled your checker (we assume the class name is MyChecker, it is located in the default package, and the class files are packed in a file called Check.jar), you can run it as follows:

java -cp Check.jar:testlib4j.jar ru.ifmo.testlib.CheckerFramework MyChecker <input> <output> <answer>

where testlib4j.jar is the JAR file containing classes from this project. On Windows, you need to separate JAR files with ;, for instance: java -cp Check.jar;testlib4j.jar.

For your convenience, you can put the checker's class name in the manifest of one of the JAR files, so you can drop it in the command line. The corresponding attribute is called Checker-Class. The example manifest file is below:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.1
Created-By: 1.8.0_172-b11 (Oracle Corporation)
Checker-Class: MyChecker

Examples

This is maybe the simplest possible checker. It checks whether the output file produced by the contest participant contains the single integer which is the same as the right answer.

import ru.ifmo.testlib.*;
import static ru.ifmo.testlib.Outcome.Type.*;

public class CompareInteger implements Checker {
    public Outcome test(InStream inf, InStream ouf, InStream ans) {
        long answer = ans.nextLong();
        long output = ouf.nextLong();
        if (answer != output) {
            return new Outcome(WA, "The answer is " + answer + ", but you output " + output);
        } else {
            return new Outcome(OK, "OK: " + answer);
        }
    }
}

testlib4j's People

Contributors

kunyavskiy avatar mbuzdalov avatar niyaznigmatullin avatar shemplo avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.