kstar / scantron-score-processing Goto Github PK

README
License: GPL v2.05
Author: Akarsh Simha <[email protected]>
This file is (c) 2011 Akarsh Simha
=============================================

CONFESSIONS:
------------
Yes, the code is dirty. Yes, the BASH scripts are not efficient. Yes,
most of the stuff here is arcane. But it works for me. I'm
nevertheless sharing it, in case someone wants to figure their way
around this.

DEPENDENCIES:
-------------
Here's a rough list of dependencies

For basic score processing:
* Qt 4 Core

For histogram generation:
* BASH

For generation response statistics plots:
* BASH
* GNUPlot with PDF via libcairo terminal
* PDFLaTeX
* LaTeX packages graphicx and geometry
* tar + gzip

COMPILING THE CODE:
-------------------
You should have Qt 4 libraries / SDK installed. Then you could cd to
the directory containing the cpp / h files and:

$ qmake

$ make

If all goes well, you should have a binary called "Process-Scores"
created in the same directory. To invoke it, type

./Process-Scores

(assuming you are in the correct directory)

It should output some usage help and exit. Read on for further
information.

PURPOSE AND SCOPE:
------------------
This project aims to make life easier by processing the output of OMR
sheets used by the College of Natural Science at the University of
Texas at Austin.

While teaching a course, one would like to have the freedom to create
a custom exam without using standard question banks. But the
exam-evaluation system is tied strongly to the exam-creation
system. Thus, one would want to use the data from the OMR sheet reader
and custom-process it to create scores in an acceptable format and
obtain statistics.

SCORE PROCESSING:
-----------------
The core of the project is a C++ program called "Process-Scores" that
takes two files -- a file containing raw data from the OMR sheet
scanners and a file containing the answer key to each version used.

The raw data file is as generated by the OMR sheet scanners used at UT
Austin.

The answer key has a line for each version of the exam, that follows
the format:

Version Number, Key to Question 1, Key to Question 2, ... <Newline>

A Key to a Question may involve answer choices separated by a '|'. In
such cases, points are awarded if any one of the answer choices was
selected (i.e. the answer choices are ORed).

For more information, run the Process-Scores program with no
arguments.

Process-Scores writes to stdout, a list of students' unprocessed
scores in the format:
EID,score
where 'score' is the number of questions correctly answered (as of
this version, the application does not support questions with
different weightages).

For the UT Austin CNS system, one needs to replace the ',' by
whitespace at the very least, and possibly scale the scores.


HISTOGRAM GENERATION:
---------------------
histogram_from_scores.sh is a BASH script that generates a histogram
from a score file of the format:
EID,score
as generated by Process-Scores.


RESPONSE STATISTICS:
--------------------
Giving a third argument to Process-Scores also causes it to produce a
file for each test version, containing statistics on the percentage of
students that chose a particular option for a particular question.

The format of the file is:

Question #, Option 0, Option 1, Option 2, ....

Option 0 is never chosen and hence the percentage is always "0" and
that is just left as it is for future convenience. However, this
should be fixed later.

The file may be passed through response-stats-to-plots.sh, which plots
the statistics for each question and collates them into PDFs.

To invoke the response-stats-to-plots.sh, one passes the response
stats file generated by Process-Scores as an argument.

TO GENERATE SCORES IN AN UPLOADABLE FORMAT:
-------------------------------------------

1. Compile the program:
$ qmake
$ make

2. Create the key:
Open your favorite editor and enter the key into a file, i.e. for each
version of the test, add a line:

version_number,answer_to_question_1,answer_to_question_2,....

3. Produce the score list:
$ ./Process-Scores PATH-TO-SCANNER-OUTPUT-FILE PATH-TO-KEY-FILE RESPONSE-STATS-PREFIX > SCORE-FILE

4. Generate the histogram if you want to:
$ ./histogram_from_scores SCORE-FILE > HISTOGRAM-FILE
$ gnuplot
gnuplot> set xtics 1
gnuplot> plot "HISTOGRAM-FILE" using 1:2 with boxes
etc.

5. Generate graphs from response statistics for each version:
$ ./response-stats-to-plots.sh RESPONSE-STATS-FILE-FOR-A-VERSION

6. Upload scores
$ cat SCORE-FILE | sed 's/,/ /' > SCORE-FILE-WITH-WHITESPACE-SEPARATOR