"Fossies" - the Fresh Open Source Software Archive

Member "chandler-1.0.3/external/PyLucene/PyLucene-2.3.1-3-418/lucene-java-2.3.1/contrib/benchmark/src/java/org/apache/lucene/benchmark/quality/package.html" (27 Jul 2007, 2357 Bytes) of package /windows/misc/Chandler_src_1.0.3.tar.gz:

Caution: In this restricted "Fossies" environment the current HTML page may not be correctly presentated and may have some non-functional links. Alternatively you can here view or download the uninterpreted raw source code. A member file download can also be achieved by clicking within a package contents listing on the according byte size field.

Search Quality Benchmarking.

This package allows to benchmark search quality of a Lucene application.

In order to use this package you should provide:

For benchmarking TREC collections with TREC QRels, take a look at the trec package.

Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:

    File topicsFile = new File("topics-701-850.txt");
    File qrelsFile = new File("qrels-701-850.txt");
    Searcher searcher = new IndexSearcher("index");

    int maxResults = 1000;
    String docNameField = "docname"; 
    PrintWriter logger = new PrintWriter(System.out,true); 

    // use trec utilities to read trec topics into quality queries
    TrecTopicsReader qReader = new TrecTopicsReader();
    QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));
    // prepare judge, with trec utilities that read from a QRels file
    Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));
    // validate topics & judgments match each other
    judge.validateData(qqs, logger);
    // set the parsing of quality queries into Lucene queries.
    QualityQueryParser qqParser = new SimpleQQParser("title", "body");
    // run the benchmark
    QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
    SubmissionReport submitLog = null;
    QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);
    // print an avarage sum of the results
    QualityStats avg = QualityStats.average(stats);
    avg.log("SUMMARY",2,logger, "  ");

Some immediate ways to modify this program to your needs are: