"Fossies" - the Fresh Open Source Software Archive

Member "phpbibliography-1.1.1/vendors/simpletest/docs/en/reporter_documentation.html" (22 Dec 2009, 22760 Bytes) of archive /linux/www/phpbibliography-1.1.1.zip:


As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) HTML source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 <html>
    2 <head>
    3 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    4 <title>SimpleTest for PHP test runner and display documentation</title>
    5 <link rel="stylesheet" type="text/css" href="docs.css" title="Styles">
    6 </head>
    7 <body>
    8 <div class="menu_back"><div class="menu">
    9 <a href="index.html">SimpleTest</a>
   10                 |
   11                 <a href="overview.html">Overview</a>
   12                 |
   13                 <a href="unit_test_documentation.html">Unit tester</a>
   14                 |
   15                 <a href="group_test_documentation.html">Group tests</a>
   16                 |
   17                 <a href="mock_objects_documentation.html">Mock objects</a>
   18                 |
   19                 <a href="partial_mocks_documentation.html">Partial mocks</a>
   20                 |
   21                 <span class="chosen">Reporting</span>
   22                 |
   23                 <a href="expectation_documentation.html">Expectations</a>
   24                 |
   25                 <a href="web_tester_documentation.html">Web tester</a>
   26                 |
   27                 <a href="form_testing_documentation.html">Testing forms</a>
   28                 |
   29                 <a href="authentication_documentation.html">Authentication</a>
   30                 |
   31                 <a href="browser_documentation.html">Scriptable browser</a>
   32 </div></div>
   33 <h1>Test reporter documentation</h1>
   34         This page...
   35         <ul>
   36 <li>
   37             Displaying <a href="#html">results in HTML</a>
   38         </li>
   39 <li>
   40             Displaying and <a href="#other">reporting results</a>
   41             in other formats
   42         </li>
   43 <li>
   44             Using <a href="#cli">SimpleTest from the command line</a>
   45         </li>
   46 <li>
   47             Using <a href="#xml">Using XML</a> for remote testing
   48         </li>
   49 </ul>
   50 <div class="content">
   51         
   52             <p>
   53                 SimpleTest pretty much follows the MVC pattern
   54                 (Model-View-Controller).
   55                 The reporter classes are the view and the model is your
   56                 test cases and their hiearchy.
   57                 The controller is mostly hidden from the user of
   58                 SimpleTest unless you want to change how the test cases
   59                 are actually run, in which case it is possible to
   60                 override the runner objects from within the test case.
   61                 As usual with MVC, the controller is mostly undefined
   62                 and there are other places to control the test run.
   63             </p>
   64         
   65         <p><a class="target" name="html"><h2>Reporting results in HTML</h2></a></p>
   66             <p>
   67                 The default test display is minimal in the extreme.
   68                 It reports success and failure with the conventional red and
   69                 green bars and shows a breadcrumb trail of test groups
   70                 for every failed assertion.
   71                 Here's a fail...
   72                 <div class="demo">
   73                     <h1>File test</h1>
   74                     <span class="fail">Fail</span>: createnewfile-&gt;True assertion failed.<br>
   75                     <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete.
   76                     <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div>
   77                 </div>
   78                 And here all tests passed...
   79                 <div class="demo">
   80                     <h1>File test</h1>
   81                     <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete.
   82                     <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div>
   83                 </div>
   84                 The good news is that there are several points in the display
   85                 hiearchy for subclassing.
   86             </p>
   87             <p>
   88                 For web page based displays there is the
   89                 <span class="new_code">HtmlReporter</span> class with the following
   90                 signature...
   91 <pre>
   92 class HtmlReporter extends SimpleReporter {
   93     public HtmlReporter($encoding) { ... }
   94     public makeDry(boolean $is_dry) { ... }
   95     public void paintHeader(string $test_name) { ... }
   96     public void sendNoCacheHeaders() { ... }
   97     public void paintFooter(string $test_name) { ... }
   98     public void paintGroupStart(string $test_name, integer $size) { ... }
   99     public void paintGroupEnd(string $test_name) { ... }
  100     public void paintCaseStart(string $test_name) { ... }
  101     public void paintCaseEnd(string $test_name) { ... }
  102     public void paintMethodStart(string $test_name) { ... }
  103     public void paintMethodEnd(string $test_name) { ... }
  104     public void paintFail(string $message) { ... }
  105     public void paintPass(string $message) { ... }
  106     public void paintError(string $message) { ... }
  107     public void paintException(string $message) { ... }
  108     public void paintMessage(string $message) { ... }
  109     public void paintFormattedMessage(string $message) { ... }
  110     protected string _getCss() { ... }
  111     public array getTestList() { ... }
  112     public integer getPassCount() { ... }
  113     public integer getFailCount() { ... }
  114     public integer getExceptionCount() { ... }
  115     public integer getTestCaseCount() { ... }
  116     public integer getTestCaseProgress() { ... }
  117 }
  118 </pre>
  119                 Here is what some of these methods mean. First the display methods
  120                 that you will probably want to override...
  121                 <ul class="api">
  122                     <li>
  123                         <span class="new_code">HtmlReporter(string $encoding)</span><br>
  124                         is the constructor.
  125                         Note that the unit test sets up the link to the display
  126                         rather than the other way around.
  127                         The display is a mostly passive receiver of test events.
  128                         This allows easy adaption of the display for other test
  129                         systems beside unit tests, such as monitoring servers.
  130                         The encoding is the character encoding you wish to
  131                         display the test output in.
  132                         In order to correctly render debug output when
  133                         using the web tester, this should match the encoding
  134                         of the site you are trying to test.
  135                         The available character set strings are described in
  136                         the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a>
  137                         function.
  138                     </li>
  139                     <li>
  140                         <span class="new_code">void paintHeader(string $test_name)</span><br>
  141                         is called once at the very start of the test when the first
  142                         start event arrives.
  143                         The first start event is usually delivered by the top level group
  144                         test and so this is where <span class="new_code">$test_name</span>
  145                         comes from.
  146                         It paints the page titles, CSS, body tag, etc.
  147                         It returns nothing (<span class="new_code">void</span>).
  148                     </li>
  149                     <li>
  150                         <span class="new_code">void paintFooter(string $test_name)</span><br>
  151                         Called at the very end of the test to close any tags opened
  152                         by the page header.
  153                         By default it also displays the red/green bar and the final
  154                         count of results.
  155                         Actually the end of the test happens when a test end event
  156                         comes in with the same name as the one that started it all
  157                         at the same level.
  158                         The tests nest you see.
  159                         Closing the last test finishes the display.
  160                     </li>
  161                     <li>
  162                         <span class="new_code">void paintMethodStart(string $test_name)</span><br>
  163                         is called at the start of each test method.
  164                         The name normally comes from method name.
  165                         The other test start events behave the same way except
  166                         that the group test one tells the reporter how large
  167                         it is in number of held test cases.
  168                         This is so that the reporter can display a progress bar
  169                         as the runner churns through the test cases.
  170                     </li>
  171                     <li>
  172                         <span class="new_code">void paintMethodEnd(string $test_name)</span><br>
  173                         backs out of the test started with the same name.
  174                     </li>
  175                     <li>
  176                         <span class="new_code">void paintFail(string $message)</span><br>
  177                         paints a failure.
  178                         By default it just displays the word fail, a breadcrumbs trail
  179                         showing the current test nesting and the message issued by
  180                         the assertion.
  181                     </li>
  182                     <li>
  183                         <span class="new_code">void paintPass(string $message)</span><br>
  184                         by default does nothing.
  185                     </li>
  186                     <li>
  187                         <span class="new_code">string _getCss()</span><br>
  188                         Returns the CSS styles as a string for the page header
  189                         method.
  190                         Additional styles have to be appended here if you are
  191                         not overriding the page header.
  192                         You will want to use this method in an overriden page header
  193                         if you want to include the original CSS.
  194                     </li>
  195                 </ul>
  196                 There are also some accessors to get information on the current
  197                 state of the test suite.
  198                 Use these to enrich the display...
  199                 <ul class="api">
  200                     <li>
  201                         <span class="new_code">array getTestList()</span><br>
  202                         is the first convenience method for subclasses.
  203                         Lists the current nesting of the tests as a list
  204                         of test names.
  205                         The first, most deeply nested test, is first in the
  206                         list and the current test method will be last.
  207                     </li>
  208                     <li>
  209                         <span class="new_code">integer getPassCount()</span><br>
  210                         returns the number of passes chalked up so far.
  211                         Needed for the display at the end.
  212                     </li>
  213                     <li>
  214                         <span class="new_code">integer getFailCount()</span><br>
  215                         is likewise the number of fails so far.
  216                     </li>
  217                     <li>
  218                         <span class="new_code">integer getExceptionCount()</span><br>
  219                         is likewise the number of errors so far.
  220                     </li>
  221                     <li>
  222                         <span class="new_code">integer getTestCaseCount()</span><br>
  223                         is the total number of test cases in the test run.
  224                         This includes the grouping tests themselves.
  225                     </li>
  226                     <li>
  227                         <span class="new_code">integer getTestCaseProgress()</span><br>
  228                         is the number of test cases completed so far.
  229                     </li>
  230                 </ul>
  231                 One simple modification is to get the HtmlReporter to display
  232                 the passes as well as the failures and errors...
  233 <pre>
  234 <strong>class ShowPasses extends HtmlReporter {
  235     
  236     function paintPass($message) {
  237         parent::paintPass($message);
  238         print "&amp;&lt;span class=\"pass\"&gt;Pass&lt;/span&gt;: ";
  239         $breadcrumb = $this-&gt;getTestList();
  240         array_shift($breadcrumb);
  241         print implode("-&amp;gt;", $breadcrumb);
  242         print "-&amp;gt;$message&lt;br /&gt;\n";
  243     }
  244     
  245     function _getCss() {
  246         return parent::_getCss() . ' .pass { color: green; }';
  247     }
  248 }</strong>
  249 </pre>
  250             </p>
  251             <p>
  252                 One method that was glossed over was the <span class="new_code">makeDry()</span>
  253                 method.
  254                 If you run this method, with no parameters, on the reporter
  255                 before the test suite is run no actual test methods
  256                 will be called.
  257                 You will still get the events of entering and leaving the
  258                 test methods and test cases, but no passes or failures etc,
  259                 because the test code will not actually be executed.
  260             </p>
  261             <p>
  262                 The reason for this is to allow for more sophistcated
  263                 GUI displays that allow the selection of individual test
  264                 cases.
  265                 In order to build a list of possible tests they need a
  266                 report on the test structure for drawing, say a tree view
  267                 of the test suite.
  268                 With a reporter set to dry run that just sends drawing events
  269                 this is easily accomplished.
  270             </p>
  271         
  272         <p><a class="target" name="other"><h2>Extending the reporter</h2></a></p>
  273             <p>
  274                 Rather than simply modifying the existing display, you might want to
  275                 produce a whole new HTML look, or even generate text or XML.
  276                 Rather than override every method in
  277                 <span class="new_code">HtmlReporter</span> we can take one
  278                 step up the class hiearchy to <span class="new_code">SimpleReporter</span>
  279                 in the <em>simple_test.php</em> source file.
  280             </p>
  281             <p>
  282                 A do nothing display, a blank canvas for your own creation, would
  283                 be...
  284 <pre>
  285 <strong>require_once('simpletest/simple_test.php');</strong>
  286 
  287 class MyDisplay extends SimpleReporter {<strong>
  288     </strong>
  289     function paintHeader($test_name) {
  290     }
  291     
  292     function paintFooter($test_name) {
  293     }
  294     
  295     function paintStart($test_name, $size) {<strong>
  296         parent::paintStart($test_name, $size);</strong>
  297     }
  298     
  299     function paintEnd($test_name, $size) {<strong>
  300         parent::paintEnd($test_name, $size);</strong>
  301     }
  302     
  303     function paintPass($message) {<strong>
  304         parent::paintPass($message);</strong>
  305     }
  306     
  307     function paintFail($message) {<strong>
  308         parent::paintFail($message);</strong>
  309     }
  310 }
  311 </pre>
  312                 No output would come from this class until you add it.
  313             </p>
  314         
  315         <p><a class="target" name="cli"><h2>The command line reporter</h2></a></p>
  316             <p>
  317                 SimpleTest also ships with a minimal command line reporter.
  318                 The interface mimics JUnit to some extent, but paints the
  319                 failure messages as they arrive.
  320                 To use the command line reporter simply substitute it
  321                 for the HTML version...
  322 <pre>
  323 &lt;?php
  324 require_once('simpletest/unit_tester.php');
  325 require_once('simpletest/reporter.php');
  326 
  327 $test = &amp;new TestSuite('File test');
  328 $test-&gt;addTestFile('tests/file_test.php');
  329 $test-&gt;run(<strong>new TextReporter()</strong>);
  330 ?&gt;
  331 </pre>
  332                 Then invoke the test suite from the command line...
  333 <pre class="shell">
  334 php file_test.php
  335 </pre>
  336                 You will need the command line version of PHP installed
  337                 of course.
  338                 A passing test suite looks like this...
  339 <pre class="shell">
  340 File test
  341 OK
  342 Test cases run: 1/1, Failures: 0, Exceptions: 0
  343 </pre>
  344                 A failure triggers a display like this...
  345 <pre class="shell">
  346 File test
  347 1) True assertion failed.
  348     in createnewfile
  349 FAILURES!!!
  350 Test cases run: 1/1, Failures: 1, Exceptions: 0
  351 </pre>
  352             </p>
  353             <p>
  354                 One of the main reasons for using a command line driven
  355                 test suite is of using the tester as part of some automated
  356                 process.
  357                 To function properly in shell scripts the test script should
  358                 return a non-zero exit code on failure.
  359                 If a test suite fails the value <span class="new_code">false</span>
  360                 is returned from the <span class="new_code">SimpleTest::run()</span>
  361                 method.
  362                 We can use that result to exit the script with the desired return
  363                 code...
  364 <pre>
  365 &lt;?php
  366 require_once('simpletest/unit_tester.php');
  367 require_once('simpletest/reporter.php');
  368 
  369 $test = &amp;new TestSuite('File test');
  370 $test-&gt;addTestFile('tests/file_test.php');
  371 <strong>exit ($test-&gt;run(new TextReporter()) ? 0 : 1);</strong>
  372 ?&gt;
  373 </pre>
  374                 Of course we don't really want to create two test scripts,
  375                 a command line one and a web browser one, for each test suite.
  376                 The command line reporter includes a method to sniff out the
  377                 run time environment...
  378 <pre>
  379 &lt;?php
  380 require_once('simpletest/unit_tester.php');
  381 require_once('simpletest/reporter.php');
  382 
  383 $test = &amp;new TestSuite('File test');
  384 $test-&gt;addTestFile('tests/file_test.php');
  385 <strong>if (TextReporter::inCli()) {</strong>
  386     exit ($test-&gt;run(new TextReporter()) ? 0 : 1);
  387 <strong>}</strong>
  388 $test-&gt;run(new HtmlReporter());
  389 ?&gt;
  390 </pre>
  391                 This is the form used within SimpleTest itself.
  392             </p>
  393         
  394         <p><a class="target" name="xml"><h2>Remote testing</h2></a></p>
  395             <p>
  396                 SimpleTest ships with an <span class="new_code">XmlReporter</span> class
  397                 used for internal communication.
  398                 When run the output looks like...
  399 <pre class="shell">
  400 &lt;?xml version="1.0"?&gt;
  401 &lt;run&gt;
  402   &lt;group size="4"&gt;
  403     &lt;name&gt;Remote tests&lt;/name&gt;
  404     &lt;group size="4"&gt;
  405       &lt;name&gt;Visual test with 48 passes, 48 fails and 4 exceptions&lt;/name&gt;
  406       &lt;case&gt;
  407         &lt;name&gt;testofunittestcaseoutput&lt;/name&gt;
  408         &lt;test&gt;
  409           &lt;name&gt;testofresults&lt;/name&gt;
  410           &lt;pass&gt;This assertion passed&lt;/pass&gt;
  411           &lt;fail&gt;This assertion failed&lt;/fail&gt;
  412         &lt;/test&gt;
  413         &lt;test&gt;
  414           ...
  415         &lt;/test&gt;
  416       &lt;/case&gt;
  417     &lt;/group&gt;
  418   &lt;/group&gt;
  419 &lt;/run&gt;
  420 </pre>
  421                 You can make use of this format with the parser
  422                 supplied as part of SimpleTest itself.
  423                 This is called <span class="new_code">SimpleTestXmlParser</span> and
  424                 resides in <em>xml.php</em> within the SimpleTest package...
  425 <pre>
  426 &lt;?php
  427 require_once('simpletest/xml.php');
  428     
  429 ...
  430 $parser = &amp;new SimpleTestXmlParser(new HtmlReporter());
  431 $parser-&gt;parse($test_output);
  432 ?&gt;
  433 </pre>
  434                 The <span class="new_code">$test_output</span> should be the XML format
  435                 from the XML reporter, and could come from say a command
  436                 line run of a test case.
  437                 The parser sends events to the reporter just like any
  438                 other test run.
  439                 There are some odd occasions where this is actually useful.
  440             </p>
  441             <p>
  442                 A problem with large test suites is thet they can exhaust
  443                 the default 8Mb memory limit on a PHP process.
  444                 By having the test groups output in XML and run in
  445                 separate processes, the output can be reparsed to
  446                 aggregate the results into a much smaller footprint top level
  447                 test.
  448             </p>
  449             <p>
  450                 Because the XML output can come from anywhere, this opens
  451                 up the possibility of aggregating test runs from remote
  452                 servers.
  453                 A test case already exists to do this within the SimpleTest
  454                 framework, but it is currently experimental...
  455 <pre>
  456 &lt;?php
  457 <strong>require_once('../remote.php');</strong>
  458 require_once('../reporter.php');
  459     
  460 $test_url = ...;
  461 $dry_url = ...;
  462     
  463 $test = &amp;new TestSuite('Remote tests');
  464 $test-&gt;addTestCase(<strong>new RemoteTestCase($test_url, $dry_url)</strong>);
  465 $test-&gt;run(new HtmlReporter());
  466 ?&gt;
  467 </pre>
  468                 The <span class="new_code">RemoteTestCase</span> takes the actual location
  469                 of the test runner, basically a web page in XML format.
  470                 It also takes the URL of a reporter set to do a dry run.
  471                 This is so that progress can be reported upward correctly.
  472                 The <span class="new_code">RemoteTestCase</span> can be added to test suites
  473                 just like any other group test.
  474             </p>
  475         
  476     </div>
  477         References and related information...
  478         <ul>
  479 <li>
  480             SimpleTest project page on <a href="http://sourceforge.net/projects/simpletest/">SourceForge</a>.
  481         </li>
  482 <li>
  483             SimpleTest download page on <a href="http://www.lastcraft.com/simple_test.php">LastCraft</a>.
  484         </li>
  485 <li>
  486             The <a href="http://simpletest.org/api/">developer's API for SimpleTest</a>
  487             gives full detail on the classes and assertions available.
  488         </li>
  489 </ul>
  490 <div class="menu_back"><div class="menu">
  491 <a href="index.html">SimpleTest</a>
  492                 |
  493                 <a href="overview.html">Overview</a>
  494                 |
  495                 <a href="unit_test_documentation.html">Unit tester</a>
  496                 |
  497                 <a href="group_test_documentation.html">Group tests</a>
  498                 |
  499                 <a href="mock_objects_documentation.html">Mock objects</a>
  500                 |
  501                 <a href="partial_mocks_documentation.html">Partial mocks</a>
  502                 |
  503                 <span class="chosen">Reporting</span>
  504                 |
  505                 <a href="expectation_documentation.html">Expectations</a>
  506                 |
  507                 <a href="web_tester_documentation.html">Web tester</a>
  508                 |
  509                 <a href="form_testing_documentation.html">Testing forms</a>
  510                 |
  511                 <a href="authentication_documentation.html">Authentication</a>
  512                 |
  513                 <a href="browser_documentation.html">Scriptable browser</a>
  514 </div></div>
  515 <div class="copyright">
  516             Copyright<br>Marcus Baker 2006
  517         </div>
  518 </body>
  519 </html>