Package qm :: Package test :: Module result :: Class Result
[hide private]
[frames] | no frames]

Class Result
source code

A 'Result' describes the outcome of a test.

A 'Result' contains two pieces of data: an outcome and a set of annotations. The outcome indicates whether the test passed or failed. More specifically, the outcome may be one of the following constants:

'Result.PASS' -- The test passed.

'Result.FAIL' -- The test failed.

'Result.ERROR' -- Something went wrong in the process of trying to execute the test. For example, if the Python code implementing the 'Run' method in the test class raised an exception, the outcome would be 'Result.ERROR'.

'Result.UNTESTED' -- QMTest did not even try to run the test. For example, if a prerequiste was not satisfied, then this outcome will be used.'

The annotations are a dictionary, mapping strings to strings.

The indices should be of the form 'class.name' where 'class' is the name of the test class that created the annotation. Any annotations created by QMTest, as opposed to the test class, will have indices of the form 'qmtest.name'.

The annotation values are HTML. When displayed in the GUI, the HTML is inserted directly into the result page; when the command-line interface is used the HTML is converted to plain text.

Currently, QMTest recognizes the following built-in annotations:

'Result.CAUSE' -- For results whose outcome is not 'FAIL', this annotation gives a brief description of why the test failed. The preferred form of this message is a phrase like "Incorrect output." or "Exception thrown." The message should begin with a capital letter and end with a period. Most results formatters will display this information prominently.

'Result.EXCEPTION' -- If an exeption was thrown during the test execution, a brief description of the exception.

'Result.TARGET' -- This annotation indicates on which target the test was executed.

'Result.TRACEBACK' -- If an exeption was thrown during the test execution, a representation of the traceback indicating where the exception was thrown.

A 'Result' object has methods that allow it to act as a dictionary from annotation names to annotation values. You can directly add an annotation to a 'Result' by writing code of the form 'result[CAUSE] = "Exception thrown."'.

A 'Result' object is also used to describe the outcome of executing either setup or cleanup phase of a 'Resource'.

Instance Methods [hide private]
  __init__(self, kind, id, outcome=PASS, annotations={})
Construct a new 'Result'.
  __getstate__(self)
Return a representation of this result for pickling.
  __setstate__(self, pickled_state)
Construct a 'Result' from its pickled form.
  GetKind(self)
Return the kind of result this is.
  GetOutcome(self)
Return the outcome associated with the test.
  SetOutcome(self, outcome, cause=None, annotations={})
Set the outcome associated with the test.
  Annotate(self, annotations)
Add 'annotations' to the current set of annotations.
  Fail(self, cause=None, annotations={})
Mark the test as failing.
  GetId(self)
Return the label for the test or resource.
  GetCause(self)
Return the cause of failure, if the test failed.
  SetCause(self, cause)
Set the cause of failure.
  Quote(self, string)
Return a version of string suitable for an annotation value.
  NoteException(self, exc_info=None, cause=None, outcome=ERROR)
Note that an exception occurred during execution.
  CheckExitStatus(self, prefix, desc, status, non_zero_exit_ok=0)
Check the exit status from a command.
  MakeDomNode(self, document)
Generate a DOM element node for this result.
  __getitem__(self, key)
  __setitem__(self, key, value)
  __delitem__(self, key)
  get(self, key, default=None)
  has_key(self, key)
  keys(self)
  items(self)

Class Variables [hide private]
RESOURCE_SETUP  
RESOURCE_CLEANUP  
TEST  
FAIL  
ERROR  
UNTESTED  
PASS  
CAUSE  
EXCEPTION  
RESOURCE  
TARGET  
TRACEBACK  
START_TIME  
END_TIME  
kinds A list of the possible kinds.
outcomes A list of the possible outcomes.

Method Details [hide private]

__init__(self, kind, id, outcome=PASS, annotations={})
(Constructor)

source code 

Construct a new 'Result'.

'kind' -- The kind of result. The value must be one of the 'Result.kinds'.

'id' -- The label for the test or resource to which this result corresponds.

'outcome' -- The outcome associated with the test. The value must be one of the 'Result.outcomes'.

'annotations' -- The annotations associated with the test.

__getstate__(self)

source code 

Return a representation of this result for pickling.

By using an explicit tuple representation of 'Result's when storing them in a pickle file, we decouple our storage format from internal implementation details (e.g., the names of private variables).

__setstate__(self, pickled_state)

source code 
Construct a 'Result' from its pickled form.

GetKind(self)

source code 

Return the kind of result this is.

returns -- The kind of entity (one of the 'kinds') to which this result corresponds.

GetOutcome(self)

source code 

Return the outcome associated with the test.

returns -- The outcome associated with the test. This value will be one of the 'Result.outcomes'.

SetOutcome(self, outcome, cause=None, annotations={})

source code 

Set the outcome associated with the test.

'outcome' -- One of the 'Result.outcomes'.

'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation.

'annotations' -- The annotations are added to the current set of annotations.

Annotate(self, annotations)

source code 
Add 'annotations' to the current set of annotations.

Fail(self, cause=None, annotations={})

source code 

Mark the test as failing.

'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation.

'annotations' -- The annotations are added to the current set of annotations.

GetId(self)

source code 

Return the label for the test or resource.

returns -- A label indicating indicating to which test or resource this result corresponds.

GetCause(self)

source code 

Return the cause of failure, if the test failed.

returns -- If the test failed, return the cause of the failure, if available.

SetCause(self, cause)

source code 

Set the cause of failure.

'cause' -- A string indicating the cause of failure. Like all annotations, 'cause' will be interested as HTML.

Quote(self, string)

source code 

Return a version of string suitable for an annotation value.

Performs appropriate quoting for a string that should be taken verbatim; this includes HTML entity escaping, and addition of <pre> tags.

'string' -- The verbatim string to be quoted.

returns -- The quoted string.

NoteException(self, exc_info=None, cause=None, outcome=ERROR)

source code 

Note that an exception occurred during execution.

'exc_info' -- A triple, in the same form as that returned from 'sys.exc_info'. If 'None', the value of 'sys.exc_info()' is used instead.

'cause' -- The value of the 'Result.CAUSE' annotation. If 'None', a default message is used.

'outcome' -- The outcome of the test, now that the exception has occurred.

A test class can call this method if an exception occurs while the test is being run.

CheckExitStatus(self, prefix, desc, status, non_zero_exit_ok=0)

source code 

Check the exit status from a command.

'prefix' -- The prefix that should be used when creating result annotations.

'desc' -- A description of the executing program.

'status' -- The exit status, as returned by 'waitpid'.

'non_zero_exit_ok' -- True if a non-zero exit code is not considered failure.

returns -- False if the test failed, true otherwise.

MakeDomNode(self, document)

source code 

Generate a DOM element node for this result.

Note that the context is not represented in the DOM node.

'document' -- The containing DOM document.

returns -- The element created.

__getitem__(self, key)
(Indexing operator)

source code 

__setitem__(self, key, value)
(Index assignment operator)

source code 

__delitem__(self, key)
(Index deletion operator)

source code 

get(self, key, default=None)

source code 

has_key(self, key)

source code 

keys(self)

source code 

items(self)

source code 

Class Variable Details [hide private]

RESOURCE_SETUP

Value:
'resource_setup'                                                       
      

RESOURCE_CLEANUP

Value:
'resource_cleanup'                                                     
      

TEST

Value:
'test'                                                                 
      

FAIL

Value:
'FAIL'                                                                 
      

ERROR

Value:
'ERROR'                                                                
      

UNTESTED

Value:
'UNTESTED'                                                             
      

PASS

Value:
'PASS'                                                                 
      

CAUSE

Value:
'qmtest.cause'                                                         
      

EXCEPTION

Value:
'qmtest.exception'                                                     
      

RESOURCE

Value:
'qmtest.resource'                                                      
      

TARGET

Value:
'qmtest.target'                                                        
      

TRACEBACK

Value:
'qmtest.traceback'                                                     
      

START_TIME

Value:
'qmtest.start_time'                                                    
      

END_TIME

Value:
'qmtest.end_time'                                                      
      

kinds

A list of the possible kinds.
Value:
['resource_setup', 'resource_cleanup', 'test']                         
      

outcomes

A list of the possible outcomes.

The order of the 'outcomes' is significant; they are ordered from most interesting to least interesting from the point of view of someone browsing results.
Value:
['ERROR', 'FAIL', 'UNTESTED', 'PASS']