A 'Result' describes the outcome of a test.
A 'Result' contains two pieces of data: an outcome and a set of
annotations. The outcome indicates whether the test passed or failed.
More specifically, the outcome may be one of the following constants:
'Result.PASS' -- The test passed.
'Result.FAIL' -- The test failed.
'Result.ERROR' -- Something went wrong in the process of trying to
execute the test. For example, if the Python code implementing the 'Run'
method in the test class raised an exception, the outcome would be
'Result.ERROR'.
'Result.UNTESTED' -- QMTest did not even try to run the test. For
example, if a prerequiste was not satisfied, then this outcome will be
used.'
The annotations are a dictionary, mapping strings to strings.
The indices should be of the form 'class.name' where 'class' is the
name of the test class that created the annotation. Any annotations
created by QMTest, as opposed to the test class, will have indices of the
form 'qmtest.name'.
The annotation values are HTML. When displayed in the GUI, the HTML
is inserted directly into the result page; when the command-line
interface is used the HTML is converted to plain text.
Currently, QMTest recognizes the following built-in annotations:
'Result.CAUSE' -- For results whose outcome is not 'FAIL', this
annotation gives a brief description of why the test failed. The
preferred form of this message is a phrase like "Incorrect
output." or "Exception thrown." The message should begin
with a capital letter and end with a period. Most results formatters
will display this information prominently.
'Result.EXCEPTION' -- If an exeption was thrown during the test
execution, a brief description of the exception.
'Result.TARGET' -- This annotation indicates on which target the test
was executed.
'Result.TRACEBACK' -- If an exeption was thrown during the test
execution, a representation of the traceback indicating where the
exception was thrown.
A 'Result' object has methods that allow it to act as a dictionary
from annotation names to annotation values. You can directly add an
annotation to a 'Result' by writing code of the form 'result[CAUSE] =
"Exception thrown."'.
A 'Result' object is also used to describe the outcome of executing
either setup or cleanup phase of a 'Resource'.
|
|
__init__(self,
kind,
id,
outcome=PASS,
annotations={})
Construct a new 'Result'.
|
|
__getstate__(self)
Return a representation of this result for pickling.
|
|
__setstate__(self,
pickled_state)
Construct a 'Result' from its pickled form.
|
|
GetKind(self)
Return the kind of result this is.
|
|
GetOutcome(self)
Return the outcome associated with the test.
|
|
SetOutcome(self,
outcome,
cause=None,
annotations={})
Set the outcome associated with the test.
|
|
Annotate(self,
annotations)
Add 'annotations' to the current set of annotations.
|
|
Fail(self,
cause=None,
annotations={})
Mark the test as failing.
|
|
GetId(self)
Return the label for the test or resource.
|
|
GetCause(self)
Return the cause of failure, if the test failed.
|
|
SetCause(self,
cause)
Set the cause of failure.
|
|
Quote(self,
string)
Return a version of string suitable for an annotation value.
|
|
NoteException(self,
exc_info=None,
cause=None,
outcome=ERROR)
Note that an exception occurred during execution.
|
|
CheckExitStatus(self,
prefix,
desc,
status,
non_zero_exit_ok=0)
Check the exit status from a command.
|
|
MakeDomNode(self,
document)
Generate a DOM element node for this result.
|
|
__getitem__(self,
key)
|
|
__setitem__(self,
key,
value)
|
|
__delitem__(self,
key)
|
|
get(self,
key,
default=None)
|
|
has_key(self,
key)
|
|
keys(self)
|
|
items(self)
|
__init__(self,
kind,
id,
outcome=PASS,
annotations={})
(Constructor)
| source code |
Construct a new 'Result'.
'kind' -- The kind of result. The value must be one of the
'Result.kinds'.
'id' -- The label for the test or resource to which this result
corresponds.
'outcome' -- The outcome associated with the test. The value must be
one of the 'Result.outcomes'.
'annotations' -- The annotations associated with the test.
-
|
Return a representation of this result for pickling.
By using an explicit tuple representation of 'Result's when storing
them in a pickle file, we decouple our storage format from internal
implementation details (e.g., the names of private variables).
-
|
Construct a 'Result' from its pickled form.
-
|
Return the kind of result this is.
returns -- The kind of entity (one of the 'kinds') to which this
result corresponds.
-
|
Return the outcome associated with the test.
returns -- The outcome associated with the test. This value will be
one of the 'Result.outcomes'.
-
|
SetOutcome(self,
outcome,
cause=None,
annotations={})
| source code |
Set the outcome associated with the test.
'outcome' -- One of the 'Result.outcomes'.
'cause' -- If not 'None', this value becomes the value of the
'Result.CAUSE' annotation.
'annotations' -- The annotations are added to the current set of
annotations.
-
|
Add 'annotations' to the current set of annotations.
-
|
Mark the test as failing.
'cause' -- If not 'None', this value becomes the value of the
'Result.CAUSE' annotation.
'annotations' -- The annotations are added to the current set of
annotations.
-
|
Return the label for the test or resource.
returns -- A label indicating indicating to which test or resource
this result corresponds.
-
|
Return the cause of failure, if the test failed.
returns -- If the test failed, return the cause of the failure, if
available.
-
|
Set the cause of failure.
'cause' -- A string indicating the cause of failure. Like all
annotations, 'cause' will be interested as HTML.
-
|
Return a version of string suitable for an annotation value.
Performs appropriate quoting for a string that should be taken
verbatim; this includes HTML entity escaping, and addition of <pre>
tags.
'string' -- The verbatim string to be quoted.
returns -- The quoted string.
-
|
NoteException(self,
exc_info=None,
cause=None,
outcome=ERROR)
| source code |
Note that an exception occurred during execution.
'exc_info' -- A triple, in the same form as that returned from
'sys.exc_info'. If 'None', the value of 'sys.exc_info()' is used
instead.
'cause' -- The value of the 'Result.CAUSE' annotation. If 'None', a
default message is used.
'outcome' -- The outcome of the test, now that the exception has
occurred.
A test class can call this method if an exception occurs while the
test is being run.
-
|
CheckExitStatus(self,
prefix,
desc,
status,
non_zero_exit_ok=0)
| source code |
Check the exit status from a command.
'prefix' -- The prefix that should be used when creating result
annotations.
'desc' -- A description of the executing program.
'status' -- The exit status, as returned by 'waitpid'.
'non_zero_exit_ok' -- True if a non-zero exit code is not considered
failure.
returns -- False if the test failed, true otherwise.
-
|
Generate a DOM element node for this result.
Note that the context is not represented in the DOM node.
'document' -- The containing DOM document.
returns -- The element created.
-
|
__setitem__(self,
key,
value)
(Index assignment operator)
| source code |
-
|
__delitem__(self,
key)
(Index deletion operator)
| source code |
-
|
kinds
A list of the possible kinds.
-
- Value:
['resource_setup', 'resource_cleanup', 'test']
|
|
outcomes
A list of the possible outcomes.
The order of the 'outcomes' is significant; they are ordered from most
interesting to least interesting from the point of view of someone
browsing results.
-
- Value:
['ERROR', 'FAIL', 'UNTESTED', 'PASS']
|
|