xref: /kernel/linux/linux-5.10/tools/testing/selftests/tc-testing/creating-plugins/AddingPlugins.txt

tdc - Adding plugins for tdc

Author: Brenda J. Butler - bjb@mojatatu.com

ADDING PLUGINS
--------------

A new plugin should be written in python as a class that inherits from TdcPlugin.
There are some examples in plugin-lib.

The plugin can be used to add functionality to the test framework,
such as:

- adding commands to be run before and/or after the test suite
- adding commands to be run before and/or after the test cases
- adding commands to be run before and/or after the execute phase of the test cases
- ability to alter the command to be run in any phase:
    pre        (the pre-suite stage)
    prepare
    execute
    verify
    teardown
    post       (the post-suite stage)
- ability to add to the command line args, and use them at run time


The functions in the class should follow the following interfaces:

    def __init__(self)
    def pre_suite(self, testcount, testidlist)     # see "PRE_SUITE" below
    def post_suite(self, ordinal)                  # see "SKIPPING" below
    def pre_case(self, test_ordinal, testid)       # see "PRE_CASE" below
    def post_case(self)
    def pre_execute(self)
    def post_execute(self)
    def adjust_command(self, stage, command)       # see "ADJUST" below
    def add_args(self, parser)                     # see "ADD_ARGS" below
    def check_args(self, args, remaining)          # see "CHECK_ARGS" below


PRE_SUITE

This method takes a testcount (number of tests to be run) and
testidlist (array of test ids for tests that will be run).  This is
useful for various things, including when an exception occurs and the
rest of the tests must be skipped.  The info is stored in the object,
and the post_suite method can refer to it when dumping the "skipped"
TAP output.  The tdc.py script will do that for the test suite as
defined in the test case, but if the plugin is being used to run extra
tests on each test (eg, check for memory leaks on associated
co-processes) then that other tap output can be generated in the
post-suite method using this info passed in to the pre_suite method.


SKIPPING

The post_suite method will receive the ordinal number of the last
test to be attempted.  It can use this info when outputting
the TAP output for the extra test cases.


PRE_CASE

The pre_case method will receive the ordinal number of the test
and the test id.  Useful for outputing the extra test results.


ADJUST

The adjust_command method receives a string representing
the execution stage and a string which is the actual command to be
executed.  The plugin can adjust the command, based on the stage of
execution.

The stages are represented by the following strings:

    'pre'
    'setup'
    'command'
    'verify'
    'teardown'
    'post'

The adjust_command method must return the adjusted command so tdc
can use it.


ADD_ARGS

The add_args method receives the argparser object and can add
arguments to it.  Care should be taken that the new arguments do not
conflict with any from tdc.py or from other plugins that will be used
concurrently.

The add_args method should return the argparser object.


CHECK_ARGS

The check_args method is so that the plugin can do validation on
the args, if needed.  If there is a problem, and Exception should
be raised, with a string that explains the problem.

eg:  raise Exception('plugin xxx, arg -y is wrong, fix it')