xref: /third_party/ltp/testcases/open_posix_testsuite/functional/timers/plan.txt

This document defines the plan for functional testing of POSIX Clocks
and Timers.

First a list of the features that will and will not be tested is given.

Then, an attempt is made to break down the features into possible testing
scenarios.  Not all of these may be implemented.  The assertions.xml file
will have the additional detail on which were chosen for implementation.

Features To Be Tested
=====================
The following features will be tested:
clocks - Testing that clocks can accurately keep time in certain scenarios.
sleep  - Testing that processes can go to sleep for specified periods of time.
timers - The following types of timers will be tested.
         absolute timer, no repeat	think:  alarm clock
         relative timer, no repeat	think:  stopwatch
         absolute timer, repeats	think:  grandfather clock
         relative timer, repeats	think:  repeating stopwatch
         Also, timer overruns will be counted.

Features Not To Be Tested
=========================
API testing of the clocks/timers functions is considered to be done by
the conformance section.
Load and other stress testing will be done in the stress section.
A few error handling tests will be created, but the majority will not for
this initial release.  (Over time, these may be added from regression
tests, or regression tests may suggest additional error handling tests to
create.)

Approach
========
In order to generate the test cases listed below, the following approaches
were taken:
- If it was not covered in the conformance area fully, determined the
  correct behavior of the features listed above and generated test scenarios
  to verify this behavior happened.
- Determined certain critical values that may cause failures to appear and
  generated test scenarios setting clocks or timers to these critical
  values.
- Determined race conditions that could appear or time dependencies for
  things happening and generated test cases to expose these issues.

Test Case Scenarios
===================
Some potential test case scenarios are listed after each functional
area.

Clocks
------
- Set clock time to: 0, a large number, all the dates that were
  critical for Y2K testing, set to time before daylight savings (and then
  after daylight savings). ==> This is probably multiple test cases.
- Call clock_gettime() mulitple times and ensure the time is always
  increasing (call also at nsec increments to ensure both nsec and sec
  are always increasing).
- Ensure we can tell time within acceptable jitter (over some period of time).
  - Test for a few different clocks.
  - Test after setting the time.
- Send a signal while setting the time.
- Kill the process (example of previous with SIGKILL) when setting the time.
- Have a timer expire when setting the time.
- Set the time from two different processes.

Sleep
-----
- Mimic the non-repeating timers tests as they apply to sleep.

Timers
------
 Non-repeating (relative and absolute)
 - Set timer expiration to: 0, a large number, all the dates that were
   critical for Y2K testing, set to time before daylight savings (and then
   after daylight savings). ==> This is probably multiple test cases.  Some
   are absolute timers only.
 - Have the clock change underneath a timer and ensure it expires
   correctly.
 - [Relative only] Set up multiple timers to expire after the same duration
   and ensure they expire in the order set up.  (May need to nanosleep
   for CLOCKRES nanoseconds in between set ups to get correct results.  Also,
   even if this fails, not necessarily a "bug" since many factors influence
   expiration time.)
 - Test out all the various events that can be sent for a timer expiration.
 - Set two timers to expire at the same time, different events, in the
   same process.
 - Set up two timers to expire for the same time in two different processes.
 - Set up one repeating timer to expire after X time and one non-repeating
   to expire after 2*X time.

 Repeating (relative and absolute)
 - Perform all non-repeating timer tests on repeating timers.
 - Ensure that these are still going off at the same time, over time.

 Overruns
 - Have multiple timers in a process blocked and ensure overrun count
   for each is correct.
 - Have a repeating timer blocked for a period of time and ensure overrun
   count is correct.
 - Ensure overrun count and timer_gettime() return correct values when
   SIGEV_NONE is used in ev.sigev_notify.
 - Try to call timer_getoverrun() as an overrun is happening.

Contributors:	julie.n.fleischer REMOVE-THIS AT intel DOT com
		geoffrey.r.gustafson REMOVE-THIS AT intel DOT com
		rolla.n.selbak REMOVE-THIS AT intel DOT com
		majid.awad REMOVE-THIS AT intel DOT com
		salwan.searty REMOVE-THIS AT intel DOT com
		george REMOVE-THIS AT mvista DOT com