186d7f5d3SJohn Marino====== 286d7f5d3SJohn MarinoResult values: 386d7f5d3SJohn Marino====== 486d7f5d3SJohn Marino 586d7f5d3SJohn Marino#define RESULT_TIMEOUT 0x01 686d7f5d3SJohn Marino#define RESULT_SIGNALLED 0x02 786d7f5d3SJohn Marino#define RESULT_NOTRUN 0x03 886d7f5d3SJohn Marino#define RESULT_FAIL 0x04 986d7f5d3SJohn Marino#define RESULT_PASS 0x05 1086d7f5d3SJohn Marino#define RESULT_UNKNOWN 0x06 1186d7f5d3SJohn Marino#define RESULT_PREFAIL 0x07 1286d7f5d3SJohn Marino#define RESULT_POSTFAIL 0x08 1386d7f5d3SJohn Marino#define RESULT_BUILDFAIL 0x09 1486d7f5d3SJohn Marino 1586d7f5d3SJohn Marino 1686d7f5d3SJohn MarinoA testcase can be considered not run when the result is any of the following: 1786d7f5d3SJohn MarinoRESULT_BUILDFAIL, RESULT_PREFAIL, RESULT_NOTRUN, RESULT_UNKNOWN 1886d7f5d3SJohn Marino 1986d7f5d3SJohn MarinoA testcase can be considered run when the result is any of the following: 2086d7f5d3SJohn MarinoRESULT_TIMEOUT, RESULT_SIGNALLED, RESULT_FAIL, RESULT_PASS, RESULT_POSTFAIL 2186d7f5d3SJohn Marino 2286d7f5d3SJohn Marino 2386d7f5d3SJohn Marino 2486d7f5d3SJohn Marino 2586d7f5d3SJohn Marino====== 2686d7f5d3SJohn MarinoExecution of a single test case: 2786d7f5d3SJohn Marino====== 2886d7f5d3SJohn Marino1) chdir to testcase directory 2986d7f5d3SJohn Marino - if it fails, set RESULT_PREFAIL (sysbuf is of interest), goto (6) 3086d7f5d3SJohn Marino 3186d7f5d3SJohn Marino2) build testcase (make) (unless nobuild flag is set). 3286d7f5d3SJohn Marino + build_buf is used for stdout/stderr 3386d7f5d3SJohn Marino - if there is an internal driver error (that leads to not running the 3486d7f5d3SJohn Marino build command), set RESULT_PREFAIL (sysbuf is of interest), goto (6) 3586d7f5d3SJohn Marino - if the build command has a non-zero exit value, set the result to 3686d7f5d3SJohn Marino BUILDFAIL, unless it's a buildonly test case, in which it is set to 3786d7f5d3SJohn Marino the actual result value (TIMEOUT, SIGNALLED, FAIL) 3886d7f5d3SJohn Marino goto (6) 3986d7f5d3SJohn Marino 4086d7f5d3SJohn Marino3) run 'pre' command if intpre or pre is set. 4186d7f5d3SJohn Marino + precmd_buf is used for stdout/stderr 4286d7f5d3SJohn Marino - if there is an internal driver error (that leads to not running the 4386d7f5d3SJohn Marino command), set RESULT_PREFAIL (sysbuf is of interest), goto (6) 4486d7f5d3SJohn Marino - if the pre command has a non-zero exit value, set RESULT_PREFAIL and 4586d7f5d3SJohn Marino goto (6) 4686d7f5d3SJohn Marino 4786d7f5d3SJohn Marino4) run actual testcase, depending on type 4886d7f5d3SJohn Marino + stdout_buf is used for stdout 4986d7f5d3SJohn Marino + stderr_buf is used for stderr 5086d7f5d3SJohn Marino - for BUILDONLY: set RESULT_PASS since the build already succeeded 5186d7f5d3SJohn Marino - for userland and kernel: run the testcase, possibly as a different 5286d7f5d3SJohn Marino user (depending on the runas option), set the result to the actual 5386d7f5d3SJohn Marino result value (TIMEOUT, SIGNALLED, FAIL, NOTRUN) 5486d7f5d3SJohn Marino - if there is an internal driver error (that leads to not running the 5586d7f5d3SJohn Marino command), RESULT_NOTRUN is set (sysbuf is of interest) 5686d7f5d3SJohn Marino 5786d7f5d3SJohn Marino5) run 'post' command if intpost or post is set. 5886d7f5d3SJohn Marino + postcmd_buf is used for stdout/stderr 5986d7f5d3SJohn Marino - if there is an internal driver error (that leads to not running the 6086d7f5d3SJohn Marino command), set RESULT_POSTFAIL (sysbuf is of interest), goto (6) 6186d7f5d3SJohn Marino - if the post command has a non-zero exit value, set RESULT_POSTFAIL 6286d7f5d3SJohn Marino and goto (6) 6386d7f5d3SJohn Marino 6486d7f5d3SJohn Marino6) clean testcase directory (make clean) (unless nobuild flag is set). 6586d7f5d3SJohn Marino + cleanup_buf is used for stdout/stderr and system (driver error) buffer 6686d7f5d3SJohn Marino - no further action. 6786d7f5d3SJohn Marino 6886d7f5d3SJohn Marino7) results are saved. 6986d7f5d3SJohn Marino 7086d7f5d3SJohn Marino 7186d7f5d3SJohn Marino 7286d7f5d3SJohn Marino====== 7386d7f5d3SJohn MarinoTODO 7486d7f5d3SJohn Marino====== 7586d7f5d3SJohn Marino - read config defaults from file, not statically coded into config.c 7686d7f5d3SJohn Marino - add collected rusage into output dictionary 77