summaryrefslogtreecommitdiffstats
path: root/compare-results.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-04-06add compare-results.sh, call from start-testsuite.shNeels Hofmeyr1-0/+198
Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d