summaryrefslogtreecommitdiffstats
path: root/ggsn_tests
diff options
context:
space:
mode:
authorNeels Hofmeyr <neels@hofmeyr.de>2018-04-05 16:56:38 +0200
committerNeels Hofmeyr <neels@hofmeyr.de>2018-04-06 01:07:15 +0200
commit3cf797d7a59a9f5fc05bd22af32d21c23409e60d (patch)
treed3dd5b5741ba5e576814b8879319e8d85a076f7b /ggsn_tests
parentdef4ad4bcefbe065bc3446b20fb8b32769158425 (diff)
add compare-results.sh, call from start-testsuite.sh
Compare current test results to the expected results, and exit in error on discrepancies. Add compare-result.sh: (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In start-testsuite.sh, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
Diffstat (limited to 'ggsn_tests')
-rw-r--r--ggsn_tests/expected-results.log14
1 files changed, 14 insertions, 0 deletions
diff --git a/ggsn_tests/expected-results.log b/ggsn_tests/expected-results.log
new file mode 100644
index 0000000..bff953a
--- /dev/null
+++ b/ggsn_tests/expected-results.log
@@ -0,0 +1,14 @@
+<?xml version="1.0"?>
+<testsuite name='GGSN_Tests' tests='11' failures='0' errors='0' skipped='0' inconc='0' time='205.00'>
+ <testcase classname='GGSN_Tests' name='TC_pdp4_act_deact' time='0.006364'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp4_act_deact_ipcp' time='0.003752'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp4_act_deact_pcodns' time='0.003510'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp4_act_deact_gtpu_access' time='6.011141'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp4_clients_interact' time='0.006517'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp6_act_deact' time='0.003312'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp6_act_deact_pcodns' time='0.003403'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp6_act_deact_icmp6' time='0.005114'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp6_act_deact_gtpu_access' time='9.018065'/>
+ <testcase classname='GGSN_Tests' name='TC_pdp6_clients_interact' time='6.014695'/>
+ <testcase classname='GGSN_Tests' name='TC_echo_req_resp' time='0.001674'/>
+</testsuite>