path: root/
diff options
authorNeels Hofmeyr <>2018-04-05 16:56:38 +0200
committerNeels Hofmeyr <>2018-04-06 01:07:15 +0200
commit3cf797d7a59a9f5fc05bd22af32d21c23409e60d (patch)
treed3dd5b5741ba5e576814b8879319e8d85a076f7b /
parentdef4ad4bcefbe065bc3446b20fb8b32769158425 (diff)
add, call from
Compare current test results to the expected results, and exit in error on discrepancies. Add (trivially) grep junit xml output to determine which tests passed and which didn't, and compare against an expected-result.log, another junit file from a previous run. Summarize and determine success. Include an "xfail" feature: tests that are expected to fail are marked as "xfail", unexpected failures as "FAIL". In various subdirs, copy the current jenkins jobs' junit xml outputs as expected-results.log, so that we will start getting useful output in both jenkins runs and manual local runs. In, after running the tests, invoke the results comparison. Due to the single-line parsing nature, the script so far does not distinguish between error and failure. I doubt that we actually need to do that though. Related: OS#3136 Change-Id: I87d62a8be73d73a5eeff61a842e7c27a0066079d
Diffstat (limited to '')
1 files changed, 26 insertions, 2 deletions
diff --git a/ b/
index 8affdbae..5b187fa1 100755
--- a/
+++ b/
@@ -10,7 +10,9 @@ if [ $# -lt 1 ]; then
-CFG=$(basename "$SUITE").cfg
+SUITE_DIR="$(dirname "$SUITE")"
+SUITE_NAME="$(basename "$SUITE")"
if [ $# -gt 1 ]; then
@@ -19,4 +21,26 @@ if [ $# -gt 2 ]; then
-LD_LIBRARY_PATH=$(dirname "$SUITE"):/usr/lib/titan:/usr/ttcn3/lib ttcn3_start $SUITE $CFG $TEST
+LD_LIBRARY_PATH="$SUITE_DIR:/usr/lib/titan:/usr/ttcn3/lib" ttcn3_start $SUITE $CFG $TEST
+if [ ! -f "$expected" ]; then
+ echo "No expected results found, not comparing outcome. ($expected)"
+ exit 0
+# find the most recent junit output log here
+last_log="$(ls -1tr junit*.log | tail -n 1)"
+if [ ! -f "$last_log" ]; then
+ echo "No junit log found."
+ exit 1
+if [ ! -x "$compare" ]; then
+ echo "ERROR: cannot find $compare"
+ exit 1
+set -e
+"$compare" "$expected" "$last_log" $OSMO_TTCN3_COMPARE_ARGS