aboutsummaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
authorPau Espin Pedrol <pespin@sysmocom.de>2020-06-04 19:45:13 +0200
committerPau Espin Pedrol <pespin@sysmocom.de>2020-06-04 20:03:07 +0200
commitab78df2902144f705776022b9910bc02a20c9328 (patch)
treedf9f0720fa0aaa39c613ebdbda152f627f0cce24 /doc
parent166dc10769b9df7c0ea21a9f08e608ad6a56e560 (diff)
doc: manual: Write initial Test API section
Diffstat (limited to 'doc')
-rw-r--r--doc/manuals/chapters/test_api.adoc107
1 files changed, 105 insertions, 2 deletions
diff --git a/doc/manuals/chapters/test_api.adoc b/doc/manuals/chapters/test_api.adoc
index f541231..f159348 100644
--- a/doc/manuals/chapters/test_api.adoc
+++ b/doc/manuals/chapters/test_api.adoc
@@ -1,4 +1,107 @@
== Test API
-*TODO* (in the meantime, look at src/osmo_gsm_tester/test.py, as well as
-suite.py, which calls the test's setup() function to get an idea)
+All tests run by {app-name} are python script files. On top of usual python
+standard features, {app-name} provides a set of public APIs and tools that
+these tests can use in order to interact with the core of {app-name}, like
+creating object classes, run processes synchronously or asynchronously, wait for
+events, retrieve specific configuration, etc. This section aims at documenting
+the most relevant tools available for tests.
+
+First of all, it is important to avoid blocking out of the core's main event loop in
+the test, since doing that will prevent {app-name} core functionalities to work
+properly, such as control of asynchronous processes.
+
+To get access to those functionalities, a test must import a test environment
+previously prepared by {app-name} before the test was started:
+[source,python]
+----
+#!/usr/bin/env python3
+from osmo_gsm_tester.testenv import *
+----
+
+After the test environment is imported, aome usual functionalities are available
+directly under the global scope. Specially noticeable is the existence of object
+_tenv_, which provides access to most of the functionalities.
+
+The test can simply ask {app-name} to sleep some time, while giving control back
+to {app-name} core's mainloop:
+[source,python]
+----
+sleep(3) # sleep for 3 seconds
+----
+
+One can also wait for events in the background, for instance launch a child
+process locally in the same host and wait for its termination:
+[source,python]
+----
+proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4') <1>
+tenv.remember_to_stop(proc) <2>
+proc.launch() <3>
+proc.wait() <4>
+----
+<1> Create process object. This line doesn't yet runs it.
+<2> Make sure the core will kill the process if this test fails
+<3> Start process asynchronously
+<4> wait until process is done. One could waiting generically here too: _wait(proc.terminated)_
+
+If running asynchronously is not needed, one can run synchronously in an easy
+way:
+[source,python]
+----
+proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4')
+proc.launch_sync()
+----
+
+One can also log output using either the regular _print_ function from python,
+or using {app-name} specific functions available:
+[source,python]
+----
+log('this is a regular log message')
+dbg('this is a dbg message, only printed on outputs where dbg is enabled')
+err('outputs log message for non-expected events')
+print('this is the same as log()')
+----
+
+The test also gains access to suite and/or test specific configuration through
+different APIs:
+[source,python]
+----
+test_config = tenv.config_test_specific()
+threshold = int(test_config.get('threshold', 2))
+suite_config = tenv.config_suite_specific()
+foobar = suite_config['foobar']
+----
+
+A test requiring a really specific config file for an object class it is going
+to run can provide its own template files by overlaying an own directory
+containing them on top of the usual default directory where object class
+templates are (_osmo-gsm-tester.git/src/osmo_gsm_tester/obj/templates/_):
+[source,python]
+----
+tenv.set_overlay_template_dir(os.path.join(os.path.dirname(__file__), 'mytemplatedir'))
+----
+
+Several tests in a suite can also share code by using some APIs provided by
+{app-names}. The shared python code must be placed in files under the 'lib/'
+subdirectory in the suite directory where the test belongs to.
+[source,python]
+----
+# File containing function foobar() available under ${suite_dir}/lib/testlib.py:
+import testlib
+tenv.test_import_modules_register_for_cleanup(testlib)
+from testlib import foobar
+----
+
+For a complete set of features and how to use them, one can have a look at real
+examples present in {app-name} git repository under the _sysmocom/_ directory.
+Besides those, have a look too a _testenv.py_ file, which implements the 'tenv'
+object available to tests.
+
+=== Test verdict
+
+In general, a test reaching the end of the file and returning control to
+{app-name} core will be flagged as a successful test (PASS).
+
+If an exception is thrown from within the test file and propagated to
+{app-name}, the test will be considered as failed and {app-name} will store all
+failure related information from the caught exception.