diff options
author | Neels Hofmeyr <neels@hofmeyr.de> | 2018-11-28 17:15:42 +0100 |
---|---|---|
committer | Neels Hofmeyr <neels@hofmeyr.de> | 2018-11-28 17:15:42 +0100 |
commit | 09b0b24093725bb85130356832b1c8bb9b8e7c94 (patch) | |
tree | 7e4b87e1aef177ccb6e076bae7dab00c3335a508 /doc | |
parent | 5b84115d22014bd23e9360cc0dc80a677638a5e5 (diff) | |
parent | 4c47659fb50ae9cc3b66d75e257d590ee8b52e08 (diff) |
Merge history from osmo-gsm-manuals.git
Diffstat (limited to 'doc')
-rw-r--r-- | doc/manuals/Makefile | 7 | ||||
-rw-r--r-- | doc/manuals/chapters/config.adoc | 361 | ||||
-rw-r--r-- | doc/manuals/chapters/debugging.adoc | 4 | ||||
-rw-r--r-- | doc/manuals/chapters/install.adoc | 664 | ||||
-rw-r--r-- | doc/manuals/chapters/intro.adoc | 400 | ||||
-rw-r--r-- | doc/manuals/chapters/test_api.adoc | 4 | ||||
-rw-r--r-- | doc/manuals/chapters/trial.adoc | 31 | ||||
-rw-r--r-- | doc/manuals/osmo-gsm-tester-manual-docinfo.xml | 47 | ||||
-rw-r--r-- | doc/manuals/osmo-gsm-tester-manual.adoc | 20 |
9 files changed, 1538 insertions, 0 deletions
diff --git a/doc/manuals/Makefile b/doc/manuals/Makefile new file mode 100644 index 0000000..43c5a37 --- /dev/null +++ b/doc/manuals/Makefile @@ -0,0 +1,7 @@ +TOPDIR = .. + +ASCIIDOC = osmo-gsm-tester-manual.adoc +ASCIIDOC_DEPS = chapters/*.adoc +include $(TOPDIR)/build/Makefile.asciidoc.inc + +include $(TOPDIR)/build/Makefile.common.inc diff --git a/doc/manuals/chapters/config.adoc b/doc/manuals/chapters/config.adoc new file mode 100644 index 0000000..7e250e0 --- /dev/null +++ b/doc/manuals/chapters/config.adoc @@ -0,0 +1,361 @@ +== Configuration + +[[config_paths]] +=== Config Paths + +The osmo-gsm-tester looks for configuration files in various standard +directories in this order: + +- '$HOME/.config/osmo-gsm-tester/' +- '/usr/local/etc/osmo-gsm-tester/' +- '/etc/osmo-gsm-tester/' + +The config location can also be set by an environment variable +'$OSMO_GSM_TESTER_CONF', which then overrides the above locations. + +The osmo-gsm-tester expects to find the following configuration files in a +configuration directory: + +- 'paths.conf' +- 'resources.conf' +- 'default-suites.conf' (optional) +- 'defaults.conf' (optional) + +These are described in detail in the following sections. + +=== Format: YAML, and its Drawbacks + +The general configuration format used is YAML. The stock python YAML parser +does have several drawbacks: too many complex possibilities and alternative +ways of formatting a configuration, but at the time of writing seems to be the +only widely used configuration format that offers a simple and human readable +formatting as well as nested structuring. It is recommended to use only the +exact YAML subset seen in this manual in case the osmo-gsm-tester should move +to a less bloated parser in the future. + +Careful: if a configuration item consists of digits and starts with a zero, you +need to quote it, or it may be interpreted as an octal notation integer! Please +avoid using the octal notation on purpose, it is not provided intentionally. + +[[paths_conf]] +=== 'paths.conf' + +The 'paths.conf' file defines where to store the global state (of reserved +resources) and where to find suite and scenario definitions. + +Any relative paths found in a 'paths.conf' file are interpreted as relative to +the directory of that 'paths.conf' file. + +Example: + +---- +state_dir: '/var/tmp/osmo-gsm-tester/state' +suites_dir: '/usr/local/src/osmo-gsm-tester/suites' +scenarios_dir: './scenarios' +---- + +[[state_dir]] +==== 'state_dir' + +It contains global or system-wide state for osmo-gsm-tester. In a typical state +dir you can find the following files: + +'last_used_msisdn.state':: + Contains last used msisdn number, which is automatically increased every + time osmo-gsm-tester needs to assign a new subscriber in a test. +'lock':: + Lock file used to implement a mutual exclusion zone around the + 'reserved_resources.state' file. +'reserved_resources.state':: + File containing a set of reserved resources by any number of + osmo-gsm-tester instances. Each osmo-gsm-tester instance is responsible + to clear its resources from the list once it is done using them and are + no longer reserved. + +If you would like to set up several separate configurations (not typical), note +that the 'state_dir' is used to reserve resources, which only works when all +configurations that share resources also use the same 'state_dir'. + +This way, several concurrent users of osmo-gsm-tester (ie. several +osmo-gsm-tester processes running in parallel) can run without interfering with +each other (e.g. using same ARFCN, same IP or same ofono modem path). + +[[suites_dir]] +==== 'suites_dir' + +Suites contain a set of tests which are designed to be run together to test a +set of features given a specific set of resources. As a result, resources are +allocated per suite and not per test. + +Tests for a given suite are located in the form of '.py' python scripts in the +same directory where the 'suite.conf' lays. + +[[scenarios_dir]] +==== 'scenarios_dir' + +This dir contains scenario configuration files. + +Scenarios define constraints to serve the resource requests of a 'suite.conf', +to select specific resources from the general resource pool specified in 'resources.conf'. + +All 'times' attributes are expanded before matching. For example, if a 'suite.conf' +requests two BTS, we may enforce that both BTS should be of type 'osmo-bts-sysmo' in +these ways: + +---- +resources: + bts: + - type: osmo-bts-sysmo + - type: osmo-bts-sysmo +---- + +or alternatively, + +---- +resources: + bts: + - times: 2 + type: osmo-bts-sysmo +---- + +If only one resource is specified in the scenario, then the resource allocator +assumes the restriction is to be applied to the first resource and that remaining +resources have no restrictions to be taken into consideration. + +To apply restrictions only on the second resource, the first element can be left +emtpy, like: + +---- +resources: + bts: + - {} + - type: osmo-bts-sysmo +---- + +On the 'osmo_gsm_tester.py' command line and the 'default_suites.conf', any number of +such scenario configurations can be combined in the form: + +---- +<suite_name>:<scenario>[+<scenario>[+...]] +---- + +e.g. + +---- +my_suite:sysmo+tch_f+amr +---- + +[[resources_conf]] +=== 'resources.conf' + +The 'resources.conf' file defines which hardware is connected to the main unit, +as well as which limited configuration items (like IP addresses or ARFCNs) +should be used. + +These resources are allocated dynamically and are not configured explicitly: + +- MSISDN: phone numbers are dealt out to test scripts in sequence on request. + +A 'resources.conf' is structured as a list of items for each resource type, +where each item has one or more settings -- for an example, see +<<resources_conf_example>>. + +These kinds of resource are known: + +'ip_address':: + List of IP addresses to run osmo-nitb instances on. The main unit + typically has a limited number of such IP addresses configured, which + the connected BTS models can see on their network. + 'addr'::: + IPv4 address of the local interface. + +'bts':: + List of available BTS hardware. + 'label'::: + human readable label for your own reference + 'type'::: + which way to launch this BTS, one of + - 'osmo-bts-sysmo' + - 'osmo-bts-trx' + - 'osmo-bts-octphy' + - 'ipa-nanobts' + 'ipa_unit_id'::: + ip.access unit id to be used by the BTS, written into BTS and BSC config. + 'addr'::: + Remote IP address of the BTS for BTS like sysmoBTS, and local IP address + to bind to for locally run BTS such as osmo-bts-trx. + 'band'::: + GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of: + - 'GSM-1800' + - 'GSM-1900' + - (*TODO*: more bands) + 'trx_list'::: + Specific TRX configurations for this BTS. There should be as many of + these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without + special configuration for them.) + 'hw_addr':::: + Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66', + only used for osmo-bts-octphy. (*TODO*: and nanobts??) + 'net_device':::: + Local network device to reach the TRX's 'hw_addr' at, only used for + osmo-bts-octphy. Example: 'eth0'. + 'nominal_power':::: + Nominal power to be used by the TRX. + 'max_power_red':::: + Max power reduction to apply to the nominal power of the TRX. +'arfcn':: + List of ARFCNs to use for running BTSes, which defines the actual RF + frequency bands used. + 'arfcn'::: + ARFCN number, see e.g. + https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number + (note that the resource type 'arfcn' contains an item trait also named + 'arfcn'). + 'band'::: + GSM band name to use this ARFCN for, same as for 'bts:band' above. + +'modem':: + List of modems reachable via ofono and information on the inserted SIM + card. (Note: the MSISDN is allocated dynamically in test scripts). + 'label'::: + Human readable label for your own reference, which also appears in logs. + 'path'::: + Ofono's path for this modem, like '/modemkind_99'. + 'imsi'::: + IMSI of the inserted SIM card, like '"123456789012345"'. + 'ki'::: + 16 byte authentication/encryption KI of the inserted SIM card, in + hexadecimal notation (32 characters) like + + '"00112233445566778899aabbccddeeff"'. + 'auth_algo'::: + Authentication algorithm to be used with the SIM card. One of: + - 'none' + - 'xor' + - 'comp128v1' + 'ciphers'::: + List of ciphers that this modem supports, used to match + requirements in suites or scenarios. Any combination of: + - 'a5_0' + - 'a5_1' + - 'a5_2' + - 'a5_3' + - 'a5_4' + - 'a5_5' + - 'a5_6' + - 'a5_7' + 'features'::: + List of features that this modem supports, used to match requirements in + suites or scenarios. Any combination of: + - 'sms' + - 'gprs' + - 'voice' + - 'ussd' + +Side note: at first sight it might make sense to the reader to rather structure +e.g. the 'ip_address' or 'arfcn' configuration as + +'"arfcn: GSM-1800: [512, 514, ...]"', + +but the more verbose format is chosen to stay consistent with the general +structure of resource configurations, which the resource allocation algorithm +uses to resolve required resources according to their traits. These +configurations look cumbersome because they exhibit only one trait / a trait +that is repeated numerous times. No special notation for these cases is +available (yet). + +[[default_suites]] +=== 'default-suites.conf' (optional) + +The 'default-suites.conf' file contains a list of 'suite:scenario+scenario+...' +combination strings as defined by the 'osmo-gsm-tester.py -s' commandline +option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the +'-s' arguments are taken from this file instead. Each of these suite + scenario +combinations is run in sequence. + +A suite name must match the name of a directory in the 'suites_dir' as defined +by 'paths.conf'. + +A scenario name must match the name of a configuration file in the +'scenarios_dir' as defined by 'paths.conf' (optionally without the '.conf' +suffix). + +For 'paths.conf', see <<paths_conf>>. + +Example of a 'default-suites.conf' file: + +---- +- sms:sysmo +- voice:sysmo+tch_f +- voice:sysmo+tch_h +- voice:sysmo+dyn_ts +- sms:trx +- voice:trx+tch_f +- voice:trx+tch_h +- voice:trx+dyn_ts +---- + +=== 'defaults.conf' (optional) + +Each binary run by osmo-gsm-tester, e.g. 'osmo-nitb' or 'osmo-bts-sysmo', +typically has a configuration file template that is populated with values for a +trial run. + +Some of these values are provided by the 'resources.conf' from the allocated +resource(s), but not all values can be populated this way: some osmo-nitb +configuration values like the network name, encryption algorithm or timeslot +channel combinations are in fact not resources (only the nitb's interface +address is). These additional settings may be provided by the scenario +configurations, but in case the provided scenarios leave some values unset, +they are taken from this 'defaults.conf'. (A 'scenario.conf' or a +'resources.conf' providing a similar setting always has precedence over the +values given in a 'defaults.conf'). + +Example of a 'defaults.conf': + +---- +nitb: + net: + mcc: 901 + mnc: 70 + short_name: osmo-gsm-tester-nitb + long_name: osmo-gsm-tester-nitb + auth_policy: closed + encryption: a5_0 + +bsc: + net: + mcc: 901 + mnc: 70 + short_name: osmo-gsm-tester-msc + long_name: osmo-gsm-tester-msc + auth_policy: closed + encryption: a5_0 + authentication: optional + +msc: + net: + mcc: 901 + mnc: 70 + short_name: osmo-gsm-tester-msc + long_name: osmo-gsm-tester-msc + auth_policy: closed + encryption: a5_0 + authentication: optional + +bsc_bts: + location_area_code: 23 + base_station_id_code: 63 + stream_id: 255 + osmobsc_bts_type: sysmobts + trx_list: + - nominal_power: 23 + max_power_red: 0 + arfcn: 868 + timeslot_list: + - phys_chan_config: CCCH+SDCCH4 + - phys_chan_config: SDCCH8 + - phys_chan_config: TCH/F_TCH/H_PDCH + - phys_chan_config: TCH/F_TCH/H_PDCH + - phys_chan_config: TCH/F_TCH/H_PDCH + - phys_chan_config: TCH/F_TCH/H_PDCH + - phys_chan_config: TCH/F_TCH/H_PDCH + - phys_chan_config: TCH/F_TCH/H_PDCH +---- diff --git a/doc/manuals/chapters/debugging.adoc b/doc/manuals/chapters/debugging.adoc new file mode 100644 index 0000000..bea1e5c --- /dev/null +++ b/doc/manuals/chapters/debugging.adoc @@ -0,0 +1,4 @@ +[[debugging]] +== Debugging + +*TODO*: describe how to invoke 'ipdb3' and step into a suite's test script diff --git a/doc/manuals/chapters/install.adoc b/doc/manuals/chapters/install.adoc new file mode 100644 index 0000000..d19f909 --- /dev/null +++ b/doc/manuals/chapters/install.adoc @@ -0,0 +1,664 @@ +== Installation on Main Unit + +The main unit is a general purpose computer that orchestrates the tests. It +runs the core network components, controls the modems and so on. This can be +anything from a dedicated production rack unit to your laptop at home. + +This manual will assume that tests are run from a jenkins build slave, by a user +named 'jenkins' that belong to group 'osmo-gsm-tester'. The user configuration +for manual test runs and/or a different user name is identical, simply replace +the user name or group. + +=== Osmo-gsm-tester Dependencies + +On a Debian/Ubuntu based system, these commands install the packages needed to +run the osmo-gsm-tester.py code, i.e. install these on your main unit: + +---- +apt-get install \ + dbus \ + tcpdump \ + sqlite3 \ + python3 \ + python3-yaml \ + python3-mako \ + python3-gi \ + ofono \ + patchelf \ + sudo \ + libcap2-bin \ + python3-pip +pip3 install pydbus +pip3 install git+git://github.com/podshumok/python-smpplib.git +---- + +IMPORTANT: ofono may need to be installed from source to contain the most +recent fixes needed to operate your modems. This depends on the modem hardware +used and the tests run. Please see <<hardware_modems>>. + +To run osmo-bts-trx with a USRP attached, you may need to install a UHD driver. +Please refer to http://osmocom.org/projects/osmotrx/wiki/OsmoTRX#UHD for +details; the following is an example for the B200 family USRP devices: + +---- +apt-get install libuhd-dev uhd-host +/usr/lib/uhd/utils/uhd_images_downloader.py +---- + +[[jenkins_deps]] +==== Osmocom Build Dependencies + +Each of the jenkins builds requires individual dependencies. This is generally +the same as for building the software outside of osmo-gsm-tester and will not +be detailed here. For the Osmocom projects, refer to +http://osmocom.org/projects/cellular-infrastructure/wiki/Build_from_Source . Be +aware of specific requirements for BTS hardware: for example, the +osmo-bts-sysmo build needs the sysmoBTS SDK installed on the build slave, which +should match the installed sysmoBTS firmware. + + +[[configure_jenkins_slave]] +=== Jenkins Build and Run Slave + +==== Create 'jenkins' User on Main Unit + +On the main unit, create a jenkins user: + +---- +useradd -m jenkins +---- + +==== Install Java on Main Unit + +To be able to launch the Jenkins build slave, a Java RE must be available on +the main unit. For example: + +---- +apt-get install default-jdk +---- + +==== Allow SSH Access from Jenkins Master + +Create an SSH keypair to be used for login on the osmo-gsm-tester. This may be +entered on the jenkins web UI; alternatively, use the jenkins server's shell: + +Login on the main jenkins server shell and create an SSH keypair, for example: + +---- +# su jenkins +$ mkdir -p /usr/local/jenkins/keys +$ ssh-keygen +Generating public/private rsa key pair. +Enter file in which to save the key (/home/jenkins/.ssh/id_rsa): /usr/local/jenkins/keys/osmo-gsm-tester-rnd +Enter passphrase (empty for no passphrase): <enter a passphrase> +Enter same passphrase again: <enter a passphrase> +Your identification has been saved in /usr/local/jenkins/keys/osmo-gsm-tester-rnd +Your public key has been saved in /usr/local/jenkins/keys/osmo-gsm-tester-rnd.pub. +The key fingerprint is: +... +---- + +Copy the public key to the main unit, e.g. copy-paste: + +---- +cat /usr/local/jenkins/keys/osmo-gsm-tester-rnd.pub +# copy this public key +---- + +On the main unit: + +---- +mkdir ~jenkins/.ssh +cat > ~jenkins/.ssh/authorized_keys +# paste above public key and hit Ctrl-D +chown -R jenkins: ~jenkins/.ssh +---- + +Make sure that the user running the jenkins master accepts the main unit's host +identification. There must be an actual RSA host key available in the +known_hosts file for the jenkins master to be able to log in. Simply calling +ssh and accepting the host key as usual is not enough. Jenkins may continue to +say "Host key verification failed". + +To place an RSA host key in the jenkins' known_hosts file, you may do: + +On the Jenkins master: + +---- +main_unit_ip=10.9.8.7 +ssh-keyscan -H $main_unit_ip >> ~jenkins/.ssh/known_hosts +chown jenkins: ~jenkins/.ssh/known_hosts +---- + +Verify that the jenkins user on the Jenkins master has SSH access to the main +unit: + +---- +su jenkins +main_unit_ip=10.9.8.7 +ssh -i /usr/local/jenkins/keys/osmo-gsm-tester-rnd jenkins@$main_unit_ip +exit +---- + +[[install_add_jenkins_slave]] +==== Add Jenkins Slave + +In the jenkins web UI, add a new build slave for the osmo-gsm-tester: + +* 'Manage Jenkins' +** 'Manage Nodes' +*** 'New Node' +**** Enter a node name, e.g. "osmo-gsm-tester-1" + + (the "-1" is just some identification in case you'd like to add another + setup later). +**** 'Permanent Agent' + +Configure the node as: + +* '# of executors': 1 +* 'Remote root directory': "/home/jenkins" +* 'Labels': "osmo-gsm-tester" + + (This is a general label common to all osmo-gsm-tester build slaves you may set up in the future.) +* 'Usage': 'Only build jobs with label expressions matching this node' +* 'Launch method': 'Launch slave agents via SSH' +** 'Host': your main unit's IP address +** 'Credentials': choose 'Add' / 'Jenkins' +*** 'Domain': 'Global credentials (unrestricted)' +*** 'Kind': 'SSH Username with private key' +*** 'Scope': 'Global' +*** 'Username': "jenkins" + + (as created on the main unit above) +*** 'Private Key': 'From a file on Jenkins master' +**** 'File': "/usr/local/jenkins/keys/osmo-gsm-tester-rnd" +*** 'Passphrase': enter same passphrase as above +*** 'ID': "osmo-gsm-tester-1" +*** 'Name': "jenkins for SSH to osmo-gsm-tester-1" + +The build slave should be able to start now. + + +==== Add Build Jobs + +There are various jenkins-build-* scripts in osmo-gsm-tester/contrib/, which +can be called as jenkins build jobs to build and bundle binaries as artifacts, +to be run on the osmo-gsm-tester main unit and/or BTS hardware. + +Be aware of the dependencies, as hinted at in <<jenkins_deps>>. + +While the various binaries could technically be built on the osmo-gsm-tester +main unit, it is recommended to use a separate build slave, to take load off +of the main unit. + +On your jenkins master, set up build jobs to call these scripts -- typically +one build job per script. Look in contrib/ and create one build job for each of +the BTS types you would like to test, as well as one for the 'build-osmo-nitb'. + +These are generic steps to configure a jenkins build +job for each of these build scripts, by example of the +jenkins-build-osmo-nitb.sh script; all that differs to the other scripts is the +"osmo-nitb" part: + +* 'Project name': "osmo-gsm-tester_build-osmo-nitb" + + (Replace 'osmo-nitb' according to which build script this is for) +* 'Discard old builds' + + Configure this to taste, for example: +** 'Max # of build to keep': "20" +* 'Restrict where this project can be run': Choose a build slave label that + matches the main unit's architecture and distribution, typically a Debian + system, e.g.: "linux_amd64_debian8" +* 'Source Code Management': +** 'Git' +*** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" +*** 'Branch Specifier': "*/master" +*** 'Additional Behaviors' +**** 'Check out to a sub-directory': "osmo-gsm-tester" +* 'Build Triggers' + + The decision on when to build is complex. Here are some examples: +** Once per day: + + 'Build periodically': "H H * * *" +** For the Osmocom project, the purpose is to verify our software changes. + Hence we would like to test every time our code has changed: +*** We could add various git repositories to watch, and enable 'Poll SCM'. +*** On jenkins.osmocom.org, we have various jobs that build the master branches + of their respective git repositories when a new change was merged. Here, we + can thus trigger e.g. an osmo-nitb build for osmo-gsm-tester everytime the + master build has run: + + 'Build after other projects are built': "OpenBSC" +*** Note that most of the Osmocom projects also need to be re-tested when their + dependencies like libosmo* have changed. Triggering on all those changes + typically causes more jenkins runs than necessary: for example, it rebuilds + once per each dependency that has rebuilt due to one libosmocore change. + There is so far no trivial way known to avoid this. It is indeed safest to + rebuild more often. +* 'Build' +** 'Execute Shell' ++ +---- +#!/bin/sh +set -e -x +./osmo-gsm-tester/contrib/jenkins-build-osmo-nitb.sh +---- ++ +(Replace 'osmo-nitb' according to which build script this is for) + +* 'Post-build Actions' +** 'Archive the artifacts': "*.tgz, *.md5" + + (This step is important to be able to use the built binaries in the run job + below.) + + +TIP: When you've created one build job, it is convenient to create further +build jobs by copying the first and, e.g., simply replacing all "osmo-nitb" +with "osmo-bts-trx". + +==== Add Run Job + +This is the jenkins job that runs the tests on the GSM hardware: + +* It sources the artifacts from jenkins' build jobs. +* It runs on the osmo-gsm-tester main unit. + +Here is the configuration for the run job: + +* 'Project name': "osmo-gsm-tester_run" +* 'Discard old builds' + + Configure this to taste, for example: +** 'Max # of build to keep': "20" +* 'Restrict where this project can be run': "osmo-gsm-tester" + + (to match the 'Label' configured in <<install_add_jenkins_slave>>). +* 'Source Code Management': +** 'Git' +*** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" +*** 'Branch Specifier': "*/master" +*** 'Additional Behaviors' +**** 'Check out to a sub-directory': "osmo-gsm-tester" +**** 'Clean before checkout' +* 'Build Triggers' + + The decision on when to build is complex. For this run job, it is suggested + to rebuild: +** after each of above build jobs that produced new artifacts: + + 'Build after other projects are built': "osmo-gsm-tester_build-osmo-nitb, + osmo-gsm-tester_build-osmo-bts-sysmo, osmo-gsm-tester_build-osmo-bts-trx" + + (Add each build job name you configured above) +** as well as once per day: + + 'Build periodically': "H H * * *" +** and, in addition, whenever the osmo-gsm-tester scripts have been modified: + + 'Poll SCM': "H/5 * * * *" + + (i.e. look every five minutes whether the upstream git has changed) +* 'Build' +** Copy artifacts from each build job you have set up: +*** 'Copy artifacts from another project' +**** 'Project name': "osmo-gsm-tester_build-osmo-nitb" +**** 'Which build': 'Latest successful build' +**** enable 'Stable build only' +**** 'Artifacts to copy': "*.tgz, *.md5" +*** Add a separate similar 'Copy artifacts...' section for each build job you + have set up. +** 'Execute Shell' ++ +---- +#!/bin/sh +set -e -x + +# debug: provoke a failure +#export OSMO_GSM_TESTER_OPTS="-s debug -t fail" + +PATH="$PWD/osmo-gsm-tester/src:$PATH" \ + ./osmo-gsm-tester/contrib/jenkins-run.sh +---- ++ +Details: + +*** The 'jenkins-run.sh' script assumes to find the 'osmo-gsm-tester.py' in the + '$PATH'. To use the most recent osmo-gsm-tester code here, we direct + '$PATH' to the actual workspace checkout. This could also run from a sytem + wide install, in which case you could omit the explicit PATH to + "$PWD/osmo-gsm-tester/src". +*** This assumes that there are configuration files for osmo-gsm-tester placed + on the system (see <<config_paths>>). +*** If you'd like to check the behavior of test failures, you can uncomment the + line below "# debug" to produce a build failure on every run. Note that + this test typically produces a quite empty run result, since it launches no + NITB nor BTS. +* 'Post-build Actions' +** 'Archive the artifacts' +*** 'Files to archive': "*-run.tgz, *-bin.tgz" + + This stores the complete test report with config files, logs, stdout/stderr + output, pcaps as well as the binaries used for the test run in artifacts. + This allows analysis of older builds, instead of only the most recent build + (which cleans up the jenkins workspace every time). The 'trial-N-run.tgz' + and 'trial-N-bin.tgz' archives are produced by the 'jenkins-run.sh' script, + both for successful and failing runs. + +=== Install osmo-gsm-tester on Main Unit + +This assumes you have already created the jenkins user (see <<configure_jenkins_slave>>). + +==== User Permissions + +On the main unit, create a group for all users that should be allowed to use +the osmo-gsm-tester, and add users (here 'jenkins') to this group. + +---- +groupadd osmo-gsm-tester +gpasswd -a jenkins osmo-gsm-tester +---- + +NOTE: you may also need to add users to the 'usrp' group, see +<<user_config_uhd>>. + +A user added to a group needs to re-login for the group permissions to take +effect. + +This group needs the following permissions: + +===== Paths + +Assuming that you are using the example config, prepare a system wide state +location in '/var/tmp': + +---- +mkdir -p /var/tmp/osmo-gsm-tester/state +chown -R :osmo-gsm-tester /var/tmp/osmo-gsm-tester +chmod -R g+rwxs /var/tmp/osmo-gsm-tester +setfacl -d -m group:osmo-gsm-tester:rwx /var/tmp/osmo-gsm-tester/state +---- + +IMPORTANT: the state directory needs to be shared between all users potentially +running the osmo-gsm-tester to resolve resource allocations. Above 'setfacl' +command sets the access control to keep all created files group writable. + +With the jenkins build as described here, the trials will live in the build +slave's workspace. Other modes of operation (a daemon scheduling concurrent +runs, *TODO*) may use a system wide directory to manage trials to run: + +---- +mkdir -p /var/tmp/osmo-gsm-tester/trials +chown -R :osmo-gsm-tester /var/tmp/osmo-gsm-tester +chmod -R g+rwxs /var/tmp/osmo-gsm-tester +---- + +===== Allow DBus Access to ofono + +Put a DBus configuration file in place that allows the 'osmo-gsm-tester' group +to access the org.ofono DBus path: + +---- +cat > /etc/dbus-1/system.d/osmo-gsm-tester.conf <<END +<!-- Additional rules for the osmo-gsm-tester to access org.ofono from user + land --> + +<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN" + "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"> +<busconfig> + + <policy group="osmo-gsm-tester"> + <allow send_destination="org.ofono"/> + </policy> + +</busconfig> +END +---- + +(No restart of dbus nor ofono necessary.) + +[[install_capture_packets]] +===== Capture Packets + +In order to allow collecting pcap traces of the network communication for later +reference, allow the osmo-gsm-tester group to capture packets using the 'tcpdump' +program: + +---- +chgrp osmo-gsm-tester /usr/sbin/tcpdump +chmod 750 /usr/sbin/tcpdump +setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump +---- + +Put 'tcpdump' in the '$PATH' -- assuming that 'tcpdump' is available for root: + +---- +ln -s `which tcpdump` /usr/local/bin/tcpdump +---- + +TIP: Why a symlink in '/usr/local/bin'? On Debian, 'tcpdump' lives in +'/usr/sbin', which is not part of the '$PATH' for non-root users. To avoid +hardcoding non-portable paths in the osmo-gsm-tester source, 'tcpdump' must be +available in the '$PATH'. There are various trivial ways to modify '$PATH' for +login shells, but the jenkins build slave typically runs in a *non-login* +shell; modifying non-login shell enviroments is not trivially possible without +also interfering with files installed from debian packages. Probably the +easiest way to allow all users and all shells to find the 'tcpdump' binary is +to actually place a symbolic link in a directory that is already part of the +non-login shell's '$PATH'. Above example places such in '/usr/local/bin'. + +Verify that a non-login shell can find 'tcpdump': + +---- +su jenkins -c 'which tcpdump' +# should print: "/usr/local/bin/tcpdump" +---- + +WARNING: When logged in via SSH on your main unit, running 'tcpdump' to capture +packets may result in a feedback loop: SSH activity to send tcpdump's output to +your terminal is in turn is picked up in the tcpdump trace, and so forth. When +testing 'tcpdump' access, make sure to have proper filter expressions in place. + +===== Allow Core Files + +In case a binary run for the test crashes, a core file of the crash should be +written. This requires a limit rule. Create a file with the required rule: + +---- +sudo -s +echo "@osmo-gsm-tester - core unlimited" > /etc/security/limits.d/osmo-gsm-tester_allow-core.conf +---- + +Re-login the user to make these changes take effect. + +Set the *kernel.core_pattern* sysctl to *core* (usually the default). For each +binary run by osmo-gsm-tester, a core file will then appear in the same dir that +contains stdout and stderr for that process (because this dir is set as CWD). + +---- +sysctl -w kernel.core_pattern=core +---- + +===== Allow Realtime Priority + +Certain binaries should be run with real-time priority, like 'osmo-bts-trx'. +Add this permission on the main unit: + +---- +sudo -s +echo "@osmo-gsm-tester - rtprio 99" > /etc/security/limits.d/osmo-gsm-tester_allow-rtprio.conf +---- + +Re-login the user to make these changes take effect. + +[[user_config_uhd]] +===== UHD + +Grant permission to use the UHD driver to run USRP devices for osmo-bts-trx, by +adding the jenkins user to the 'usrp' group: + +---- +gpasswd -a jenkins usrp +---- + +===== Allow CAP_NET_RAW capability + +Certain binaries require 'CAP_NET_RAW' to be set, like 'osmo-bts-octphy' as it +uses a 'AF_PACKET' socket. + +To be able to set the following capability without being root, osmo-gsm-tester +uses sudo to gain permissions to set the capability. + +This is the script that osmo-gsm-tester expects on the main unit: + +---- +echo /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh <<EOF +#!/bin/bash +/sbin/setcap cap_net_raw+ep $1 +EOF +chmod +x /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh +---- + +Now, again on the main unit, we need to provide sudo access to this script for +osmo-gsm-tester: + +---- +echo "%osmo-gsm-tester ALL=(root) NOPASSWD: /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh" > /etc/sudoers.d/osmo-gsm-tester_setcap_net_raw +chmod 0440 /etc/sudoers.d/osmo-gsm-tester_setcap_net_raw +---- + +The script file name 'osmo-gsm-tester_setcap_net_raw.sh' is important, as +osmo-gsm-tester expects to find a script with this name in '$PATH' at run time. + + +==== Log Rotation + +To avoid clogging up /var/log, it makes sense to choose a sane maximum log size: + +---- +echo maxsize 10M > /etc/logrotate.d/maxsize +---- + +==== Install Scripts + +IMPORTANT: When using the jenkins build slave as configured above, *there is no +need to install the osmo-gsm-tester sources on the main unit*. The jenkins job +will do so implicitly by checking out the latest osmo-gsm-tester sources in the +workspace for every run. If you're using only the jenkins build slave, you may +skip this section. + +If you prefer to use a fixed installation of the osmo-gsm-tester sources +instead of the jenkins workspace, you can: + +. From the run job configured above, remove the line that says ++ +---- +PATH="$PWD/osmo-gsm-tester/src:$PATH" \ +---- ++ +so that this uses a system wide installation instead. + +. Install the sources e.g. in '/usr/local/src' as indicated below. + +On the main unit, to install the latest in '/usr/local/src': + +---- +apt-get install git +mkdir -p /usr/local/src +cd /usr/local/src +git clone git://git.osmocom.org/osmo-gsm-tester +---- + +To allow all users to run 'osmo-gsm-tester.py', from login as well as non-login +shells, the easiest solution is to place a symlink in '/usr/local/bin': + +---- +ln -s /usr/local/src/osmo-gsm-tester/src/osmo-gsm-tester.py /usr/local/bin/ +---- + +(See also the tip in <<install_capture_packets>> for a more detailed +explanation.) + +The example configuration provided in the source is suitable for running as-is, +*if* your hardware setup matches (you could technically use that directly by a +symlink e.g. from '/usr/local/etc/osmo-gsm-tester' to the 'example' dir). If in +doubt, rather copy the example, point 'paths.conf' at the 'suites' dir, and +adjust your own configuration as needed. For example: + +---- +cd /etc +cp -R /usr/local/src/osmo-gsm-tester/example osmo-gsm-tester +sed -i 's#\.\./suites#/usr/local/src/osmo-gsm-tester/suites#' osmo-gsm-tester/paths.conf +---- + +NOTE: The configuration will be looked up in various places, see +<<config_paths>>. + + +== Hardware Choice and Configuration + +=== SysmoBTS + +To use the SysmoBTS in the osmo-gsm-tester, the following systemd services must +be disabled: + +---- +systemctl mask osmo-nitb osmo-bts-sysmo osmo-pcu sysmobts-mgr +---- + +This stops the stock setup keeping the BTS in operation and hence allows the +osmo-gsm-tester to install and launch its own versions of the SysmoBTS +software. + +==== IP Address + +To ensure that the SysmoBTS is always reachable at a fixed known IP address, +configure the eth0 to use a static IP address: + +Adjust '/etc/network/interfaces' and replace the line + +---- +iface eth0 inet dhcp +---- + +with + +---- +iface eth0 inet static + address 10.42.42.114 + netmask 255.255.255.0 + gateway 10.42.42.1 +---- + +You may set the name server in '/etc/resolve.conf' (most likely to the IP of +the gateway), but this is not really needed by the osmo-gsm-tester. + +==== Allow Core Files + +In case a binary run for the test crashes, a core file of the crash should be +written. This requires a limits rule. Append a line to /etc/limits like: + +---- +ssh root@10.42.42.114 +echo "* C16384" >> /etc/limits +---- + +==== Reboot + +Reboot the BTS and make sure that the IP address for eth0 is now indeed +10.42.42.114, and that no osmo* programs are running. + +---- +ip a +ps w | grep osmo +---- + +==== SSH Access + +Make sure that the jenkins user on the main unit is able to login on the +sysmoBTS, possibly erasing outdated host keys after a new rootfs was loaded: + +On the main unit, for example do: + +---- +su - jenkins +ssh root@10.42.42.114 +---- + +Fix any problems until you get a login on the sysmoBTS. + + +[[hardware_modems]] +=== Modems + +TODO: describe modem choices and how to run ofono + +[[hardware_trx]] +=== osmo-bts-trx + +TODO: describe B200 family diff --git a/doc/manuals/chapters/intro.adoc b/doc/manuals/chapters/intro.adoc new file mode 100644 index 0000000..14daba4 --- /dev/null +++ b/doc/manuals/chapters/intro.adoc @@ -0,0 +1,400 @@ +== Introduction with Examples + +The osmo-gsm-tester is software to run automated tests of real GSM hardware, +foremost to verify that ongoing Osmocom software development continues to work +with various BTS models, while being flexibly configurable and extendable. + +A 'main unit' (general purpose computer) is connected via ethernet and/or USB to +any number of BTS models and to any number of GSM modems via USB. The modems +and BTS instances' RF transceivers are typically wired directly to each other +via RF distribution chambers to bypass the air medium and avoid disturbing real +production cellular networks. Furthermore, the setup may include adjustable RF +attenuators to model various distances between modems and base stations. + +The osmo-gsm-tester software runs on the main unit to orchestrate the various +GSM hardware and run predefined test scripts. It typically receives binary +packages from a jenkins build service. It then automatically configures and +launches an Osmocom core network on the main unit and sets up and runs BTS +models as well as modems to form a complete ad-hoc GSM network. On this setup, +predefined test suites, combined with various scenario definitions, are run to +verify stability of the system. + +The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono +daemon to control the modems connected via USB. BTS software is either run +directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH +(e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for +ip.access nanoBTS). + +.Typical osmo-gsm-tester setup +[graphviz] +---- +digraph G { + rankdir=LR; + jenkins + subgraph cluster_gsm_hardware { + label = "GSM Hardware"; + style=dotted + + modem0 [shape=box label="Modems..."] + modem1 [shape=box label="Modems..."] + osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box] + B200 [label="Ettus B200" shape=box] + sysmoCell5K [label="sysmocom sysmoCell5000" shape=box] + octphy [label="Octasic octphy BTS" shape=box] + nanoBTS [label="ip.access nanoBTS" shape=box] + rf_distribution [label="RF distribution"] + + {modem0 modem1 osmo_bts_sysmo B200 octphy nanoBTS sysmoCell5K}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"] + } + subgraph cluster_main_unit { + label = "Main Unit" + osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"] + subgraph { + rank=same + ofono [label="ofono daemon"] + osmo_trx [label="osmo-trx"] + osmo_bts_trx [label="osmo-bts-trx"] + osmo_bts_octphy [label="osmo-bts-octphy"] + OsmoNITB [label="BSC + Core Network\n(Osmo{NITB,MSC,BSC,HLR,...})"] + } + } + + + jenkins->osmo_gsm_tester [label="trial\n(binaries)"] + osmo_gsm_tester->jenkins [label="results"] + ofono->{modem0 modem1} [label="USB"] + osmo_gsm_tester->{OsmoNITB osmo_bts_trx osmo_bts_octphy} + osmo_gsm_tester->osmo_bts_sysmo [taillabel="SSH"] + osmo_gsm_tester->ofono [taillabel="DBus"] + osmo_trx->B200 [label="USB"] + osmo_bts_trx->{osmo_trx sysmoCell5K} [dir=both label="UDP"] + osmo_bts_octphy->octphy [label="raw eth"] + {osmo_bts_sysmo nanoBTS}->OsmoNITB [label="IP"] + {B200 octphy}->OsmoNITB [label="eth" style=invis] + {osmo_bts_trx osmo_bts_octphy}->OsmoNITB +} +---- + +.Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface) +[graphviz] +---- +digraph G { + rankdir=TB; + + suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800"] + + subgraph { + rank=same; + suite + scenarios + } + + subgraph cluster_scenarios { + label = "Scenarios"; + u_sysmoBTS [label="Scenario: sysmo\nbts: type: osmo-bts-sysmo"] + u_trx [label="Scenario: trx\nbts: type: osmo-bts-trx"] + u_arfcn [label="Scenario: band1800\narfcn: band: GSM-1800"] + } + + subgraph cluster_suite { + label = "Suite: sms"; + requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"] + subgraph cluster_tests { + label = "Test Scripts (py)"; + mo_mt_sms + etc + } + } + + subgraph cluster_resources { + label = "Resources"; + rankdir=TB; + nitb_addr1 [label="NITB interface addr\n10.42.42.1"] + nitb_addr2 [label="NITB interface addr\n10.42.42.2"] + Modem0 + Modem1 + Modem2 + sysmoBTS [label="osmo-bts-sysmo"] + osmo_bts_trx [label="osmo-bts-trx"] + arfcn1 [label="arfcn: 512\nband: GSM-1800"] + arfcn2 [label="arfcn: 540\nband: GSM-1900"] + + arfcn1->arfcn2 [style=invis] + nitb_addr1->nitb_addr2 [style=invis] + Modem0 -> Modem1 -> Modem2 [style=invis] + sysmoBTS -> osmo_bts_trx [style=invis] + } + + suite_scenarios -> {suite scenarios} + scenarios -> { u_arfcn u_sysmoBTS } + + suite -> requires + requires -> Modem0 + requires -> Modem1 + requires -> sysmoBTS + requires -> arfcn1 + requires -> nitb_addr1 + + { u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"] +} +---- + +.Example of a "trial" containing binaries built by a jenkins +[graphviz] +---- +digraph G { + subgraph cluster_trial { + label = "Trial (binaries)" + sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"] + trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"] + nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"] + checksums [label="checksums.md5"] + + checksums -> {sysmo trx nitb} + } +} +---- + +=== Typical Test Script + +A typical single test script (part of a suite) may look like this: + +---- +#!/usr/bin/env python3 +from osmo_gsm_tester.testenv import * + +hlr = suite.hlr() +bts = suite.bts() +mgcpgw = suite.mgcpgw(bts_ip=bts.remote_addr()) +msc = suite.msc(hlr, mgcpgw) +bsc = suite.bsc(msc) +stp = suite.stp() +ms_mo = suite.modem() +ms_mt = suite.modem() + +hlr.start() +stp.start() +msc.start() +mgcpgw.start() + +bsc.bts_add(bts) +bsc.start() + +bts.start() + +hlr.subscriber_add(ms_mo) +hlr.subscriber_add(ms_mt) + +ms_mo.connect(msc.mcc_mnc()) +ms_mt.connect(msc.mcc_mnc()) + +ms_mo.log_info() +ms_mt.log_info() + +print('waiting for modems to attach...') +wait(ms_mo.is_connected, msc.mcc_mnc()) +wait(ms_mt.is_connected, msc.mcc_mnc()) +wait(msc.subscriber_attached, ms_mo, ms_mt) + +sms = ms_mo.sms_send(ms_mt) +wait(ms_mt.sms_was_received, sms) +---- + +=== Resource Resolution + +- A global configuration 'resources.conf' defines which hardware is connected to the + osmo-gsm-tester main unit. +- Each suite contains a number of test scripts. The amount of resources a test + may use is defined by the test suite's 'suite.conf'. +- Which specific modems, BTS models, NITB IP addresses etc. are made available + to a test run is typically determined by 'suite.conf' and a combination of scenario + configurations -- or picked automatically if not. + +[[resources_conf_example]] +=== Typical 'resources.conf' + +A global configuration of hardware may look like below; for details, see +<<resources_conf>>. + +---- +ip_address: +- addr: 10.42.42.2 +- addr: 10.42.42.3 +- addr: 10.42.42.4 +- addr: 10.42.42.5 +- addr: 10.42.42.6 + +bts: +- label: sysmoBTS 1002 + type: osmo-bts-sysmo + ipa_unit_id: 1 + addr: 10.42.42.114 + band: GSM-1800 + ciphers: + - a5_0 + - a5_1 + - a5_3 + +- label: Ettus B200 + type: osmo-bts-trx + ipa_unit_id: 6 + addr: 10.42.42.50 + band: GSM-1800 + launch_trx: true + ciphers: + - a5_0 + - a5_1 + +- label: sysmoCell 5000 + type: osmo-bts-trx + ipa_unit_id: 7 + addr: 10.42.42.51 + band: GSM-1800 + trx_remote_ip: 10.42.42.112 + ciphers: + - a5_0 + - a5_1 + +- label: OCTBTS 3500 + type: osmo-bts-octphy + ipa_unit_id: 8 + addr: 10.42.42.52 + band: GSM-1800 + trx_list: + - hw_addr: 00:0c:90:2e:80:1e + net_device: eth1 + - hw_addr: 00:0c:90:2e:87:52 + net_device: eth1 + +arfcn: + - arfcn: 512 + band: GSM-1800 + - arfcn: 514 + band: GSM-1800 + - arfcn: 516 + band: GSM-1800 + - arfcn: 546 + band: GSM-1900 + - arfcn: 548 + band: GSM-1900 + +modem: +- label: sierra_1 + path: '/sierra_1' + imsi: '901700000009031' + ki: '80A37E6FDEA931EAC92FFA5F671EFEAD' + auth_algo: 'xor' + ciphers: + - a5_0 + - a5_1 + features: + - 'sms' + - 'voice' + +- label: gobi_0 + path: '/gobi_0' + imsi: '901700000009030' + ki: 'BB70807226393CDBAC8DD3439FF54252' + auth_algo: 'xor' + ciphers: + - a5_0 + - a5_1 + features: + - 'sms' +---- + +=== Typical 'suites/*/suite.conf' + +The configuration that reserves a number of resources for a test suite may look +like this: + +---- +resources: + ip_address: + - times: 1 + bts: + - times: 1 + modem: + - times: 2 + features: + - sms +---- + +It may also request e.g. specific BTS models, but this is typically left to +scenario configurations. + +=== Typical 'scenarios/*.conf' + +For a suite as above run as-is, any available resources are picked. This may be +combined with any number of scenario definitions to constrain which specific +resources should be used, e.g.: + +---- +resources: + bts: + - type: osmo-bts-sysmo +---- + +Which 'ip_address' or 'modem' is used in particular doesn't really matter, so +it can be left up to the osmo-gsm-tester to pick these automatically. + +Any number of such scenario configurations can be combined in the form +'<suite_name>:<scenario>+<scenario>+...', e.g. 'my_suite:sysmo+tch_f+amr'. + +=== Typical Invocations + +Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for +the Osmocom core network as well as for the Osmocom based BTS models. To create +such a set of binaries, see <<trials>>. + +Examples for launching test trials: + +- Run the default suites (see <<default_suites>>) on a given set of binaries: + +---- +osmo-gsm-tester.py path/to/my-trial +---- + +- Run an explicit choice of 'suite:scenario' combinations: + +---- +osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts +---- + +- Run one 'suite:scenario' combination, setting log level to 'debug' and + enabling logging of full python tracebacks, and also only run just the + 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: + +---- +osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt +---- + +A test script may also be run step-by-step in a python debugger, see +<<debugging>>. + +=== Resource Reservation for Concurrent Trials + +While a test suite runs, the used resources are noted in a global state +directory in a reserved-resources file. This way, any number of trials may be +run consecutively without resource conflicts. Any test trial will only use +resources that are currently not reserved by any other test suite. The +reservation state is human readable. + +The global state directory is protected by a file lock to allow access by +separate processes. + +Also, the binaries from a trial are never installed system-wide, but are run +with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that +several trials can run consecutively without conflicting binary versions. For +some specific binaries which require extra permissions (such as osmo-bts-octphy +requiring 'CAP_NET_RAW'), 'patchelf' program is used to modify the binary +'RPATH' field instead because the OS dynamic linker skips 'LD_LIBRARY_PATH' for +binaries with special permissions. + +Once a test suite run is complete, all its reserved resources are torn down (if +the test scripts have not done so already), and the reservations are released +automatically. + +If required resources are unavailable, the test trial fails. For consecutive +test trials, a test run needs to either wait for resources to become available, +or test suites need to be scheduled to make sense. (*<- TODO*) diff --git a/doc/manuals/chapters/test_api.adoc b/doc/manuals/chapters/test_api.adoc new file mode 100644 index 0000000..f541231 --- /dev/null +++ b/doc/manuals/chapters/test_api.adoc @@ -0,0 +1,4 @@ +== Test API + +*TODO* (in the meantime, look at src/osmo_gsm_tester/test.py, as well as +suite.py, which calls the test's setup() function to get an idea) diff --git a/doc/manuals/chapters/trial.adoc b/doc/manuals/chapters/trial.adoc new file mode 100644 index 0000000..bc9fe05 --- /dev/null +++ b/doc/manuals/chapters/trial.adoc @@ -0,0 +1,31 @@ +[[trials]] +== Trial: Binaries to be Tested + +A trial is a set of pre-built binaries to be tested. They are typically built +by jenkins using the build scripts found in osmo-gsm-tester's source in the +'contrib/' dir, see <<install_add_jenkins_slave>>. + +A trial comes in the form of a directory containing a number of '*.tgz' tar +archives as well as a 'checksums.md5' file to verify the tar archives' +integrity. + +When the osmo-gsm-tester is invoked to run on such a trial directory, it will +create a sub directory named 'inst' and unpack the tar archives into it. + +For each test run on this trial, a new subdirectory in the trial dir is +created, named in the form of 'run.<timestamp>'. A symbolic link 'last-run' +will point at the most recently created run dir. This run dir will accumulate: + +* the rendered configuration files used to run the binaries +* stdout and stderr outputs of the binaries +* pcap files for processes doing relevant network communication +* a test log +* jenkins parsable XML (Junit) reports + +The script in 'contrib/jenkins-run.sh' takes care of related tasks such as + +* creating the dir structure, +* generating md5 sums for the various tar.gz containing software builds to be tested, +* cleaning up after the build, +* saving extra logs such as journalctl output from ofonod, +* generating a final .tar.gz file with all the logs and reports. diff --git a/doc/manuals/osmo-gsm-tester-manual-docinfo.xml b/doc/manuals/osmo-gsm-tester-manual-docinfo.xml new file mode 100644 index 0000000..923b8ad --- /dev/null +++ b/doc/manuals/osmo-gsm-tester-manual-docinfo.xml @@ -0,0 +1,47 @@ +<revhistory> + <revision> + <revnumber>1</revnumber> + <date>April 13, 2017</date> + <authorinitials>NH</authorinitials> + <revremark> + Initial version. + </revremark> + </revision> +</revhistory> + +<authorgroup> + <author> + <firstname>Neels</firstname> + <surname>Hofmeyr</surname> + <email>nhofmeyr@sysmocom.de</email> + <authorinitials>NH</authorinitials> + <affiliation> + <shortaffil>sysmocom</shortaffil> + <orgname>sysmocom - s.f.m.c. GmbH</orgname> + <jobtitle>Senior Developer</jobtitle> + </affiliation> + </author> +</authorgroup> + +<copyright> + <year>2017</year> + <holder>sysmocom - s.f.m.c. GmbH</holder> +</copyright> + +<legalnotice> + <para> + Permission is granted to copy, distribute and/or modify this + document under the terms of the GNU Free Documentation License, + Version 1.3 or any later version published by the Free Software + Foundation; with the Invariant Sections being just 'Foreword', + 'Acknowledgements' and 'Preface', with no Front-Cover Texts, + and no Back-Cover Texts. A copy of the license is included in + the section entitled "GNU Free Documentation License". + </para> + <para> + The Asciidoc source code of this manual can be found at + <ulink url="http://git.osmocom.org/osmo-gsm-manuals/"> + http://git.osmocom.org/osmo-gsm-manuals/ + </ulink> + </para> +</legalnotice> diff --git a/doc/manuals/osmo-gsm-tester-manual.adoc b/doc/manuals/osmo-gsm-tester-manual.adoc new file mode 100644 index 0000000..b728384 --- /dev/null +++ b/doc/manuals/osmo-gsm-tester-manual.adoc @@ -0,0 +1,20 @@ +Osmo-GSM-Tester Manual +====================== +Neels Hofmeyr <nhofmeyr@sysmocom.de> + +== WARNING: Work in Progress + +*NOTE: The osmo-gsm-tester is still in pre-alpha stage: some parts are still +incomplete, and details will still change and move around.* + +include::chapters/intro.adoc[] + +include::chapters/install.adoc[] + +include::chapters/config.adoc[] + +include::chapters/trial.adoc[] + +include::chapters/test_api.adoc[] + +include::chapters/debugging.adoc[] |