From 7e0b2ddfb8f835f207758805e2dc4484e3633b2c Mon Sep 17 00:00:00 2001 From: Pau Espin Pedrol Date: Tue, 10 Mar 2020 11:46:39 +0100 Subject: doc/manual: Refactor, rewrite, improve and update most of the User Manual * Some TODOs are added as comments which actually require code changes. These are details which showed up as incongruences or missing bits while writing the documentation for them. * Some sections are introduced but still waiting to be writen soon: ** Debugging section ** Docker Setup section ** Ansible Setup section ** Troubleshooting (add jenkins red cross button sending kill -9) ** resources.conf attribute list needs to be converted to a table * Device related setup needs to be updated and extended * Parametrized scenarios need to be documented * 4G resources documentation needs to be added. Change-Id: Ifc2a3c74d45336cc988b76c0ff68a85311e4dd40 --- doc/manuals/chapters/ansible.adoc | 6 + doc/manuals/chapters/config.adoc | 546 +++++++++++++++++-------- doc/manuals/chapters/docker.adoc | 6 + doc/manuals/chapters/install.adoc | 431 ++++++++++--------- doc/manuals/chapters/install_device.adoc | 82 ++++ doc/manuals/chapters/intro.adoc | 418 +++---------------- doc/manuals/chapters/resource_pool.adoc | 107 +++++ doc/manuals/chapters/trial.adoc | 44 +- doc/manuals/chapters/troubleshooting.adoc | 15 + doc/manuals/osmo-gsm-tester-manual-docinfo.xml | 13 +- doc/manuals/osmo-gsm-tester-manual.adoc | 27 +- 11 files changed, 929 insertions(+), 766 deletions(-) create mode 100644 doc/manuals/chapters/ansible.adoc create mode 100644 doc/manuals/chapters/docker.adoc create mode 100644 doc/manuals/chapters/install_device.adoc create mode 100644 doc/manuals/chapters/resource_pool.adoc create mode 100644 doc/manuals/chapters/troubleshooting.adoc (limited to 'doc') diff --git a/doc/manuals/chapters/ansible.adoc b/doc/manuals/chapters/ansible.adoc new file mode 100644 index 0000000..b672528 --- /dev/null +++ b/doc/manuals/chapters/ansible.adoc @@ -0,0 +1,6 @@ +[[ansible]] +== Ansible Setup + +Available in osmocom's osmo-ci.git subdirectory 'ansible/', see there 'gsm-tester/README.md'. + +//TODO: Explain more where to find, how to build, how to use. diff --git a/doc/manuals/chapters/config.adoc b/doc/manuals/chapters/config.adoc index 7e250e0..bb0cec2 100644 --- a/doc/manuals/chapters/config.adoc +++ b/doc/manuals/chapters/config.adoc @@ -1,5 +1,184 @@ == Configuration +=== Schemas + +All configuration attributes in {app-name} are stored and provided as YAML +files, which are handled internally mostly as sets of dictionaries, lists and +scalars. Each of these configurations have a known format, which is called +'schema'. Each provided configuration is validated against its 'schema' at parse +time. Hence, 'schemas' can be seen as a namespace containing a structured tree +of configuration attributes. Each attribute has a schema type assigned which +constrains the type of value it can hold. + +There are several well-known schemas used across {app-name}, and they are +described in following sub-sections. + +[[schema_resources]] +==== Schema 'resources' + +This schema defines all the attributes which can be assigned to +a _resource_, and it is used to validate the <> +file. Hence, the <> contains a list of elements +for each resource type. + +It is important to understand that the content in this schema refers to a list of +resources for each resource class. Since a list is ordered by definition, it +clearly identifies specific resources by order. This is important when applying +filters or modifiers, since they are applied per-resource in the list. One can +for instance apply attribute A to first resource of class C, while not applying +it or applying another attribute B to second resources of the same class. As a +result, complex forms can be used to filter and modify a list of resources +required by a testsuite. + +On the other hand, it's also important to note that lists for simple or scalar +types are currently being treated as unordered sets, which mean combination of +filters or modifiers apply differently. In the future, it may be possible to +have both behaviors for scalar/simple types by using also the YAML 'set' type in +{app-handle}. + +//TODO: update this list and use a table for each resource type +These kinds of resources and their attributes are known: + +'ip_address':: + List of IP addresses to run osmo-nitb instances on. The main unit + typically has a limited number of such IP addresses configured, which + the connected BTS models can see on their network. + 'addr'::: + IPv4 address of the local interface. + +'bts':: + List of available BTS hardware. + 'label'::: + human readable label for your own reference + 'type'::: + which way to launch this BTS, one of + - 'osmo-bts-sysmo' + - 'osmo-bts-trx' + - 'osmo-bts-octphy' + - 'ipa-nanobts' + 'ipa_unit_id'::: + ip.access unit id to be used by the BTS, written into BTS and BSC config. + 'addr'::: + Remote IP address of the BTS for BTS like sysmoBTS, and local IP address + to bind to for locally run BTS such as osmo-bts-trx. + 'band'::: + GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of: + - 'GSM-1800' + - 'GSM-1900' + - (*TODO*: more bands) + 'trx_list'::: + Specific TRX configurations for this BTS. There should be as many of + these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without + special configuration for them.) + 'hw_addr':::: + Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66', + only used for osmo-bts-octphy. (*TODO*: and nanobts??) + 'net_device':::: + Local network device to reach the TRX's 'hw_addr' at, only used for + osmo-bts-octphy. Example: 'eth0'. + 'nominal_power':::: + Nominal power to be used by the TRX. + 'max_power_red':::: + Max power reduction to apply to the nominal power of the TRX. +'arfcn':: + List of ARFCNs to use for running BTSes, which defines the actual RF + frequency bands used. + 'arfcn'::: + ARFCN number, see e.g. + https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number + (note that the resource type 'arfcn' contains an item trait also named + 'arfcn'). + 'band'::: + GSM band name to use this ARFCN for, same as for 'bts:band' above. + +'modem':: + List of modems reachable via ofono and information on the inserted SIM + card. (Note: the MSISDN is allocated dynamically in test scripts). + 'label'::: + Human readable label for your own reference, which also appears in logs. + 'path'::: + Ofono's path for this modem, like '/modemkind_99'. + 'imsi'::: + IMSI of the inserted SIM card, like '"123456789012345"'. + 'ki'::: + 16 byte authentication/encryption KI of the inserted SIM card, in + hexadecimal notation (32 characters) like + + '"00112233445566778899aabbccddeeff"'. + 'auth_algo'::: + Authentication algorithm to be used with the SIM card. One of: + - 'none' + - 'xor' + - 'comp128v1' + 'ciphers'::: + List of ciphers that this modem supports, used to match + requirements in suites or scenarios. Any combination of: + - 'a5_0' + - 'a5_1' + - 'a5_2' + - 'a5_3' + - 'a5_4' + - 'a5_5' + - 'a5_6' + - 'a5_7' + 'features'::: + List of features that this modem supports, used to match requirements in + suites or scenarios. Any combination of: + - 'sms' + - 'gprs' + - 'voice' + - 'ussd' + +[[schema_want]] +==== Schema 'want' + +This schema is basically the same as the <> one, but +with an extra 'times' attribute for each resource item. All 'times' attributes +are expanded before matching. For example, if a 'suite.conf' requests two BTS, +one may enforce that both BTS should be of type 'osmo-bts-sysmo' in these ways: + +---- +resources: + bts: + - type: osmo-bts-sysmo + - type: osmo-bts-sysmo +---- + +or alternatively, + +---- +resources: + bts: + - times: 2 + type: osmo-bts-sysmo +---- + +[[schema_conf]] +==== Schema 'conf' + +This schema is used by <> and <> files. It contains 3 main element sections::: +[[schema_conf_sec_resources]] +- Section 'resources': Contains a set of elements validated with <> + schema. In <> it is used to construct the list of + requested resources. In <>, it is used to inject + attributes to the initial <> _resources_ section and + hence further restrain it. +[[schema_conf_sec_modifiers]] +- Section 'modifiers': Both in <> and + <>, values presented in here are injected into + the content of the <> after + _resource_ allocation, hereby overwriting attributes passed to the object + class instance managing the specific _resource_ (matches by resource type and + list position). Since it is combined with the content of + <>, it is clear that the + <> is used to validate this content. +[[schema_conf_sec_config]] +- Section 'config': Contains configuration attributes for {app-name} classes which are + not _resources_, and hence cannot be configured with <>. + They can overwrite values provided in the <> file. + +//TODO: defaults.timeout should be change in code to be config.test_timeout or similar +//TODO: 'config' should be split into its own schema and validate defaults.conf + [[config_paths]] === Config Paths @@ -16,29 +195,15 @@ The config location can also be set by an environment variable The osmo-gsm-tester expects to find the following configuration files in a configuration directory: -- 'paths.conf' -- 'resources.conf' -- 'default-suites.conf' (optional) -- 'defaults.conf' (optional) +- <> +- <> +- <> (optional) +- <> (optional) These are described in detail in the following sections. -=== Format: YAML, and its Drawbacks - -The general configuration format used is YAML. The stock python YAML parser -does have several drawbacks: too many complex possibilities and alternative -ways of formatting a configuration, but at the time of writing seems to be the -only widely used configuration format that offers a simple and human readable -formatting as well as nested structuring. It is recommended to use only the -exact YAML subset seen in this manual in case the osmo-gsm-tester should move -to a less bloated parser in the future. - -Careful: if a configuration item consists of digits and starts with a zero, you -need to quote it, or it may be interpreted as an octal notation integer! Please -avoid using the octal notation on purpose, it is not provided intentionally. - [[paths_conf]] -=== 'paths.conf' +==== 'paths.conf' The 'paths.conf' file defines where to store the global state (of reserved resources) and where to find suite and scenario definitions. @@ -46,8 +211,10 @@ resources) and where to find suite and scenario definitions. Any relative paths found in a 'paths.conf' file are interpreted as relative to the directory of that 'paths.conf' file. -Example: +There's not yet any well-known schema to validate this file contents since it +has only 3 attributes. +.Sample paths.conf file: ---- state_dir: '/var/tmp/osmo-gsm-tester/state' suites_dir: '/usr/local/src/osmo-gsm-tester/suites' @@ -55,69 +222,143 @@ scenarios_dir: './scenarios' ---- [[state_dir]] -==== 'state_dir' +===== 'state_dir' It contains global or system-wide state for osmo-gsm-tester. In a typical state dir you can find the following files: -'last_used_msisdn.state':: - Contains last used msisdn number, which is automatically increased every - time osmo-gsm-tester needs to assign a new subscriber in a test. -'lock':: - Lock file used to implement a mutual exclusion zone around the - 'reserved_resources.state' file. +'last_used_*.state':: + Contains stateful content spanning accross {app-name} instances and + runs. For instance, 'last used msisdn number.state' is automatically + (and atomically) increased every time osmo-gsm-tester needs to assign a + new subscriber in a test, ensuring tests get unique msisdn numbers. 'reserved_resources.state':: File containing a set of reserved resources by any number of - osmo-gsm-tester instances. Each osmo-gsm-tester instance is responsible - to clear its resources from the list once it is done using them and are - no longer reserved. - -If you would like to set up several separate configurations (not typical), note -that the 'state_dir' is used to reserve resources, which only works when all -configurations that share resources also use the same 'state_dir'. + osmo-gsm-tester instances (aka pool of allocated resources). Each + osmo-gsm-tester instance is responsible to clear its resources from the + list once it is done using them and are no longer reserved. +'lock':: + Lock file used to implement a mutual exclusion zone around any state + files in the 'state_dir', to prevent race conditions between different + {app-name} instances running in parallel. This way, several concurrent users of osmo-gsm-tester (ie. several osmo-gsm-tester processes running in parallel) can run without interfering with each other (e.g. using same ARFCN, same IP or same ofono modem path). +If you would like to set up several separate configurations (not typical), note +that the 'state_dir' is used to reserve resources, which only works when all +configurations that share resources also use the same 'state_dir'. It's also +important to notice that since resources are stored in YAML dictionary form, if +same physical device is described differently in several +<> files (used by different {app-name} instances), +resource allocation may not work as expected. + [[suites_dir]] -==== 'suites_dir' +===== 'suites_dir' Suites contain a set of tests which are designed to be run together to test a set of features given a specific set of resources. As a result, resources are allocated per suite and not per test. Tests for a given suite are located in the form of '.py' python scripts in the -same directory where the 'suite.conf' lays. +same directory where the <> lays. -[[scenarios_dir]] -==== 'scenarios_dir' +Tests in the same testsuite willing to use some shared code can do so by putting +it eg. in '$suites_dir/$suitename/lib/testlib.py': +---- +#!/usr/bin/env python3 +from osmo_gsm_tester.testenv import * -This dir contains scenario configuration files. +def my_shared_code(foo): + return foo.bar() +---- -Scenarios define constraints to serve the resource requests of a 'suite.conf', -to select specific resources from the general resource pool specified in 'resources.conf'. +and then in the test itself use it this way: +---- +#!/usr/bin/env python3 +from osmo_gsm_tester.testenv import * -All 'times' attributes are expanded before matching. For example, if a 'suite.conf' -requests two BTS, we may enforce that both BTS should be of type 'osmo-bts-sysmo' in -these ways: +import testlib +suite.test_import_modules_register_for_cleanup(testlib) +from testlib import my_shared_code +bar = my_shared_code(foo) ---- -resources: - bts: - - type: osmo-bts-sysmo - - type: osmo-bts-sysmo + +.Sample 'suites_dir' directory tree: +---- +suites_dir/ +|-- suiteA +| |-- suite.conf +| '-- testA.py +|-- suiteB +| |-- testB.py +| |-- testC.py +| |-- lib +| | '-- testlib.py +| '-- suite.conf ---- -or alternatively, +[[suite_conf]] +===== 'suite.conf' +This file content is parsed using the <> schema. + +It provides +{app-name} with the base restrictions (later to be further filtered by +<> files) to apply when allocating resources. + +It can also override attributes for the allocated resources through the +<> section (to be further modified by +<> files later on). Similary it can do the same for +general configuration options (no per-resource) through the +<> section. + +.Sample 'suite.conf' file: ---- resources: + ip_address: + - times: 9 # msc, bsc, hlr, stp, mgw*2, sgsn, ggsn, iperf3srv bts: + - times: 1 + modem: - times: 2 - type: osmo-bts-sysmo + features: + - gprs + - voice + - times: 2 + features: + - gprs + +config: + bsc: + net: + codec_list: + - fr1 + +defaults: + timeout: 50s ---- +[[scenarios_dir]] +===== 'scenarios_dir' + +This dir contains scenario configuration files. + +.Sample 'scenarios_dir' directory tree: +---- +scenarios_dir/ +|-- scenarioA.conf +'-- scenarioB.conf +---- + +[[scenario_conf]] +===== 'scenario conf file' +Scenarios define further constraints to serve the resource requests of a +<>, ie. to select specific resources from the general +resource pool specified in <>. + If only one resource is specified in the scenario, then the resource allocator assumes the restriction is to be applied to the first resource and that remaining resources have no restrictions to be taken into consideration. @@ -132,8 +373,9 @@ resources: - type: osmo-bts-sysmo ---- -On the 'osmo_gsm_tester.py' command line and the 'default_suites.conf', any number of -such scenario configurations can be combined in the form: +On the 'osmo_gsm_tester.py' command line and the +<>, any number of such scenario +configurations can be combined in the form: ---- :[+[+...]] @@ -146,111 +388,18 @@ my_suite:sysmo+tch_f+amr ---- [[resources_conf]] -=== 'resources.conf' +==== 'resources.conf' +//TODO: update this section The 'resources.conf' file defines which hardware is connected to the main unit, as well as which limited configuration items (like IP addresses or ARFCNs) should be used. -These resources are allocated dynamically and are not configured explicitly: - -- MSISDN: phone numbers are dealt out to test scripts in sequence on request. - -A 'resources.conf' is structured as a list of items for each resource type, -where each item has one or more settings -- for an example, see +A 'resources.conf' is validated by the <>. +That means it is structured as a list of items for each resource type, where +each item has one or more attributes -- for an example, see <>. -These kinds of resource are known: - -'ip_address':: - List of IP addresses to run osmo-nitb instances on. The main unit - typically has a limited number of such IP addresses configured, which - the connected BTS models can see on their network. - 'addr'::: - IPv4 address of the local interface. - -'bts':: - List of available BTS hardware. - 'label'::: - human readable label for your own reference - 'type'::: - which way to launch this BTS, one of - - 'osmo-bts-sysmo' - - 'osmo-bts-trx' - - 'osmo-bts-octphy' - - 'ipa-nanobts' - 'ipa_unit_id'::: - ip.access unit id to be used by the BTS, written into BTS and BSC config. - 'addr'::: - Remote IP address of the BTS for BTS like sysmoBTS, and local IP address - to bind to for locally run BTS such as osmo-bts-trx. - 'band'::: - GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of: - - 'GSM-1800' - - 'GSM-1900' - - (*TODO*: more bands) - 'trx_list'::: - Specific TRX configurations for this BTS. There should be as many of - these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without - special configuration for them.) - 'hw_addr':::: - Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66', - only used for osmo-bts-octphy. (*TODO*: and nanobts??) - 'net_device':::: - Local network device to reach the TRX's 'hw_addr' at, only used for - osmo-bts-octphy. Example: 'eth0'. - 'nominal_power':::: - Nominal power to be used by the TRX. - 'max_power_red':::: - Max power reduction to apply to the nominal power of the TRX. -'arfcn':: - List of ARFCNs to use for running BTSes, which defines the actual RF - frequency bands used. - 'arfcn'::: - ARFCN number, see e.g. - https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number - (note that the resource type 'arfcn' contains an item trait also named - 'arfcn'). - 'band'::: - GSM band name to use this ARFCN for, same as for 'bts:band' above. - -'modem':: - List of modems reachable via ofono and information on the inserted SIM - card. (Note: the MSISDN is allocated dynamically in test scripts). - 'label'::: - Human readable label for your own reference, which also appears in logs. - 'path'::: - Ofono's path for this modem, like '/modemkind_99'. - 'imsi'::: - IMSI of the inserted SIM card, like '"123456789012345"'. - 'ki'::: - 16 byte authentication/encryption KI of the inserted SIM card, in - hexadecimal notation (32 characters) like + - '"00112233445566778899aabbccddeeff"'. - 'auth_algo'::: - Authentication algorithm to be used with the SIM card. One of: - - 'none' - - 'xor' - - 'comp128v1' - 'ciphers'::: - List of ciphers that this modem supports, used to match - requirements in suites or scenarios. Any combination of: - - 'a5_0' - - 'a5_1' - - 'a5_2' - - 'a5_3' - - 'a5_4' - - 'a5_5' - - 'a5_6' - - 'a5_7' - 'features'::: - List of features that this modem supports, used to match requirements in - suites or scenarios. Any combination of: - - 'sms' - - 'gprs' - - 'voice' - - 'ussd' - Side note: at first sight it might make sense to the reader to rather structure e.g. the 'ip_address' or 'arfcn' configuration as + '"arfcn: GSM-1800: [512, 514, ...]"', + @@ -262,25 +411,22 @@ that is repeated numerous times. No special notation for these cases is available (yet). [[default_suites]] -=== 'default-suites.conf' (optional) +==== 'default-suites.conf' (optional) -The 'default-suites.conf' file contains a list of 'suite:scenario+scenario+...' +The 'default-suites.conf' file contains a YAML list of 'suite:scenario+scenario+...' combination strings as defined by the 'osmo-gsm-tester.py -s' commandline option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the '-s' arguments are taken from this file instead. Each of these suite + scenario combinations is run in sequence. -A suite name must match the name of a directory in the 'suites_dir' as defined -by 'paths.conf'. +A suite name must match the name of a directory in the +<> as defined by <>. A scenario name must match the name of a configuration file in the -'scenarios_dir' as defined by 'paths.conf' (optionally without the '.conf' -suffix). - -For 'paths.conf', see <>. - -Example of a 'default-suites.conf' file: +<> as defined by <> +(optionally without the '.conf' suffix). +.Sample 'default-suites.conf' file: ---- - sms:sysmo - voice:sysmo+tch_f @@ -292,24 +438,28 @@ Example of a 'default-suites.conf' file: - voice:trx+dyn_ts ---- -=== 'defaults.conf' (optional) +==== 'defaults.conf' (optional) + +In {app-name} object instances requested by the test and created by the suite +relate to a specific allocated resource. That's not always the case, and even if +it the case the information stored in <> for that +resource may not contain tons of attributes which the object class needs to +manage the resource. + +For this exact reason, the 'defaults.conf' file exist. It contains a set of +default attributes and values (in YAML format) that object classes can use to +fill in the missing gaps, or to provide values which can easily be changed or +overwritten by <> or <> +files through modifiers. Each binary run by osmo-gsm-tester, e.g. 'osmo-nitb' or 'osmo-bts-sysmo', typically has a configuration file template that is populated with values for a -trial run. - -Some of these values are provided by the 'resources.conf' from the allocated -resource(s), but not all values can be populated this way: some osmo-nitb -configuration values like the network name, encryption algorithm or timeslot -channel combinations are in fact not resources (only the nitb's interface -address is). These additional settings may be provided by the scenario -configurations, but in case the provided scenarios leave some values unset, -they are taken from this 'defaults.conf'. (A 'scenario.conf' or a -'resources.conf' providing a similar setting always has precedence over the -values given in a 'defaults.conf'). +trial run. Hence, a <>, <> +or a <> providing a similar setting always has +precedence over the values given in a 'defaults.conf' -Example of a 'defaults.conf': +.Sample 'defaults.conf' file: ---- nitb: net: @@ -359,3 +509,53 @@ bsc_bts: - phys_chan_config: TCH/F_TCH/H_PDCH - phys_chan_config: TCH/F_TCH/H_PDCH ---- + +=== Example Setup + +{app-name} comes with an example official setup which is the one used to run +Osmocom's setup. There are actually two different available setups: a +production one and an RnD one, used to develop {app-name} itself. These two set +ups share mostly all configuration, main difference being the +<> file being used. + +All {app-name} related configuration for that environment is publicly available in 'osmo-gsm-tester.git' itself::: +- <>: Available Available under 'example/', with its paths already configured to take + required bits from inside the git repository. +- <>: Available under 'suites/' +- <>: Available under 'example/scenarios/' +- <>: Available under 'example/' as + 'resources.conf.prod' for Production setup and as 'resources.conf.rnd' for the + RnD setup. One must use a symbolic link to have it available as 'resources.conf'. + +//TODO: resources.conf file path should be modifiable through paths.conf! + +==== Typical Invocations + +Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for +the Osmocom core network as well as for the Osmocom based BTS models. To create +such a set of binaries, see <>. + +Examples for launching test trials: + +- Run the default suites (see <>) on a given set of binaries: + +---- +osmo-gsm-tester.py path/to/my-trial +---- + +- Run an explicit choice of 'suite:scenario' combinations: + +---- +osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts +---- + +- Run one 'suite:scenario1+scenario2' combination, setting log level to 'debug' + and enabling logging of full python tracebacks, and also only run just the + 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: + +---- +osmo-gsm-tester.py path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt +---- + +A test script may also be run step-by-step in a python debugger, see +<>. diff --git a/doc/manuals/chapters/docker.adoc b/doc/manuals/chapters/docker.adoc new file mode 100644 index 0000000..b7560e1 --- /dev/null +++ b/doc/manuals/chapters/docker.adoc @@ -0,0 +1,6 @@ +[[docker]] +== Docker Setup + +Available in osmocom's docker-playground.git subdirectory 'osmo-gsm-tester/'. + +//TODO: Explain more where to find, how to build, how to use. diff --git a/doc/manuals/chapters/install.adoc b/doc/manuals/chapters/install.adoc index d19f909..e062b8d 100644 --- a/doc/manuals/chapters/install.adoc +++ b/doc/manuals/chapters/install.adoc @@ -1,49 +1,11 @@ -== Installation on Main Unit +== {app-name} Installation -The main unit is a general purpose computer that orchestrates the tests. It -runs the core network components, controls the modems and so on. This can be -anything from a dedicated production rack unit to your laptop at home. - -This manual will assume that tests are run from a jenkins build slave, by a user -named 'jenkins' that belong to group 'osmo-gsm-tester'. The user configuration -for manual test runs and/or a different user name is identical, simply replace -the user name or group. - -=== Osmo-gsm-tester Dependencies - -On a Debian/Ubuntu based system, these commands install the packages needed to -run the osmo-gsm-tester.py code, i.e. install these on your main unit: +=== Trial Builder ----- -apt-get install \ - dbus \ - tcpdump \ - sqlite3 \ - python3 \ - python3-yaml \ - python3-mako \ - python3-gi \ - ofono \ - patchelf \ - sudo \ - libcap2-bin \ - python3-pip -pip3 install pydbus -pip3 install git+git://github.com/podshumok/python-smpplib.git ----- - -IMPORTANT: ofono may need to be installed from source to contain the most -recent fixes needed to operate your modems. This depends on the modem hardware -used and the tests run. Please see <>. - -To run osmo-bts-trx with a USRP attached, you may need to install a UHD driver. -Please refer to http://osmocom.org/projects/osmotrx/wiki/OsmoTRX#UHD for -details; the following is an example for the B200 family USRP devices: - ----- -apt-get install libuhd-dev uhd-host -/usr/lib/uhd/utils/uhd_images_downloader.py ----- +The Trial Builder is the jenkins build slave (host) building all sysroot binary +packages used later by {app-name} to run the tests. It's purpose is to build the +sysroots and provide them to {app-anme}, for instance, as jenkins job artifacts +which the {app-name} runner job can fetch. [[jenkins_deps]] ==== Osmocom Build Dependencies @@ -56,11 +18,109 @@ aware of specific requirements for BTS hardware: for example, the osmo-bts-sysmo build needs the sysmoBTS SDK installed on the build slave, which should match the installed sysmoBTS firmware. +==== Add Build Jobs -[[configure_jenkins_slave]] -=== Jenkins Build and Run Slave +There are various jenkins-build-* scripts in osmo-gsm-tester/contrib/, which +can be called as jenkins build jobs to build and bundle binaries as artifacts, +to be run on the osmo-gsm-tester main unit and/or BTS hardware. + +Be aware of the dependencies, as hinted at in <>. + +While the various binaries could technically be built on the osmo-gsm-tester +main unit, it is recommended to use a separate build slave, to take load off +of the main unit. + +Please note nowadays we set up all the osmocom jenkins jobs (including +{app-name} ones) using 'jenkins-job-builder'. You can find all the +configuration's in Osmocom's 'osmo-ci.git' files 'jobs/osmo-gsm-tester-*.yml. +Explanation below on how to set up jobs manually is left as a reference for +other projects. + +On your jenkins master, set up build jobs to call these scripts -- typically +one build job per script. Look in contrib/ and create one build job for each of +the BTS types you would like to test, as well as one for the 'build-osmo-nitb'. + +These are generic steps to configure a jenkins build +job for each of these build scripts, by example of the +jenkins-build-osmo-nitb.sh script; all that differs to the other scripts is the +"osmo-nitb" part: + +* 'Project name': "osmo-gsm-tester_build-osmo-nitb" + + (Replace 'osmo-nitb' according to which build script this is for) +* 'Discard old builds' + + Configure this to taste, for example: +** 'Max # of build to keep': "20" +* 'Restrict where this project can be run': Choose a build slave label that + matches the main unit's architecture and distribution, typically a Debian + system, e.g.: "linux_amd64_debian8" +* 'Source Code Management': +** 'Git' +*** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" +*** 'Branch Specifier': "*/master" +*** 'Additional Behaviors' +**** 'Check out to a sub-directory': "osmo-gsm-tester" +* 'Build Triggers' + + The decision on when to build is complex. Here are some examples: +** Once per day: + + 'Build periodically': "H H * * *" +** For the Osmocom project, the purpose is to verify our software changes. + Hence we would like to test every time our code has changed: +*** We could add various git repositories to watch, and enable 'Poll SCM'. +*** On jenkins.osmocom.org, we have various jobs that build the master branches + of their respective git repositories when a new change was merged. Here, we + can thus trigger e.g. an osmo-nitb build for osmo-gsm-tester everytime the + master build has run: + + 'Build after other projects are built': "OpenBSC" +*** Note that most of the Osmocom projects also need to be re-tested when their + dependencies like libosmo* have changed. Triggering on all those changes + typically causes more jenkins runs than necessary: for example, it rebuilds + once per each dependency that has rebuilt due to one libosmocore change. + There is so far no trivial way known to avoid this. It is indeed safest to + rebuild more often. +* 'Build' +** 'Execute Shell' ++ +---- +#!/bin/sh +set -e -x +./osmo-gsm-tester/contrib/jenkins-build-osmo-nitb.sh +---- ++ +(Replace 'osmo-nitb' according to which build script this is for) + +* 'Post-build Actions' +** 'Archive the artifacts': "*.tgz, *.md5" + + (This step is important to be able to use the built binaries in the run job + below.) -==== Create 'jenkins' User on Main Unit + +TIP: When you've created one build job, it is convenient to create further +build jobs by copying the first one and, e.g., simply replacing all "osmo-nitb" +with "osmo-bts-trx". + +[[install_main_unit]] +=== Main Unit + +The main unit is a general purpose computer that orchestrates the tests. It +runs the core network components, controls the modems and so on. This can be +anything from a dedicated production rack unit to your laptop at home. + +This manual will assume that tests are run from a jenkins build slave, by a user +named 'jenkins' that belongs to group 'osmo-gsm-tester'. The user configuration +for manual test runs and/or a different user name is identical, simply replace +the user name or group. + +Please, note installation steps and dependencies needed will depend on lots of +factors, like your distribution, your specific setup, which hardware you plan to +support, etc. + +This section aims at being one place to document the rationale behind certain +configurations being done in one way or another. For an up to date step by step +detailed way to install and maintain the Osmocom {app-name} setup, one will want +to look at the <>. + +[[configure_jenkins_slave]] +==== Create 'jenkins' User On the main unit, create a jenkins user: @@ -176,81 +236,6 @@ Configure the node as: The build slave should be able to start now. - -==== Add Build Jobs - -There are various jenkins-build-* scripts in osmo-gsm-tester/contrib/, which -can be called as jenkins build jobs to build and bundle binaries as artifacts, -to be run on the osmo-gsm-tester main unit and/or BTS hardware. - -Be aware of the dependencies, as hinted at in <>. - -While the various binaries could technically be built on the osmo-gsm-tester -main unit, it is recommended to use a separate build slave, to take load off -of the main unit. - -On your jenkins master, set up build jobs to call these scripts -- typically -one build job per script. Look in contrib/ and create one build job for each of -the BTS types you would like to test, as well as one for the 'build-osmo-nitb'. - -These are generic steps to configure a jenkins build -job for each of these build scripts, by example of the -jenkins-build-osmo-nitb.sh script; all that differs to the other scripts is the -"osmo-nitb" part: - -* 'Project name': "osmo-gsm-tester_build-osmo-nitb" + - (Replace 'osmo-nitb' according to which build script this is for) -* 'Discard old builds' + - Configure this to taste, for example: -** 'Max # of build to keep': "20" -* 'Restrict where this project can be run': Choose a build slave label that - matches the main unit's architecture and distribution, typically a Debian - system, e.g.: "linux_amd64_debian8" -* 'Source Code Management': -** 'Git' -*** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" -*** 'Branch Specifier': "*/master" -*** 'Additional Behaviors' -**** 'Check out to a sub-directory': "osmo-gsm-tester" -* 'Build Triggers' + - The decision on when to build is complex. Here are some examples: -** Once per day: + - 'Build periodically': "H H * * *" -** For the Osmocom project, the purpose is to verify our software changes. - Hence we would like to test every time our code has changed: -*** We could add various git repositories to watch, and enable 'Poll SCM'. -*** On jenkins.osmocom.org, we have various jobs that build the master branches - of their respective git repositories when a new change was merged. Here, we - can thus trigger e.g. an osmo-nitb build for osmo-gsm-tester everytime the - master build has run: + - 'Build after other projects are built': "OpenBSC" -*** Note that most of the Osmocom projects also need to be re-tested when their - dependencies like libosmo* have changed. Triggering on all those changes - typically causes more jenkins runs than necessary: for example, it rebuilds - once per each dependency that has rebuilt due to one libosmocore change. - There is so far no trivial way known to avoid this. It is indeed safest to - rebuild more often. -* 'Build' -** 'Execute Shell' -+ ----- -#!/bin/sh -set -e -x -./osmo-gsm-tester/contrib/jenkins-build-osmo-nitb.sh ----- -+ -(Replace 'osmo-nitb' according to which build script this is for) - -* 'Post-build Actions' -** 'Archive the artifacts': "*.tgz, *.md5" + - (This step is important to be able to use the built binaries in the run job - below.) - - -TIP: When you've created one build job, it is convenient to create further -build jobs by copying the first and, e.g., simply replacing all "osmo-nitb" -with "osmo-bts-trx". - ==== Add Run Job This is the jenkins job that runs the tests on the GSM hardware: @@ -258,6 +243,15 @@ This is the jenkins job that runs the tests on the GSM hardware: * It sources the artifacts from jenkins' build jobs. * It runs on the osmo-gsm-tester main unit. +Sample script to run {app-name} as a jenkins job can be found in +'osmo-gsm-tester.git' file 'contrib/jenkins-run.sh'. + +Please note nowadays we set up all the osmocom jenkins jobs (including +{app-name} ones) using 'jenkins-job-builder'. You can find all the +configuration's in Osmocom's 'osmo-ci.git' files 'jobs/osmo-gsm-tester-*.yml. +Explanation below on how to set up jobs manually is left as a reference for +other projects. + Here is the configuration for the run job: * 'Project name': "osmo-gsm-tester_run" @@ -330,10 +324,47 @@ Details: and 'trial-N-bin.tgz' archives are produced by the 'jenkins-run.sh' script, both for successful and failing runs. -=== Install osmo-gsm-tester on Main Unit +==== Install osmo-gsm-tester This assumes you have already created the jenkins user (see <>). +Dependencies needed will depend on lots of factors, like your distribution, your +specific setup, which hardware you plan to support, etc. + +On a Debian/Ubuntu based system, these commands install the packages needed to +run the osmo-gsm-tester.py code, i.e. install these on your main unit: + +---- +apt-get install \ + dbus \ + tcpdump \ + sqlite3 \ + python3 \ + python3-setuptools \ + python3-yaml \ + python3-mako \ + python3-gi \ + python3-numpy \ + python3-wheel \ + ofono \ + patchelf \ + sudo \ + libcap2-bin \ + python3-pip \ + udhcpc \ + iperf3 \ + locales +pip3 install \ + "git+https://github.com/podshumok/python-smpplib.git@master#egg=smpplib" \ + pydbus \ + pyusb \ + pysispm +---- + +IMPORTANT: ofono may need to be installed from source to contain the most +recent fixes needed to operate your modems. This depends on the modem hardware +used and the tests run. Please see <>. + ==== User Permissions On the main unit, create a group for all users that should be allowed to use @@ -350,8 +381,6 @@ NOTE: you may also need to add users to the 'usrp' group, see A user added to a group needs to re-login for the group permissions to take effect. -This group needs the following permissions: - ===== Paths Assuming that you are using the example config, prepare a system wide state @@ -384,7 +413,7 @@ Put a DBus configuration file in place that allows the 'osmo-gsm-tester' group to access the org.ofono DBus path: ---- -cat > /etc/dbus-1/system.d/osmo-gsm-tester.conf < /etc/dbus-1/system.d/osmo-gsm-tester.conf < @@ -402,6 +431,21 @@ END (No restart of dbus nor ofono necessary.) +[[install_slave_unit]] +=== Slave Unit(s) + +The slave units are the hosts used by {app-name} to run proceses on. It may be +the <> itself and processes will be run locally, or +it may be a remote host were processes are run usually through SSH. + +This guide assumes slaves unit(s) use same configuration as the Main Unit, that +is, it runs under 'jenkins' user which is a member of the 'osmo-gsm-tester' user +group. In order to do so, follow the instruction under the +<> section above. Keep in mind the 'jenkins' user +on the Main Unit will need to be able to log in through SSH as the slave unit +'jenkins' user to run the processes. No direct access from Jenkins Master node +is required here. + [[install_capture_packets]] ===== Capture Packets @@ -464,6 +508,10 @@ contains stdout and stderr for that process (because this dir is set as CWD). sysctl -w kernel.core_pattern=core ---- +TIP: Files required to be installed under '/etc/security/limits.d/' can be found +under 'osmo-gsm-tester.git/utils/limits.d/', so one can simply cp them from +there. + ===== Allow Realtime Priority Certain binaries should be run with real-time priority, like 'osmo-bts-trx'. @@ -476,25 +524,20 @@ echo "@osmo-gsm-tester - rtprio 99" > /etc/security/limits.d/osmo-gsm-tester_all Re-login the user to make these changes take effect. -[[user_config_uhd]] -===== UHD - -Grant permission to use the UHD driver to run USRP devices for osmo-bts-trx, by -adding the jenkins user to the 'usrp' group: +TIP: Files required to be installed under '/etc/security/limits.d/' can be found +under 'osmo-gsm-tester.git/utils/limits.d/', so one can simply cp them from +there. ----- -gpasswd -a jenkins usrp ----- - -===== Allow CAP_NET_RAW capability +===== Allow capabilities: 'CAP_NET_RAW', 'CAP_NET_ADMIN', 'CAP_SYS_ADMIN' Certain binaries require 'CAP_NET_RAW' to be set, like 'osmo-bts-octphy' as it -uses a 'AF_PACKET' socket. +uses a 'AF_PACKET' socket. Similarly, others (like osmo-ggsn) require +'CAP_NET_ADMIN' to be able to create tun devices, and so on. To be able to set the following capability without being root, osmo-gsm-tester uses sudo to gain permissions to set the capability. -This is the script that osmo-gsm-tester expects on the main unit: +This is the script that osmo-gsm-tester expects on the host running the process: ---- echo /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh <>. - - -== Hardware Choice and Configuration - -=== SysmoBTS - -To use the SysmoBTS in the osmo-gsm-tester, the following systemd services must -be disabled: - ----- -systemctl mask osmo-nitb osmo-bts-sysmo osmo-pcu sysmobts-mgr ----- - -This stops the stock setup keeping the BTS in operation and hence allows the -osmo-gsm-tester to install and launch its own versions of the SysmoBTS -software. - -==== IP Address - -To ensure that the SysmoBTS is always reachable at a fixed known IP address, -configure the eth0 to use a static IP address: - -Adjust '/etc/network/interfaces' and replace the line - ----- -iface eth0 inet dhcp ----- - -with - ----- -iface eth0 inet static - address 10.42.42.114 - netmask 255.255.255.0 - gateway 10.42.42.1 ----- - -You may set the name server in '/etc/resolve.conf' (most likely to the IP of -the gateway), but this is not really needed by the osmo-gsm-tester. - -==== Allow Core Files - -In case a binary run for the test crashes, a core file of the crash should be -written. This requires a limits rule. Append a line to /etc/limits like: - ----- -ssh root@10.42.42.114 -echo "* C16384" >> /etc/limits ----- - -==== Reboot - -Reboot the BTS and make sure that the IP address for eth0 is now indeed -10.42.42.114, and that no osmo* programs are running. - ----- -ip a -ps w | grep osmo ----- - -==== SSH Access - -Make sure that the jenkins user on the main unit is able to login on the -sysmoBTS, possibly erasing outdated host keys after a new rootfs was loaded: - -On the main unit, for example do: - ----- -su - jenkins -ssh root@10.42.42.114 ----- - -Fix any problems until you get a login on the sysmoBTS. - - -[[hardware_modems]] -=== Modems - -TODO: describe modem choices and how to run ofono - -[[hardware_trx]] -=== osmo-bts-trx - -TODO: describe B200 family diff --git a/doc/manuals/chapters/install_device.adoc b/doc/manuals/chapters/install_device.adoc new file mode 100644 index 0000000..60c3936 --- /dev/null +++ b/doc/manuals/chapters/install_device.adoc @@ -0,0 +1,82 @@ +== Hardware Choice and Configuration + +=== SysmoBTS + +To use the SysmoBTS in the osmo-gsm-tester, the following systemd services must +be disabled: + +---- +systemctl mask osmo-nitb osmo-bts-sysmo osmo-pcu sysmobts-mgr +---- + +This stops the stock setup keeping the BTS in operation and hence allows the +osmo-gsm-tester to install and launch its own versions of the SysmoBTS +software. + +==== IP Address + +To ensure that the SysmoBTS is always reachable at a fixed known IP address, +configure the eth0 to use a static IP address: + +Adjust '/etc/network/interfaces' and replace the line + +---- +iface eth0 inet dhcp +---- + +with + +---- +iface eth0 inet static + address 10.42.42.114 + netmask 255.255.255.0 + gateway 10.42.42.1 +---- + +You may set the name server in '/etc/resolve.conf' (most likely to the IP of +the gateway), but this is not really needed by the osmo-gsm-tester. + +==== Allow Core Files + +In case a binary run for the test crashes, a core file of the crash should be +written. This requires a limits rule. Append a line to /etc/limits like: + +---- +ssh root@10.42.42.114 +echo "* C16384" >> /etc/limits +---- + +==== Reboot + +Reboot the BTS and make sure that the IP address for eth0 is now indeed +10.42.42.114, and that no osmo* programs are running. + +---- +ip a +ps w | grep osmo +---- + +==== SSH Access + +Make sure that the jenkins user on the main unit is able to login on the +sysmoBTS, possibly erasing outdated host keys after a new rootfs was loaded: + +On the main unit, for example do: + +---- +su - jenkins +ssh root@10.42.42.114 +---- + +Fix any problems until you get a login on the sysmoBTS. + + +[[hardware_modems]] +=== Modems + +TODO: describe modem choices and how to run ofono + +[[hardware_trx]] +=== osmo-bts-trx + +TODO: describe B200 family diff --git a/doc/manuals/chapters/intro.adoc b/doc/manuals/chapters/intro.adoc index 14daba4..9b7131d 100644 --- a/doc/manuals/chapters/intro.adoc +++ b/doc/manuals/chapters/intro.adoc @@ -1,31 +1,56 @@ -== Introduction with Examples +== Introduction -The osmo-gsm-tester is software to run automated tests of real GSM hardware, +{app-name} is a software to run automated tests on real GSM hardware, foremost to verify that ongoing Osmocom software development continues to work -with various BTS models, while being flexibly configurable and extendable. - -A 'main unit' (general purpose computer) is connected via ethernet and/or USB to -any number of BTS models and to any number of GSM modems via USB. The modems -and BTS instances' RF transceivers are typically wired directly to each other -via RF distribution chambers to bypass the air medium and avoid disturbing real -production cellular networks. Furthermore, the setup may include adjustable RF -attenuators to model various distances between modems and base stations. - -The osmo-gsm-tester software runs on the main unit to orchestrate the various -GSM hardware and run predefined test scripts. It typically receives binary -packages from a jenkins build service. It then automatically configures and -launches an Osmocom core network on the main unit and sets up and runs BTS -models as well as modems to form a complete ad-hoc GSM network. On this setup, -predefined test suites, combined with various scenario definitions, are run to -verify stability of the system. - -The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono -daemon to control the modems connected via USB. BTS software is either run -directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH -(e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for -ip.access nanoBTS). - -.Typical osmo-gsm-tester setup +with various BTS models, while being flexibly configurable and extendable to +work for other technologies, setups and projects. It can be used for instance to +test a 3G or 4G network. + +{app-name} (python3 process) runs on a host (general purpose computer) named +the 'main unit'. It may optionally be connected to any number of 'slave units', +which {app-name} may use to orchestrate processes remotely, usually through SSH. + +Hardware devices such as BTS, SDRs, modems, smart plugs, etc. are then connected +to either the main unit or slaves units via IP, raw ethernet, USB or any other +means· + +The modems and BTS instances' RF transceivers are typically wired directly to +each other via RF distribution chambers to bypass the air medium and avoid +disturbing real production cellular networks. Furthermore, the setup may include +adjustable RF attenuators to model various distances between modems and base +stations. + +Each of these devices, having each a different physical setup and configuration, +supported features, attributes, etc., is referred in {app-name} terminology as a +_resource_. Each _resource_ is an instance of _resource class_. A +_resource_class_ may be for instance a _modem_ or a _bts_. For instance, an +{app-name} setup may have 2 _modem_ instances and 1 _bts_ instances. Each of +these _resources_ are listed and described in configuration files passed to +{app-name}, which maintains a pool of _resources_ (available, in use, etc.). + +{app-name} typically receives from a jenkins build service the software or +firmware binary packages to be used and tested. {app-name} then launches a +specific set of testsuites which, in turn, contain each a set of python test +scripts. Each test uses the _testenv_ API provided by {app-name} to configure, +launch and manage the different nodes and processes from the provided binary +packages to form a complete ad-hoc GSM network. + +Testsuites themselves contain configuration files to list how many resources it +requires to run its tests. It also provides means to _filter_ which kind of +_resources_ will be needed based on their attributes. This allows, for instance, +asking {app-name} to provide a _modem_ supporting GPRS, or to provide a specific +model of _bts_ such as a nanoBTS. Testsuites also allow receiving _modifiers_, +which overwrite some of the default values that {app-name} itself or different +_resources_ use. + +Moreover, one may want to run the same testsuite several tiems, each with +different set of _resources_. For instance, one may want to run a testsuite with +a sysmoBTS and later with a nanoBTS. This is supported by leaving the testsuite +configuration generic enough and then passing _scenarios_ to it, which allow +applying extra _filters_ or _modifiers_. Scenarios can also be combined to +filter further or to apply further modifications. + +.Sample osmo-gsm-tester node 2G setup [graphviz] ---- digraph G { @@ -35,8 +60,8 @@ digraph G { label = "GSM Hardware"; style=dotted - modem0 [shape=box label="Modems..."] - modem1 [shape=box label="Modems..."] + modem0 [shape=box label="Modem (Quectel EC20)"] + modem1 [shape=box label="Modems (SierraWireless MC7455)"] osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box] B200 [label="Ettus B200" shape=box] sysmoCell5K [label="sysmocom sysmoCell5000" shape=box] @@ -46,13 +71,16 @@ digraph G { {modem0 modem1 osmo_bts_sysmo B200 octphy nanoBTS sysmoCell5K}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"] } + subgraph cluster_slave_unit { + label = "Slave Unit" + osmo_trx [label="osmo-trx"] + } subgraph cluster_main_unit { label = "Main Unit" osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"] subgraph { rank=same ofono [label="ofono daemon"] - osmo_trx [label="osmo-trx"] osmo_bts_trx [label="osmo-bts-trx"] osmo_bts_octphy [label="osmo-bts-octphy"] OsmoNITB [label="BSC + Core Network\n(Osmo{NITB,MSC,BSC,HLR,...})"] @@ -62,339 +90,15 @@ digraph G { jenkins->osmo_gsm_tester [label="trial\n(binaries)"] osmo_gsm_tester->jenkins [label="results"] - ofono->{modem0 modem1} [label="USB"] + ofono->{modem0 modem1} [label="QMI/USB"] osmo_gsm_tester->{OsmoNITB osmo_bts_trx osmo_bts_octphy} - osmo_gsm_tester->osmo_bts_sysmo [taillabel="SSH"] + osmo_gsm_tester->{osmo_trx, osmo_bts_sysmo} [taillabel="SSH"] osmo_gsm_tester->ofono [taillabel="DBus"] - osmo_trx->B200 [label="USB"] - osmo_bts_trx->{osmo_trx sysmoCell5K} [dir=both label="UDP"] + osmo_trx->B200 [label="UHD/USB"] + osmo_bts_trx->{osmo_trx sysmoCell5K} [dir=both label="TRXC+TRXD/UDP"] osmo_bts_octphy->octphy [label="raw eth"] {osmo_bts_sysmo nanoBTS}->OsmoNITB [label="IP"] {B200 octphy}->OsmoNITB [label="eth" style=invis] {osmo_bts_trx osmo_bts_octphy}->OsmoNITB } ---- - -.Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface) -[graphviz] ----- -digraph G { - rankdir=TB; - - suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800"] - - subgraph { - rank=same; - suite - scenarios - } - - subgraph cluster_scenarios { - label = "Scenarios"; - u_sysmoBTS [label="Scenario: sysmo\nbts: type: osmo-bts-sysmo"] - u_trx [label="Scenario: trx\nbts: type: osmo-bts-trx"] - u_arfcn [label="Scenario: band1800\narfcn: band: GSM-1800"] - } - - subgraph cluster_suite { - label = "Suite: sms"; - requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"] - subgraph cluster_tests { - label = "Test Scripts (py)"; - mo_mt_sms - etc - } - } - - subgraph cluster_resources { - label = "Resources"; - rankdir=TB; - nitb_addr1 [label="NITB interface addr\n10.42.42.1"] - nitb_addr2 [label="NITB interface addr\n10.42.42.2"] - Modem0 - Modem1 - Modem2 - sysmoBTS [label="osmo-bts-sysmo"] - osmo_bts_trx [label="osmo-bts-trx"] - arfcn1 [label="arfcn: 512\nband: GSM-1800"] - arfcn2 [label="arfcn: 540\nband: GSM-1900"] - - arfcn1->arfcn2 [style=invis] - nitb_addr1->nitb_addr2 [style=invis] - Modem0 -> Modem1 -> Modem2 [style=invis] - sysmoBTS -> osmo_bts_trx [style=invis] - } - - suite_scenarios -> {suite scenarios} - scenarios -> { u_arfcn u_sysmoBTS } - - suite -> requires - requires -> Modem0 - requires -> Modem1 - requires -> sysmoBTS - requires -> arfcn1 - requires -> nitb_addr1 - - { u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"] -} ----- - -.Example of a "trial" containing binaries built by a jenkins -[graphviz] ----- -digraph G { - subgraph cluster_trial { - label = "Trial (binaries)" - sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"] - trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"] - nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"] - checksums [label="checksums.md5"] - - checksums -> {sysmo trx nitb} - } -} ----- - -=== Typical Test Script - -A typical single test script (part of a suite) may look like this: - ----- -#!/usr/bin/env python3 -from osmo_gsm_tester.testenv import * - -hlr = suite.hlr() -bts = suite.bts() -mgcpgw = suite.mgcpgw(bts_ip=bts.remote_addr()) -msc = suite.msc(hlr, mgcpgw) -bsc = suite.bsc(msc) -stp = suite.stp() -ms_mo = suite.modem() -ms_mt = suite.modem() - -hlr.start() -stp.start() -msc.start() -mgcpgw.start() - -bsc.bts_add(bts) -bsc.start() - -bts.start() - -hlr.subscriber_add(ms_mo) -hlr.subscriber_add(ms_mt) - -ms_mo.connect(msc.mcc_mnc()) -ms_mt.connect(msc.mcc_mnc()) - -ms_mo.log_info() -ms_mt.log_info() - -print('waiting for modems to attach...') -wait(ms_mo.is_connected, msc.mcc_mnc()) -wait(ms_mt.is_connected, msc.mcc_mnc()) -wait(msc.subscriber_attached, ms_mo, ms_mt) - -sms = ms_mo.sms_send(ms_mt) -wait(ms_mt.sms_was_received, sms) ----- - -=== Resource Resolution - -- A global configuration 'resources.conf' defines which hardware is connected to the - osmo-gsm-tester main unit. -- Each suite contains a number of test scripts. The amount of resources a test - may use is defined by the test suite's 'suite.conf'. -- Which specific modems, BTS models, NITB IP addresses etc. are made available - to a test run is typically determined by 'suite.conf' and a combination of scenario - configurations -- or picked automatically if not. - -[[resources_conf_example]] -=== Typical 'resources.conf' - -A global configuration of hardware may look like below; for details, see -<>. - ----- -ip_address: -- addr: 10.42.42.2 -- addr: 10.42.42.3 -- addr: 10.42.42.4 -- addr: 10.42.42.5 -- addr: 10.42.42.6 - -bts: -- label: sysmoBTS 1002 - type: osmo-bts-sysmo - ipa_unit_id: 1 - addr: 10.42.42.114 - band: GSM-1800 - ciphers: - - a5_0 - - a5_1 - - a5_3 - -- label: Ettus B200 - type: osmo-bts-trx - ipa_unit_id: 6 - addr: 10.42.42.50 - band: GSM-1800 - launch_trx: true - ciphers: - - a5_0 - - a5_1 - -- label: sysmoCell 5000 - type: osmo-bts-trx - ipa_unit_id: 7 - addr: 10.42.42.51 - band: GSM-1800 - trx_remote_ip: 10.42.42.112 - ciphers: - - a5_0 - - a5_1 - -- label: OCTBTS 3500 - type: osmo-bts-octphy - ipa_unit_id: 8 - addr: 10.42.42.52 - band: GSM-1800 - trx_list: - - hw_addr: 00:0c:90:2e:80:1e - net_device: eth1 - - hw_addr: 00:0c:90:2e:87:52 - net_device: eth1 - -arfcn: - - arfcn: 512 - band: GSM-1800 - - arfcn: 514 - band: GSM-1800 - - arfcn: 516 - band: GSM-1800 - - arfcn: 546 - band: GSM-1900 - - arfcn: 548 - band: GSM-1900 - -modem: -- label: sierra_1 - path: '/sierra_1' - imsi: '901700000009031' - ki: '80A37E6FDEA931EAC92FFA5F671EFEAD' - auth_algo: 'xor' - ciphers: - - a5_0 - - a5_1 - features: - - 'sms' - - 'voice' - -- label: gobi_0 - path: '/gobi_0' - imsi: '901700000009030' - ki: 'BB70807226393CDBAC8DD3439FF54252' - auth_algo: 'xor' - ciphers: - - a5_0 - - a5_1 - features: - - 'sms' ----- - -=== Typical 'suites/*/suite.conf' - -The configuration that reserves a number of resources for a test suite may look -like this: - ----- -resources: - ip_address: - - times: 1 - bts: - - times: 1 - modem: - - times: 2 - features: - - sms ----- - -It may also request e.g. specific BTS models, but this is typically left to -scenario configurations. - -=== Typical 'scenarios/*.conf' - -For a suite as above run as-is, any available resources are picked. This may be -combined with any number of scenario definitions to constrain which specific -resources should be used, e.g.: - ----- -resources: - bts: - - type: osmo-bts-sysmo ----- - -Which 'ip_address' or 'modem' is used in particular doesn't really matter, so -it can be left up to the osmo-gsm-tester to pick these automatically. - -Any number of such scenario configurations can be combined in the form -':++...', e.g. 'my_suite:sysmo+tch_f+amr'. - -=== Typical Invocations - -Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for -the Osmocom core network as well as for the Osmocom based BTS models. To create -such a set of binaries, see <>. - -Examples for launching test trials: - -- Run the default suites (see <>) on a given set of binaries: - ----- -osmo-gsm-tester.py path/to/my-trial ----- - -- Run an explicit choice of 'suite:scenario' combinations: - ----- -osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts ----- - -- Run one 'suite:scenario' combination, setting log level to 'debug' and - enabling logging of full python tracebacks, and also only run just the - 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: - ----- -osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt ----- - -A test script may also be run step-by-step in a python debugger, see -<>. - -=== Resource Reservation for Concurrent Trials - -While a test suite runs, the used resources are noted in a global state -directory in a reserved-resources file. This way, any number of trials may be -run consecutively without resource conflicts. Any test trial will only use -resources that are currently not reserved by any other test suite. The -reservation state is human readable. - -The global state directory is protected by a file lock to allow access by -separate processes. - -Also, the binaries from a trial are never installed system-wide, but are run -with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that -several trials can run consecutively without conflicting binary versions. For -some specific binaries which require extra permissions (such as osmo-bts-octphy -requiring 'CAP_NET_RAW'), 'patchelf' program is used to modify the binary -'RPATH' field instead because the OS dynamic linker skips 'LD_LIBRARY_PATH' for -binaries with special permissions. - -Once a test suite run is complete, all its reserved resources are torn down (if -the test scripts have not done so already), and the reservations are released -automatically. - -If required resources are unavailable, the test trial fails. For consecutive -test trials, a test run needs to either wait for resources to become available, -or test suites need to be scheduled to make sense. (*<- TODO*) diff --git a/doc/manuals/chapters/resource_pool.adoc b/doc/manuals/chapters/resource_pool.adoc new file mode 100644 index 0000000..4a56767 --- /dev/null +++ b/doc/manuals/chapters/resource_pool.adoc @@ -0,0 +1,107 @@ +== Resource Resolution + +- A global configuration <> defines which hardware is plugged to the + {app-name} setup, be it the main unit or any slave unit. This list becomes the + 'resource pool'. +- Each suite contains a number of test scripts. The amount of resources a test + may use is defined by the test suite's <>. +- Which specific modems, BTS models, NITB IP addresses etc. are made available + to a test run is typically determined by <> and a combination of <> -- or picked automatically if not. + +.Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface) +[graphviz] +---- +digraph G { + rankdir=TB; + + suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800+mod-bts0-chanallocdescend"] + + subgraph { + rank=same; + suite + scenarios + defaults_conf [label="defaults.conf:\nbsc: net: encryption: a5_0"] + } + + subgraph cluster_scenarios { + label = "Scenarios"; + u_sysmoBTS [label="Scenario: sysmo\nresources: bts: type: osmo-bts-sysmo"] + u_trx [label="Scenario: trx\nresources: bts: type: osmo-bts-trx"] + u_arfcn [label="Scenario: band1800\nresources: arfcn: band: GSM-1800"] + u_chanallocdesc [label="Scenario: band1800\nmodifiers: bts: channel_allocator: descending"] + } + + subgraph cluster_suite { + label = "Suite: sms"; + requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"] + subgraph cluster_tests { + label = "Test mo_mt_sms.py"; + obj_nitb [label="object NITB\n(process using 10.42.42.2)"] + bts0 [label="object bts[0]"] + modem0 [label="object modem[0]"] + modem1 [label="object modem[1]"] + } + } + + subgraph cluster_resources { + label = "Available Resources (not already allocated by other Osmo-GSM-Tester instance)"; + rankdir=TB; + nitb_addrA [label="NITB interface addr\n10.42.42.1"] + nitb_addrA [label="NITB interface addr\n10.42.42.2"] + ModemA + ModemB + ModemC + sysmoBTS [label="osmo-bts-sysmo"] + osmo_bts_trx [label="osmo-bts-trx"] + arfcnA [label="arfcn: 512\nband: GSM-1800"] + arfcnB [label="arfcn: 540\nband: GSM-1900"] + + arfcnA->arfcnB [style=invis] + nitb_addrA->nitb_addrB [style=invis] + ModemA -> ModemB -> ModemC [style=invis] + sysmoBTS -> osmo_bts_trx [style=invis] + } + + suite_scenarios -> {suite scenarios} + scenarios -> { u_arfcn u_sysmoBTS u_chanallocdesc } + + suite -> requires + requires -> ModemA + requires -> ModemB + requires -> sysmoBTS + requires -> arfcnA + requires -> nitb_addrA + + { u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"] + u_chanallocdesc -> bts0 [label="influences\nbts[0]\nbehavior"] + defaults_conf -> obj_nitb [label="provides default values"] +} +---- + +=== Resource Reservation for Concurrent Trials + +While a test suite runs, the used resources are noted in a global state +directory in a reserved-resources file. This way, any number of trials may be +run consecutively without resource conflicts. Any test trial will only use +resources that are currently not reserved by any other test suite. The +reservation state is human readable. + +The global state directory is protected by a file lock to allow access by +separate processes. + +Also, the binaries from a trial are never installed system-wide, but are run +with a specific 'LD_LIBRARY_PATH' pointing at the <>, so that +several trials can run consecutively without conflicting binary versions. For +some specific binaries which require extra permissions (such as osmo-bts-octphy +requiring 'CAP_NET_RAW'), 'patchelf' program is used to modify the binary +'RPATH' field instead because the OS dynamic linker skips 'LD_LIBRARY_PATH' for +binaries with special permissions. + +Once a test suite run is complete, all its reserved resources are torn down (if +the test scripts have not done so already), and the reservations are released +automatically. + +If required resources are unavailable, the test trial fails. For consecutive +test trials, a test run needs to either wait for resources to become available, +or test suites need to be scheduled to make sense. (*<- TODO*) diff --git a/doc/manuals/chapters/trial.adoc b/doc/manuals/chapters/trial.adoc index bc9fe05..4b52608 100644 --- a/doc/manuals/chapters/trial.adoc +++ b/doc/manuals/chapters/trial.adoc @@ -1,13 +1,29 @@ [[trials]] == Trial: Binaries to be Tested -A trial is a set of pre-built binaries to be tested. They are typically built +A trial is a set of pre-built sysroot archives to be tested. They are typically built by jenkins using the build scripts found in osmo-gsm-tester's source in the 'contrib/' dir, see <>. -A trial comes in the form of a directory containing a number of '*.tgz' tar -archives as well as a 'checksums.md5' file to verify the tar archives' -integrity. +A trial comes in the form of a directory containing a number of '.*tgz' tar +archives (containing different sysroots) as well as a 'checksums.md5' file to +verify the tar archives' integrity. + +.Example of a "trial" containing binaries built by a jenkins job +[graphviz] +---- +digraph G { + subgraph cluster_trial { + label = "Trial (binaries)" + sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"] + trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"] + nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"] + checksums [label="checksums.md5"] + + checksums -> {sysmo trx nitb} + } +} +---- When the osmo-gsm-tester is invoked to run on such a trial directory, it will create a sub directory named 'inst' and unpack the tar archives into it. @@ -28,4 +44,22 @@ The script in 'contrib/jenkins-run.sh' takes care of related tasks such as * generating md5 sums for the various tar.gz containing software builds to be tested, * cleaning up after the build, * saving extra logs such as journalctl output from ofonod, -* generating a final .tar.gz file with all the logs and reports. +* generating a final .tar.gz file with all the logs and reports to store as jenkins archives. + +{app-name} tests create objects to manage the allocated resources during test +lifetime. These objects, in turn, usually run and manage processes started from +the trail's sysroot binaries. {app-name} provide APIs for those object classes +to discover, unpack and run those binaries. An object class simply needs to +request the name of the sysroot it wants to use (for instance 'osmo-bsc'), and +{app-name} will take care of preparing everything and providing the sysroot path +to it. It's a duty of the resource class to copy over the sysroot to the +destination if the intention is to run the binary remotely on another host. + +When seeking a sysroot of a given name '' in the 'inst/' directory, +{app-name} will look for 'tgz' files starting with the pattern '.' +(up to the first dot). That means, suffixes are available for {app-name} user to +identify the content, for instance having an incrementing version counter or a +commit hash. Hence, these example files are considered valid and will be +selected by {app-name} for 'osmo-bsc': 'osmo-bsc.tgz', 'osmo-bsc.build-23.tgz', +'osmo-bsc.5f3e0dd2.tgz', 'osmo-bsc.armv7.build-2.tgz'. If either none or more +than one valid file is found matching the pattern, an exception will be thrown. diff --git a/doc/manuals/chapters/troubleshooting.adoc b/doc/manuals/chapters/troubleshooting.adoc new file mode 100644 index 0000000..a3b5c8b --- /dev/null +++ b/doc/manuals/chapters/troubleshooting.adoc @@ -0,0 +1,15 @@ +== Troubleshooting + +=== Format: YAML, and its Drawbacks + +The general configuration format used is YAML. The stock python YAML parser +does have several drawbacks: too many complex possibilities and alternative +ways of formatting a configuration, but at the time of writing seems to be the +only widely used configuration format that offers a simple and human readable +formatting as well as nested structuring. It is recommended to use only the +exact YAML subset seen in this manual in case the osmo-gsm-tester should move +to a less bloated parser in the future. + +Careful: if a configuration item consists of digits and starts with a zero, you +need to quote it, or it may be interpreted as an octal notation integer! Please +avoid using the octal notation on purpose, it is not provided intentionally. diff --git a/doc/manuals/osmo-gsm-tester-manual-docinfo.xml b/doc/manuals/osmo-gsm-tester-manual-docinfo.xml index 923b8ad..d7b112f 100644 --- a/doc/manuals/osmo-gsm-tester-manual-docinfo.xml +++ b/doc/manuals/osmo-gsm-tester-manual-docinfo.xml @@ -21,10 +21,21 @@ Senior Developer + + Pau + Espin Pedrol + pespin@sysmocom.de + PE + + sysmocom + sysmocom - s.f.m.c. GmbH + Software Developer + + - 2017 + 2017-2020 sysmocom - s.f.m.c. GmbH diff --git a/doc/manuals/osmo-gsm-tester-manual.adoc b/doc/manuals/osmo-gsm-tester-manual.adoc index 6f0edf7..afee29b 100644 --- a/doc/manuals/osmo-gsm-tester-manual.adoc +++ b/doc/manuals/osmo-gsm-tester-manual.adoc @@ -1,20 +1,33 @@ -Osmo-GSM-Tester Manual -====================== -Neels Hofmeyr +:app-name: Osmo-GSM-Tester + +{app-name} Manual +================= +Neels Hofmeyr , Pau Espin Pedrol == WARNING: Work in Progress -*NOTE: The osmo-gsm-tester is still in pre-alpha stage: some parts are still -incomplete, and details will still change and move around.* +*NOTE: {app-name} is still under heavy development stage: some parts are still +incomplete, and details can still change and move around as new features are +added and improvements made.* include::{srcdir}/chapters/intro.adoc[] -include::{srcdir}/chapters/install.adoc[] +include::{srcdir}/chapters/trial.adoc[] include::{srcdir}/chapters/config.adoc[] -include::{srcdir}/chapters/trial.adoc[] +include::{srcdir}/chapters/resource_pool.adoc[] include::{srcdir}/chapters/test_api.adoc[] +include::{srcdir}/chapters/install.adoc[] + +include::{srcdir}/chapters/install_device.adoc[] + +include::{srcdir}/chapters/ansible.adoc[] + +include::{srcdir}/chapters/docker.adoc[] + include::{srcdir}/chapters/debugging.adoc[] + +include::{srcdir}/chapters/troubleshooting.adoc[] -- cgit v1.2.3