summaryrefslogtreecommitdiffstats
path: root/doc/manuals/chapters/intro.adoc
blob: 50b6dffa46325aab60cce13eff1261e06d61d316 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
== Introduction with Examples

The osmo-gsm-tester is software to run automated tests of real GSM hardware,
foremost to verify that ongoing Osmocom software development continues to work
with various BTS models, while being flexibly configurable and extendable.

A 'main unit' (general purpose computer) is connected via ethernet and/or USB to
any number of BTS models and to any number of GSM modems via USB. The modems
and BTS instances' RF transceivers are typically wired directly to each other
via RF distribution chambers to bypass the air medium and avoid disturbing real
production cellular networks. Furthermore, the setup may include adjustable RF
attenuators to model various distances between modems and base stations.

The osmo-gsm-tester software runs on the main unit to orchestrate the various
GSM hardware and run predefined test scripts. It typically receives binary
packages from a jenkins build service. It then automatically configures and
launches an Osmocom core network on the main unit and sets up and runs BTS
models as well as modems to form a complete ad-hoc GSM network. On this setup,
predefined test suites, combined with various scenario definitions, are run to
verify stability of the system.

The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono
daemon to control the modems connected via USB. BTS software is either run
directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH
(e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for
ip.access nanoBTS).

.Typical osmo-gsm-tester setup
[graphviz]
----
digraph G {
	rankdir=LR;
	jenkins
	subgraph cluster_gsm_hardware {
		label = "GSM Hardware";
		style=dotted

		modem0 [shape=box label="Modems..."]
		modem1 [shape=box label="Modems..."]
		modem2 [shape=box label="Modems..."]
		osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box]
		B200 [label="Ettus B200" shape=box]
		octphy [label="Octasic octphy BTS" shape=box]
		nanoBTS [label="ip.access nanoBTS" shape=box]
		rf_distribution [label="RF distribution"]

		{modem0 modem1 modem2 osmo_bts_sysmo B200 octphy nanoBTS}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"]
	}
	subgraph cluster_main_unit {
	  label = "Main Unit"
	  osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"]
	  subgraph {
		rank=same
		ofono [label="ofono daemon"]
		OsmoNITB
		osmo_bts_trx [label="osmo-bts-trx\n+ osmo-trx"]
		osmo_bts_octphy [label="osmo-bts-octphy"]
	  }
	}


	jenkins->osmo_gsm_tester [label="trial\n(binaries)"]
	osmo_gsm_tester->jenkins [label="results"]
	ofono->{modem0 modem1 modem2} [label="USB"]
	osmo_gsm_tester-> {OsmoNITB osmo_bts_trx osmo_bts_octphy}
	osmo_gsm_tester-> osmo_bts_sysmo [taillabel="SSH"]
	osmo_gsm_tester-> ofono [taillabel="DBus"]
	osmo_bts_trx->B200 [label="USB"]
	osmo_bts_octphy->octphy [label="raw eth"]
	{osmo_bts_sysmo nanoBTS}->OsmoNITB [label="eth"]
	{B200 octphy}->OsmoNITB [label="eth" style=invis]
	{osmo_bts_trx osmo_bts_octphy}->OsmoNITB
}
----

.Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface)
[graphviz]
----
digraph G {
	rankdir=TB;

	suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800"]

	subgraph {
		rank=same;
		suite
		scenarios
	}

	subgraph cluster_scenarios {
		label = "Scenarios";
		u_sysmoBTS [label="Scenario: sysmo\nbts: type: osmo-bts-sysmo"]
		u_trx [label="Scenario: trx\nbts: type: osmo-bts-trx"]
		u_arfcn [label="Scenario: band1800\narfcn: band: GSM-1800"]
	}

	subgraph cluster_suite {
		label = "Suite: sms";
		requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"]
		subgraph cluster_tests {
			label = "Test Scripts (py)";
			mo_mt_sms
			etc
		}
	}

	subgraph cluster_resources {
		label = "Resources";
		rankdir=TB;
			nitb_addr1 [label="NITB interface addr\n10.42.42.1"]
			nitb_addr2 [label="NITB interface addr\n10.42.42.2"]
			Modem0
			Modem1
			Modem2
			sysmoBTS [label="osmo-bts-sysmo"]
			osmo_bts_trx [label="osmo-bts-trx"]
			arfcn1 [label="arfcn: 512\nband: GSM-1800"]
			arfcn2 [label="arfcn: 540\nband: GSM-1900"]

			arfcn1->arfcn2 [style=invis]
			nitb_addr1->nitb_addr2 [style=invis]
			Modem0 -> Modem1 -> Modem2 [style=invis]
			sysmoBTS -> osmo_bts_trx [style=invis]
	}

	suite_scenarios -> {suite scenarios}
	scenarios -> { u_arfcn u_sysmoBTS }

	suite -> requires
	requires -> Modem0
	requires -> Modem1
	requires -> sysmoBTS
	requires -> arfcn1
	requires -> nitb_addr1

	{ u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"]
}
----

.Example of a "trial" containing binaries built by a jenkins
[graphviz]
----
digraph G {
	subgraph cluster_trial {
		label = "Trial (binaries)"
		sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"]
		trx [label="osmo-bts-trx.build-5.tgz\n(osmo-trx + osmo-bts-trx\n+ deps\ncompiled for main unit)"]
		nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"]
		checksums [label="checksums.md5"]

		checksums -> {sysmo trx nitb}
	}
}
----

=== Typical Test Script

A typical single test script (part of a suite) may look like this:

----
#!/usr/bin/env python3
from osmo_gsm_tester.test import *

nitb = suite.nitb()
bts = suite.bts()
ms_mo = suite.modem()
ms_mt = suite.modem()

print('start nitb and bts...')
nitb.bts_add(bts)
nitb.start()
bts.start()

nitb.subscriber_add(ms_mo)
nitb.subscriber_add(ms_mt)

ms_mo.connect(nitb.mcc_mnc())
ms_mt.connect(nitb.mcc_mnc())

print('waiting for modems to attach...')
wait(ms_mo.is_connected, nitb.mcc_mnc())
wait(ms_mt.is_connected, nitb.mcc_mnc())
wait(nitb.subscriber_attached, ms_mo, ms_mt)

sms = ms_mo.sms_send(ms_mt)
wait(ms_mt.sms_was_received, sms)
----

=== Resource Resolution

- A global configuration defines which hardware is connected to the
  osmo-gsm-tester main unit.
- Each suite contains a number of test scripts. The amount of resources a test
  may use is defined by the test suite's 'suite.conf'.
- Which specific modems, BTS models, NITB IP addresses etc. are made available
  to a test run is typically determined by a combination of scenario
  configurations -- or picked automatically if not.

[[resources_conf_example]]
=== Typical 'resources.conf'

A global configuration of hardware may look like below; for details, see
<<resources_conf>>.

----
ip_address:
- addr: 10.42.42.1
- addr: 10.42.42.2
- addr: 10.42.42.3

bts:
- label: sysmoBTS 1002
  type: osmo-bts-sysmo
  addr: 10.42.42.114
  band: GSM-1800

- label: octBTS 3000
  type: osmo-bts-octphy
  addr: 10.42.42.115
  band: GSM-1800
  trx_list:
  - hw_addr: 00:0c:90:32:b5:8a
    net_device: eth0.2342

- label: Ettus B210
  type: osmo-bts-trx
  addr: 10.42.42.116
  band: GSM-1800

- label: nanoBTS 1900
  type: nanobts
  addr: 10.42.42.190
  band: GSM-1900
  trx_list:
  - hw_addr: 00:02:95:00:41:b3

arfcn:
  - arfcn: 512
    band: GSM-1800
  - arfcn: 514
    band: GSM-1800

  - arfcn: 540
    band: GSM-1900
  - arfcn: 542
    band: GSM-1900

modem:
- label: m7801
  path: '/wavecom_0'
  imsi: 901700000007801
  ki: D620F48487B1B782DA55DF6717F08FF9

- label: m7802
  path: '/wavecom_1'
  imsi: 901700000007802
  ki: 47FDB2D55CE6A10A85ABDAD034A5B7B3

- label: m7803
  path: '/wavecom_2'
  imsi: 901700000007803
  ki: ABBED4C91417DF710F60675B6EE2C8D2
----

=== Typical 'suites/*/suite.conf'

The configuration that reserves a number of resources for a test suite may look
like this:

----
resources:
  ip_address:
  - times: 1
  bts:
  - times: 1
  modem:
  - times: 2
----

It may also request e.g. specific BTS models, but this is typically left to
scenario configurations.

=== Typical 'scenarios/*.conf'

For a suite as above run as-is, any available resources are picked. This may be
combined with any number of scenario definitions to constrain which specific
resources should be used, e.g.:

----
resources:
  bts:
  - type: osmo-bts-sysmo
----

Which 'ip_address' or 'modem' is used in particular doesn't really matter, so
it can be left up to the osmo-gsm-tester to pick these automatically.

Any number of such scenario configurations can be combined in the form
'<suite_name>:<scenario>+<scenario>+...', e.g. 'my_suite:sysmo+tch_f+amr'.

=== Typical Invocations

Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for
the Osmocom core network as well as for the Osmocom based BTS models. To create
such a set of binaries, see <<trials>>.

Examples for launching test trials:

- Run the default suites (see <<default_suites>>) on a given set of binaries:

----
osmo-gsm-tester.py path/to/my-trial
----

- Run an explicit choice of 'suite:scenario' combinations:

----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts
----

- Run one 'suite:scenario' combination, setting log level to 'debug' and
  enabling logging of full python tracebacks, and also only run just the
  'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure:

----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt
----

A test script may also be run step-by-step in a python debugger, see
<<debugging>>.

=== Resource Reservation for Concurrent Trials

While a test suite runs, the used resources are noted in a global state
directory in a reserved-resources file. This way, any number of trials may be
run consecutively without resource conflicts. Any test trial will only use
resources that are currently not reserved by any other test suite. The
reservation state is human readable.

The global state directory is protected by a file lock to allow access by
separate processes.

Also, the binaries from a trial are never installed system-wide, but are run
with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that
several trials can run consecutively without conflicting binary versions.

Once a test suite run is complete, all its reserved resources are torn down (if
the test scripts have not done so already), and the reservations are released
automatically.

If required resources are unavailable, the test trial fails. For consecutive
test trials, a test run needs to either wait for resources to become available,
or test suites need to be scheduled to make sense. (*<- TODO*)