AgeCommit message (Collapse)AuthorFilesLines
2020-09-29initial support for static userspace probes via systemtaplaforge/usdtHarald Welte7-1/+103
This adds a --enable-systemtap configure option, which will then add static tracepoints to the generated libosmocore binary. At this point, only two tracepoints are supported: stderr_start and stderr_done. They can be used to trace the amount of time a libosmocore-using application spends in blocking calls to log to stderr. Change-Id: I7e1ab664241deb524c9582cbd1bec31af46c747e
2020-09-29logging: introduce experimental 'systemd-journal' targetVadim Yanitskiy5-0/+211
This change implements 'systemd-journal' logging target, that is similar to the existing 'syslog' target. The key difference is that 'systemd-journal' allows us to offload rendering of the meta information, such as location (file name, line number), subsystem, and logging level, to systemd. Moreover, we can attach arbitrary, user-specific fields [1] to the logging messages, so they can be used for advanced log filtering (e.g. by IMSI/TMSI/TLLI): $ journalctl OSMO_SUBSYS=DMSC -f Since we don't want to make libsystemd a required dependency, this feature is optional, and needs to be enabled at build-time: $ ./configure --enable-systemd-logging The new logging target can be configured in the same way as any other one - via the VTY interface, or using the configuration file: log systemd-journal [raw] logging level set-all notice logging filter all 1 Two logging handlers are available: generic and raw. The first one behaves similarly to both 'syslog' and 'stderr', i.e. all the meta information is rendered by libosmocore itself, and then passed to systemd together with the logging message. The later is more like the 'gsmtap' target, so all available meta information is handed over to systemd in form of fields [1]: - CODE_FILE / CODE_LINE - location info, - PRIORITY - syslog-compatible logging level, - OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC), - OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex, - MESSAGE - the logging message itself, and then can be rendered in any supported format (e.g. JSON). More details about the API can be found in [2]. [1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html [2] https://www.freedesktop.org/software/systemd/man/sd-journal.html Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
2020-09-29WIP: add rate_ctr_group to log_targetHarald Welte3-3/+45
This way we account for the number of dropped log messages at write_queue overflow. However, log targets have no numerical ID and hence we're not able to extract them in a reasonable manner. Change-Id: I89b696311b823267e05d6a3e85b92c1784b220ed
2020-09-29logging: Attempt a synchronous, non-blocking write first (file, stderr)Harald Welte1-17/+30
In the old days, we performed synchronous, blocking writes to the log file or stderr. This was replaced by code that turned all log file/stderr writes into non-blocking writes behind a write_queue. This patch now introduces a further optimization: If we currently don't have any log messages pending in the write queue, we are not back-logged and assume we have a fair chance of writing the log message right now, synchronously. So we try that first, and only enqueue the log message if the write fails (no bytes or insufficient number of bytes written). This way we should get the best of both worlds: No delay/re-ordering (and lower select syscall load) for the "normal" case (benefits of the old synchronous writes) while at the same time never risking to block on log output. Change-Id: I08469a7e4be9bc5bbd39140457bb582f4a0b1703
2020-09-29logging: Avoid memcpy from stack to msgb in _file_output()Harald Welte1-18/+43
For file and stderr output, the existing code always generates the log string on a stack buffer, and then (in case of non-blocking write via write_queue) copies it over to a msgb. Let's optimize this by turning _file_output() into a raw_output callback which first allocates the msgb and then format-prints directly to that msgb instaed of stack + memcpy. This has the disadvantage that we don't know how long the buffer has to be in order to print the entire string to it. As a result we always have to allocate a 4k-sized buffer (plus msgb overhead). The write_queue length for log file output has been decreased from 1024 entries to 156 entries in order to stay within the same memory requirements for each log target memory pool (about 648 kBytes). Change-Id: I0d10b0199576d2e7ff6421a6dba19ae5ffafd946
2020-09-29logging: Change stderr + file target to use non-blocking writeHarald Welte3-14/+215
So far, we used blocking, buffered fwrite() to write to stderr and file targets. This causes problems if there are [slow] consumers causing delays, such as gnome-terminal (when the program is started interactively) or systemd/journald (where we observe 64..128ms blocks on stderr). This patch introduces stderr/file based logging via write_queue and osmo_select_main(), i.e. switch from glibc-buffered, blocking to internally buffered, non-blocking writes. Closes: OS#4311 Change-Id: Ia58fd78535c41b3da3aeb7733aadc785ace610da
2020-09-29logging: Introduce MAX_LOG_SIZE for the magic number 4096Harald Welte1-1/+4
Change-Id: Ia9645098dffecd29242857de8f1aea6b9aca5918
2020-09-29logging_vty: set osmo_stderr_target to NULL on "no log stderr"Harald Welte1-0/+2
We cannot keep a reference to the old log_target around if we just destroyed/deleted it. Change-Id: I98a801cf83347a4422534d020d5ed5e2d1eb5482
2020-09-29write_queue: Re-enqueue msgb if write_cb returns -EAGAINHarald Welte1-3/+8
By adding this functionality, the write_cb() handler can "un-dequeue" the msgb in case of some error. The msgb might have been modified meanwhile, e.g. due to a partial write already pulling some data off the head of the msgb. Change-Id: I97bb0d64ec991adf5dd0b3708e0c7cf029e03b5f
2020-09-29write_queue: use msgb_{en,de}queue_count()Harald Welte1-6/+3
The write_queue.c implemetation predates the msgb_*queue_count() functions for maintaining a count alongside witha msgb queue. Let's migrate over to those implementations. Change-Id: I0ebd42a50f239dd7e9f663ce4c42824a5c1b3ce7
2020-09-29write_queue: Add osmo_wqueue_enqueue_quiet()Harald Welte2-5/+19
Same as osmo_wqueue_enqueue() but without logging queue overruns. Change-Id: Ic082eb39795b08631284eeb421fef3c28f2e90dc
2020-09-28vty/command: reflect global attributes in the XML referenceVadim Yanitskiy1-0/+26
Given that commands with either/both of the following attributes: - CMD_ATTR_DEPRECATED, - CMD_ATTR_HIDDEN, never end up in the XML reference, only CMD_ATTR_IMMEDIATE would be reflected for commands taking effect immediately as follows: <command id='foo'> <!-- Global attributes --> <attributes scope='global'> <attribute doc='This command applies immediately' /> </attributes> <!-- Application specific attributes --> <attributes scope='application'> <!-- ... --> </attributes> <params> <!-- ... --> </params> </command> Change-Id: I8476c1163c23a9a52641987acf3df0b8c49d8f7b Related: SYS#4937
2020-09-28vty/command: introduce new attribute CMD_ATTR_IMMEDIATEVadim Yanitskiy1-0/+1
This attribute indicates that a VTY command applies immediately. Change-Id: Ia185dfd0c89214dc893af70736ff01dca3a7627e Related: SYS#4937
2020-09-27cosmetic: Fix typo in API docPau Espin Pedrol1-1/+1
Change-Id: I2c8f61768f96eda8d9f3e9b867a7e9625f4f6078
2020-09-26ns2: fix a msg leak when receiving REJECTED messagesAlexander Couzens2-4/+4
Change-Id: I7edc53c29c3dd09c395a275ceb8648b8eb19c06c
2020-09-25gprs_ns2: Make reason constDaniel Willmann2-3/+3
Change-Id: Ia0db18703d9fd137a5c5fe4474e2d76868a9af9d
2020-09-25ns2: refactor handle_nsip_read/handle_nsfrgre_readAlexander Couzens2-12/+6
Might be more readable. Change-Id: Ib2dbcae55a8c7a973cfa1123784f509708de7f3b
2020-09-25ns2: vty: fix behavior of vtyvc_by_nsei when vtyvc isn't foundAlexander Couzens1-7/+8
Fixes the parsing of persistent nsvcs. Change-Id: I297409f557f17df680ac76018b4202eb99713021
2020-09-25ns2: ns2_recv_vc: remove unused parameter nsiAlexander Couzens4-11/+8
Change-Id: I0dbc079e83d37de01a4260442363ed5fdcd51d28
2020-09-25ns2: refactor nsvc_by_ functionsAlexander Couzens4-19/+14
gprs_ns2_nsvc_by_sockaddr_nsei is doing the lookup within a NSE. gprs_ns2_nsvc_by_sockaddr_bind is doing the lookup within a bind. Make both function look similiar and take similiar arguments. Change-Id: Ia499fc279013668abe7348e578a0768f7d16faf9
2020-09-25ns2: check the specific bit of NS SDU Control bitsAlexander Couzens1-1/+2
Change-Id: I9632abe70585cb7aff2602590453d903dd88b48a
2020-09-24ns2: vty: add missing docs for IPv6 addressAlexander Couzens1-3/+6
Change-Id: I0ac04e1352ceadb7b787bcb3358c70e5bfd7c65b
2020-09-25ns2: remove bssgp specific msgb->cb partsAlexander Couzens1-3/+3
There shouldn't be any knowledge of the upper layer in the NS layer. The PCU / SGSN / gbproxy have to add the pointer when parsing the primitives. Change-Id: Id7edb8feb96436ba170383fc62d43ceb16955d53
2020-09-24gb/gprs_bssgb: ensure the fc timer has been stopped when freeing bssgp_bvcAlexander Couzens1-0/+2
Change-Id: Id03ebef31bb513e2b3b5bc709a91e9352ebbbc79
2020-09-24gb/gprs_bssgb: check if talloc failed on btsctx->fcAlexander Couzens1-4/+12
Change-Id: I1cfccc2cb696d9e95f590b99559d0a987031adfe
2020-09-22vty: cosmetic: fix missing curly braces in vty_describe_command()Vadim Yanitskiy1-2/+4
Change-Id: I1585ee70a3db77aa8737f9463c4d0665f4e90d90
2020-09-22vty: cosmetic: s/width/cmd_width/g in vty_describe_command()Vadim Yanitskiy1-10/+10
Change-Id: Ifcf5f7be925c5421040d3cac2c4c5e4dd497c0f5
2020-09-22vty: cosmetic: drop redundant 'break' statementsVadim Yanitskiy1-2/+0
Change-Id: I54ee7c3c6fe16a413d0e1128c7358ff6e4b76c6a
2020-09-22osmo_strlcpy: Clarify length calculationPau Espin Pedrol1-1/+1
Change-Id: I78dc7b87d5b42595c0b1392fce61606eae8f6ec8
2020-09-22osmo_strlcpy: Avoid calling memcpy with size=0Pau Espin Pedrol1-2/+2
Change-Id: Ic84af86d33c8a20f06036b13d0a10cf312582f0d
2020-09-21gsm_08_16.h: Add missing headerDaniel Willmann1-0/+1
utils.h is needed for struct value_string This probably never caused a problem because every file including gsm_08_16.h also included utils.h, but we should still include the file here. Change-Id: Iae09b4e8e42be6c371fb34279b7981db2af8cf4c
2020-09-20vty: print program specific attributes in the XML referenceVadim Yanitskiy1-1/+26
Change-Id: I1f35368ba9178e1454f2e3ddfcad9d96576143ef Related: SYS#4937
2020-09-20vty: add program specific attributes to VTY commandsVadim Yanitskiy3-1/+34
Change-Id: I2c6c7b317b2b28ce70784c0cabd1b913e721be02 Related: SYS#4937
2020-09-20add osmo_use_count_to_str_c()Neels Hofmeyr2-1/+27
So far there is only osmo_use_count_name_buf(). Also provide a use count to string using a talloc context, allowing to use OTC_SELECT. - instead of foo_name(), rather use foo_to_str(). - osmo_use_count_name_buf() returns the buf and not the chars_needed. So add osmo_use_count_to_str_buf() with a signature that is usable by OSMO_NAME_C_IMPL(). - provide osmo_use_count_to_str_c() using OSMO_NAME_C_IMPL(). Change-Id: I1d2e7ee979f8c316ef99f7c65675b36d092ddfca
2020-09-18gprs_ns2: Mark gprs_ns2_validate_* as staticHarald Welte1-5/+5
They are all called indirectly via gprs_ns2_validate(), so mark them as static. Change-Id: I8425ed3da32019b86cf12d3c27132f07d931cf27
2020-09-18ns2: Use NULL and not '0' when returning a NULL-PointerHarald Welte1-1/+1
Change-Id: I9458e6dcf8133fbcfbc89437e58aad049a6d432b
2020-09-18ns2: Improve/extend doxygen comments for new ns2 implementationHarald Welte6-126/+197
I was reading through the code and noticed many functions not documented yet, or with incomplete documentation. Change that. Change-Id: I85a2419604a9fd9ff3c4828a7463e222652f77bf
2020-09-18ipaccess.h: Add more enum values and 'official' namesHarald Welte1-14/+23
Originally we only learned about the protocol from looking at hexdumps without any specification or the like. Due to a GPL request to ip.acecss, we actually do have an 'official' resource: The packet-ipa.c from their wireshark-1.0.6ipa27.tar.gz Let's use its contents to complete our definitions here. Change-Id: Ic1f2b32c72d162f31b422293d2a361d528443f01
2020-09-18gsm0808: fix: do not encode invalid encryption algorithmVadim Yanitskiy1-7/+9
According to 3GPP TS 48.008, section, the Chosen Encryption Algorithm IE, which may be included in the following messages: - ASSIGNMENT COMPLETE - HANDOVER REQUEST - HANDOVER REQUEST ACKNOWLEDGE - HANDOVER COMPLETE - HANDOVER PERFORMED - CIPHER MODE COMPLETE is coded as follows: 0000 0001 No encryption used 0000 0010 GSM A5/1 0000 0011 GSM A5/2 0000 0100 GSM A5/3 0000 0101 GSM A5/4 0000 0110 GSM A5/5 0000 0111 GSM A5/6 0000 1000 GSM A5/7 basically A5/X => X + 1. All other values are Reserved for future international use. As can be seen, value 0x00 is RFU. Passing this value to some encoding functions would result in a PDU with this IE omitted. Although, some functions would still encode Chosen Encryption Algorithm IE with this RFU value. Let's ensure that all functions behave consistently. Change-Id: If10e433a8174eabe6aa6d2c2937bf9cf5d14d7c9
2020-09-18gprs_ns2_sns: Fix compilation on Debian 8Harald Welte1-10/+16
[ 198s] for (unsigned i = 0; i < gss->num_ip6_remote; i++) { [ 198s] ^ [ 198s] gprs_ns2_sns.c: In function 'ns2_sns_st_configured_change': [ 198s] gprs_ns2_sns.c:1053:3: error: 'for' loop initial declarations are only allowed in C99 or C11 mode [ 198s] for (int i = 0; i < num_v4; i++) { [ 198s] ^ [ 198s] gprs_ns2_sns.c:1067:3: error: 'for' loop initial declarations are only allowed in C99 or C11 mode [ 198s] for (int i = 0; i < num_v6; i++) { [ 198s] ^ [ 198s] Makefile:535: recipe for target 'gprs_ns2_sns.lo' failed Change-Id: I4b7c576fcdf9d35f85e00ad076af7c48d5eb34a5
2020-09-15libgb/ns: allow to create NS_ALIVE NSVCAlexander Couzens1-13/+28
Change-Id: I562c3ac4a8baaa2c97852bbd6a5a10ba1f144004
2020-09-15Gb: add a second NS implementationAlexander Couzens17-5/+6165
Reimplement NS with FSM. Change-Id: I3525beef205588dfab9d3880a34115f1a2676e48
2020-09-14bitXXgen: add osmo_loadXXbe_ext_2() to get right-adjusted valuesNeels Hofmeyr3-4/+98
As shown in the recently added bitgen_test.c, using osmo_loadXXbe_ext() with a smaller n produces results aligned on the most significant bytes, which is cumbersome, since it does not return a previously stored value. This problem exists only for the big-endian functions, the little-endian osmo_loadXXle_ext() properly return values adjusted on the least significant octets. Add osmo_loadXXbe_ext_2() variants that properly right-adjust the returned value. Prominently highlight this behavior in API doc. Test the new functions in bitgen_test.c. For example, this eases handling of 24bit integers (e.g. loaded from buffer to uint32_t, and stored into buffer from uint32_t). Also explicitly show this 24 bit case in bitgen_test.c Change-Id: I2806df6f0f7bf1ad705d52fa386d4525b892b928
2020-09-14bitXXgen: add bitgen_test.cNeels Hofmeyr4-0/+279
The autogenerated bitXXgen.h headers for osmo_load16le_ext() thru osmo_store64_be() are not actually tested at all. Add a test. The test output shows that the osmo_load*be_ext for a shorter len do not return nicely matching results. A practical example showing the difficulty in storing and loading 24bit integer values as/from big-endian: uint8_t buf[4]; memset(buf, 0, sizeof(buf)); osmo_store32be_ext(0x00112233, buf, 3); // stores 11 22 33 printf("%s\n", osmo_hexdump(buf, 4)); uint32_t r = osmo_load32be_ext(buf, 3); // returns 0x11223300, not 0x00112233 printf("0x%x\n", r); output is: 11 22 33 00 0x11223300 In contrast, the little-endian variant properly aligns the loaded bytes on the least significant octet: uint8_t buf[4]; memset(buf, 0, sizeof(buf)); osmo_store32le_ext(0x00112233, buf, 3); // stores 33 22 11 printf("%s\n", osmo_hexdump(buf, 4)); uint32_t r = osmo_load32le_ext(buf, 3); // returns 0x00112233 as expected printf("0x%x\n", r); output for le is: 33 22 11 00 0x112233 Change-Id: I5542ace54376a206aa8574812d4c742c86c293b4
2020-09-14bitXXgen: ensure not reading/storing past valid sizeNeels Hofmeyr1-0/+6
Add OSMO_ASSERT()s to ensure bounds checking. For example, for osmo_store32le_ext(), passing n > 5 would read past the end of the uint32_t. Similarly, osmo_load32le_ext() for n > 4 would write past the uint32_t's end. Change-Id: I2dc21582cd8a679b6624cefbc0c1678b093a3d08
2020-09-14logging: Avoid printing OSMO_LOGCOLOR_END if no color was usedPau Espin Pedrol1-1/+1
There's no point in printing that code if no color was used in first place, and looks strange when using logging with color enabled but no color assigned to the category printing lines. Only affected unit test output by this fix is osmo-bts'x tx_power_test.c, which has been fixed in osmo-bts.git Change-Id I5aa95997c8df4ce5ba8271acae99c45f68b96e11. Change-Id: Ie38cc639d7f4acd908f357e5bfb3ced07147583e
2020-09-11vty: fix 'Unsigned compared against 0' generate_cpu_hex_mask()Vadim Yanitskiy1-1/+1
Change-Id: I49549439d5bbfbb83954ba92957304b03062d003 Fixes: CID#212137
2020-09-09logging: fix memleak in log_target_create_file()Vadim Yanitskiy1-1/+5
Change-Id: If16d8c6417698bbb5cf87716f9507c7256aeaecb
2020-09-09logging: fix log_target_destroy(): properly close syslogVadim Yanitskiy1-0/+10
Change-Id: I122445e8ee842b5f807d21a272520648660e7e65
2020-09-09logging: refactor and simplify log_target_destroy()Vadim Yanitskiy1-10/+8
Change-Id: Ic6b9906e6fd24de6af9c869348c100d9ef6a7df6