From das.signal at freecalypso.org Wed Mar 15 07:34:35 2023 From: das.signal at freecalypso.org (Das Signal) Date: Wed, 15 Mar 2023 07:34:35 +0000 Subject: Phones jumping ship from commercial networks to test network In-Reply-To: <20230124013306.1EA0E3740188@freecalypso.org> References: <20230124013306.1EA0E3740188@freecalypso.org> Message-ID: <20230315073435.GB29340@freecalypso.org> On Mon, Jan 23, 2023 at 05:33:00PM -0800, Mychaela Falconia wrote: > Hello GSM community, > > I have a question for those who operate their own GSM networks (be it > for fun or for research or for any other purpose) in places that DO > have regular commercial cell service, i.e., NOT ship-at-sea, middle of > desert or Rhizomatica-type environments: how do you deal with, and > ideally prevent, the highly undesirable situation of other people's > phones, not related to your operation, "jumping ship" from being > registered to their regular commercial network to trying to register > to your test network instead? Hi Mychaela, An idea could be to use this recent feature from osmo-hlr: https://github.com/osmocom/osmo-hlr/commit/268a33e58b9de3a2a6d42f27b1b9542ffd42f584 Currently, your wife's phone probably gets an "IMSI unknown in HLR" reject and I suppose the modem bugs after that, requiring a reboot. With the above commit, you could try the reject cause "PLMN not allowed" which would lead the PLMN being written on the forbidden PLMN list (EF FPLMN) and so in the future the phone should not even try to connect (and other phones as well). Alternatively I suppose you could update EF FPLMN manually, that should work as well. --DS From falcon at freecalypso.org Wed Mar 15 18:50:51 2023 From: falcon at freecalypso.org (Mychaela Falconia) Date: Wed, 15 Mar 2023 10:50:51 -0800 Subject: Phones jumping ship from commercial networks to test network In-Reply-To: <20230315073435.GB29340@freecalypso.org> References: <20230124013306.1EA0E3740188@freecalypso.org> <20230315073435.GB29340@freecalypso.org> Message-ID: <20230315185106.36BFB374024E@freecalypso.org> Hi DS! > An idea could be to use this recent feature from osmo-hlr: > https://github.com/osmocom/osmo-hlr/commit/268a33e58b9de3a2a6d42f27b1b9542ffd42f584 > > Currently, your wife's phone probably gets an "IMSI unknown in HLR" > reject and I suppose the modem bugs after that, requiring a reboot. Yup, the LU Reject cause was indeed the problem, as I quickly figured out once Neels' reply on the osmo-bsc list pointed me to the approximately-right part of the right spec. > With the above commit, you could try the reject cause "PLMN not allowed" Yes, I made the change to my local osmo-hlr instance to return "PLMN not allowed" instead of "IMSI unknown in HLR" as the LU Reject cause, and since I made that change, my wife's Nokia C3-00 has been behaving just fine in the presence of ThemWi GSM signal: I do see occasional registration attempts from it, but it stays registered to T-Mobile, thus it must be going back to working TMO registration after failed LU attempt to ThemWi. Now that I am confident that my ThemWi GSM signal is not "luring away" other neighbourhood phones from being registered to their respective commercial operators, I have been running the network for longer time intervals, and sometimes also at higher power levels. But I still run it at 3 dBm output (maximal reduction with max_power_red set to 20 in OsmoBSC config) most of the time. > which would lead the PLMN being written on the forbidden PLMN list (EF FPLMN) > and so in the future the phone should not even try to connect (and > other phones as well). Alternatively I suppose you could update EF FPLMN > manually, that should work as well. Writing into EF_FPLMN doesn't work because of T-Mobile SIM peculiarity. By the specs this EF is supposed to be writable by ordinary users (PIN1 access), but apparently TMO wanted to make it NOT writable by anyone other than them, so they made this hack: when I try to write something to EF_FPLMN, the UPDATE BINARY command appears to succeed (returns SW of 9000), but if I read the file afterward, I see that it is unchanged! But at least the affected phones (those that try to register to ThemWi despite having service from their own respective operator) seem to remember the "PLMN not allowed" state in their RAM until reboot, as they do go back to their rightful operator after failing to register to ThemWi. M~ P.S. I am hoping to provide more updates on ThemWi later, but right now I gotta run - I want to catch Harald's OsmoDevCall presentation on CSD, a very interesting topic for me. From falcon at freecalypso.org Sat Mar 25 21:10:13 2023 From: falcon at freecalypso.org (Mychaela Falconia) Date: Sat, 25 Mar 2023 13:10:13 -0800 Subject: Seeking pcap of RTP stream from nanoBTS Message-ID: <20230325211020.B6AA63740185@freecalypso.org> Hello GSM community, Would anyone in either of the two sub-communities of GSM (OsmoCNI and FC) happen to have a working setup with an ip.access nanoBTS, specifically with working voice calls? For the purpose of this inquiry, that "working setup" can be either with Osmocom CNI sw or with whatever original proprietary sw was once "official" for these ip.access nanoBTS units. If anyone does have a working nanoBTS setup with any sw, would you be willing to produce and share a packet capture (pcap file) of a working voice call, particularly the RTP stream originating from the nanoBTS? I am particularly interested in seeing a nanoBTS-generated RTP stream in either FR1 or EFR codec, as opposed to AMR or HR, coming from a GSM call UL with DTX enabled - having a 'dtx uplink' line in OsmoBSC config under 'bts N' should do it. The reason for my interest? I am looking to see what the pre-existing (before Osmocom) implementation of GSM-UL-to-RTP conversion in nanoBTS does in the two corner cases of (1) the MS exercising DTX during speech pauses and (2) speech frame 20 ms windows on TCH UL being stolen for FACCH. In case 1, does nanoBTS produce an intentional gap in its transmitted RTP stream (no packets sent at all) like OsmoBTS does, or does it do something different? Does it perhaps send RTP packets with zero-length payload, or some in-band bit pattern that is meant to indicate "bad frame, no data"? Case 2 is also interesting: current osmo-bts-trx (based on my reading of code, no hw to test on) invokes FR1 ECU and emits its output in this case, whereas stock (without my hacky patches) osmo-bts-sysmo exhibits an outright bug whereby nothing is emitted on RTP during the FACCH-stolen 20 ms window, and that gap in the RTP stream is NOT accounted for in the timestamps of subsequent RTP packets. Once again, I can only wonder what the pre-Osmocom implementation in nanoBTS does in this case. I would really like to produce a clean, potentially-mergeable patch to OsmoBTS and submit it to Gerrit, a patch that would add vty config settings selecting among several possible behaviours for RTP output in cases of DTX silence, FACCH stealing or bad radio Rx - but I really need to know what the different "reasonable" behaviour choices are, and I feel that we as in FOSS GSM community also need to know what our proprietary predecessors did in this area. I am not able to test this nanoBTS behaviour myself because even though I have nanoBTS hw (one PCS1900 unit and one GSM850), I have had no success in getting it to work with Osmocom - and my troubleshooting attempts hit a brick wall when the misbehaviour appears to be somewhere in the proprietary black box of nanoBTS. Hence I really need help from someone in the community who does have a working nanoBTS setup (with any sw) and who could make some test voice calls (ideally one FR1 and one EFR, but even just one of these two codecs would be interesting to see) with an RTP packet capture running, and then share the resulting pcap file. During that test voice call, it would be ideal if the tester could alternate between speaking and silence, and also cause some FACCH activity, perhaps by pressing DTMF keys. M~ From falcon at freecalypso.org Sun Mar 26 23:47:44 2023 From: falcon at freecalypso.org (Mychaela Falconia) Date: Sun, 26 Mar 2023 15:47:44 -0800 Subject: Seeking pcap of RTP stream from nanoBTS In-Reply-To: References: <20230325211020.B6AA63740185@freecalypso.org> <20230326200543.5D75E374019A@freecalypso.org> Message-ID: <20230326234754.79D2C374019A@freecalypso.org> Thank-you to Harald for capturing and publishing these files: > Please see the pcap files in https://people.osmocom.org/laforge/pcap/ Here is my analysis, focusing on FR1 and EFR codecs in osmobsc-nanobts-fr-dtx-dtmf.pcap and osmobsc-nanobts-efr-dtx.pcap - I don't feel ready to look at AMR yet. The behaviour I see from the nanoBTS is the same as what has been implemented in mainline OsmoBTS: whenever the UL from the MS goes into DTX or whenever a 20 ms traffic frame slot has been stolen for FACCH, the BTS transmits nothing on RTP and generates an intentional gap in its outgoing RTP stream. By "intentional gap" I mean that when the next RTP packet does go out at some later time, the sequence number is incremented by 1 relative to the last packet before the gap, but the timestamp is incremented by (N+1)*160, where N is the number of 20 ms packets that were suppressed. This finding is disappointing - I was hoping that they did something more PSTN-transcoder-friendly - but it is what it is, and thanks to Harald's experiments and pcap files, we now know the facts as to what at least one historical proprietary vendor had implemented prior to Osmocom. Now let me share some observations about USA PSTN and its "modern" incarnation in the form of SIP+RTP. My observations are based on the service I get via bulkvs.com - it's an ultra-cheap bulk telephony service providing access in the form of VoIP/SIP to the USA+Canada portion of the global worldwide E.164 PSTN. Through this service I have reserved several fully "real" and functional E.164 numbers in the USA number space (NANP), calls from other people's phones to any of these numbers turn into SIP INVITEs going to my server, and I can likewise dial outbound calls from my gateway to any phone number in +1. They support G.711 and G.729, but I only use G.711. The aspect of USA PSTN and its SIP+RTP form that is most relevant to the present discussion of RTP from a GSM BTS is the "shape" of G.711 PCMU RTP streams which I receive whenever I am SIP-connected to any USA phone via BulkVS. I've been playing with this service since 2022-06, I have made calls to and received calls from many kinds of different phones (GSM phones on legacy T-Mobile 2G, "modern" VoLTE phones, POTS analog land line, various "office" phones as in banks etc), and the PCMU RTP stream I receive always looks the same: perfectly smooth and continuous, a packet of 160 PCMU samples coming every 20 ms without fail, as if the stream is coming from a TDM to RTP converter. Seeing that apparently-all phone systems connected to USA E.164 number space produce such perfectly smooth and continuous RTP streams, I conclude that my public-interfaced GSM network needs to produce outgoing RTP streams (emanating from my PSTN-via-SIP interconnection gateway) that are no worse than the others, i.e., just as smooth and continuous, sending a packet of 160 samples every 20 ms without fail, no matter what is happening on the GSM side. Whenever GSM call UL goes into DTX, it is the GSM network's responsibility to generate comfort noise toward PSTN per the rules of GSM 06.12 or 06.62, and whenever a traffic frame is stolen for FACCH or lost to bad radio conditions, it is likewise the GSM network's responsiblity to apply the rules of GSM 06.11 or 06.61 - suddenty stopping the RTP stream going to PSTN is clearly NOT the right course of action, as no other parties at least in the USA publicly-interconnected telecom environment are seen to do so. In the Olden Days it was the T1/E1 TRAU that did all these functions, emitting a perfectly continuous G.711 sample stream on its output, and in the present-day all-IP world it becomes the job of my transcoding fiefdom-boundary MGW to do the same. And the easiest way to produce such a perfectly smooth G.711 RTP stream from a transcoding MGW is to force the BTS (the ultimate RTP stream timing origin) to emit *some* RTP packet in every 20 ms window, be it rain or shine, i.e., indicate what classic GSM specs call BFI conditions (times when you would see BFI=1 in UL TRAU frames) by way of some special "error" or BFI packet, rather than by complete absence of RTP packet transmission as is done in nanoBTS and current mainline OsmoBTS. I have raised this issue previously: https://osmocom.org/issues/5742 Now the new info we have is that the behaviour of intentional gaps in the RTP stream (the part which I consider to be a bad design) is not an original Osmocom invention, but was already present in OsmoBTS' and sysmoBTS' direct predecessor nanoBTS. For everyone reading my rants - please understand that I am *NOT* bashing Osmocom here! The only "thing" which I am "bashing" is the de facto, seemingly unwritten set of industry "standards" for IP-based GSM RAN, which in my opinion are an utter mess compared to the marvel of beauty that is traditional GSM RAN based on channelized T1/E1 lines, TRAU frames and perfect TDM timing. Anyway, moving from rants to constructive ideas, I am going to start working on a patch to OsmoBTS that will add a vty config option to select the behaviour of what it should do when a BFI condition occurs. The default will of course be to keep the current behaviour of generating an intentional gap in the RTP stream, but I see the following options as other reasonable alternatives (for FR/EFR): 1) Emit an RTP packet with a zero-length payload; 2) Emit an RTP packet containing an FR/EFR frame of all 0 bits - that's what osmo-bts-trx does currently in the corner case of the BFI condition being for some reason other than DTXu, but the codec being EFR (no ECU) or the FR1 ECU failing; 3) Emit the ad hoc FR/EFR BFI RTP packet format which I invented for Themyscira Wireless, hereby called Themyscira FR/EFR BFI packets. I anticipate that option 3 will likely raise the most objection from the established Osmocom community - but please let me point out that my themwi-mgw code (that receives and intelligently acts upon these Themyscira BFI packets) is public domain (see the newly added LICENSE file), free for the FOSS community to copy and modify and reintegrate in all kinds of ways, and I am not aware of anyone else having implemented a published-source transcoding MGW that is specifically designed to work with Osmocom GSM networks, connecting them to PSTN. And if the grand vision of Osmocom includes having transcoding functionality in OsmoMGW, then I'll make the argument that copying the logic from themwi-mgw would provide a solid start. I would also be more than happy to do an OsmoDevCall presentation covering ThemWi system sw, perhaps with a special focus on themwi-mgw if that is the part of most interest to Osmocom community, and explain how it interfaces with existing Osmocom CNI, why I made each design decision the way I did, and how it could potentially be brought within Osmocom CNI. With devotion to GSM Forever, (Hasta la Victoria, Siempre,) Mother Mychaela From falcon at freecalypso.org Fri Mar 31 18:59:38 2023 From: falcon at freecalypso.org (Mychaela Falconia) Date: Fri, 31 Mar 2023 10:59:38 -0800 Subject: 3GPP TS 28.062 section C.3.2.1.1 for EFR: seeking help Message-ID: <20230331185950.1035537401BE@freecalypso.org> Hello GSM community, I realize that most of you over in Osmocom land would much rather see me submit Gerrit patches than write lengthy ML posts, but right now I really need some help with the algorithmic logic of a feature before I can develop patches implementing said feature - so please bear with me. The fundamental question is: what is the most correct way for a GSM network (let's ignore divisions between network elements for the moment) to construct the DL speech frame stream for call leg B if it is coming from the UL of call leg A? I am talking about call scenarios where call leg A and call leg B use the same codec, thus no transcoding is done (TrFO), and let me also further restrict this question to old-style FR/HR/EFR codecs, as opposed to AMR. At first the answer may seem so obvious that many people will probably wonder why I am asking such a silly question: just take the speech frame stream from call leg A UL, feed it to call leg B DL and be done with it, right? But the question is not so simple. What should the UL-to-DL mapper do when the UL stream hits a BFI instead of a valid speech frame? What should this mapper do if call leg A does DTXu but there is no DTXd on call leg B? The only place in 3GPP specs where I could find an answer to this question is TS 28.062 section C.3.2.1.1. Yes, I know that it's the spec for in-band TFO within G.711, a feature which I reason no one other than me probably cares about, but that particular section - I am talking about section C.3.2.1.1 specifically, you can ignore the rest of TFO for the purpose of this question - seems to me like it should apply to _any_ scenario where an FR/HR/EFR frame stream is directly passed from call leg A to call leg B without transcoding, including scenarios like a self-contained Osmocom network with OsmoMSC switching from one MS to another without any external MNCC. Let us first consider the case of FR1 codec, which is the simplest. Suppose call leg A has DTXu but call leg B has no DTXd - one can't do DTXd on C0, so if 200 kHz of spectrum is all you got, operating a BTS with just C0, then no one can do DTXd. When Alice on call leg A is silent, her MS will send a SID every 480 ms and have its Tx off the rest of the time, and the frame stream from the BTS serving her call leg will exhibit a SID frame in every 24th position and BFI placemarkers in all other positions. So what should the DL frame stream going to Bob look like in this scenario? My reading of section C.3.2.1.1 (second paragraph from the top is the one that covers this scenario) tells me that the *network* (set aside the question of which element) is supposed to turn that stream of BFIs with occasional interspersed SIDs into a stream of valid *speech* frames going to Bob, a stream of valid speech frames representing comfort noise as produced by a network-located CN generator. The spec says in that paragraph: "The Downlink TRAU Frames shall not contain the SID codeword, but parameters that allow a direct decoding." Needless to say, there is no code anywhere in Osmocom currently that does the above, thus current Osmocom is not able to produce the fancy TrFO behavior which the spec(s) seem to call for. (I said "spec(s)" vaguely because I only found a spec for TFO, not for TrFO, but I don't see any reason why this aspect of TFO spec shouldn't also apply to TrFO when the actual problem at hand is exactly the same.) But no no no guys, I am *not* bashing Osmocom here, I am seeking to improve it! As it happens, fully implementing the complete set of TS 28.062 section C.3.2.1.1 rules (I shall hereafter call them C3211 rules for short) for the original FR1 codec would be quite easy, and I already have a code implementation which I am eyeing to integrate into Osmocom. Themyscira libgsmfrp is a FLOSS library that implements a complete, spec-compliant Rx DTX handler for FR1, and it is 100% my own original work, not based on ETSI or TI or any other sources, thus no silly license issues - and I am eyeing the idea of integrating the same functions, appropriately renamed, repackaged and re-API-ed, into libosmocodec, and then invoking that functionality in OsmoBTS, in the code path that goes from RTP Rx to feeding TCH DL to PHY layers. But while FR1 is easy, doing the same for EFR is where the real difficulty lies, and this is the part where I come to the community for help. The key diff between FR1 and EFR that matters here is how their respective Rx DTX handlers are defined in the specs: for FR1 the Rx DTX handler is a separate piece, with the interface from this Rx DTX handler to the main body of the decoder being another 260-bit FR1 frame (this time without possibility of SID or BFI), and the specs for DTX (06.31 plus 06.11 and 06.12) define and describe the needed Rx DTX handler in terms of emitting that secondary 260-bit FR1 frame. Thus implementing this functionality in Themyscira libgsmfrp was a simple matter of taking the logic described in the specs and turning it into code. But for EFR the specs do not define the Rx DTX handler as a separate piece, instead it is integrated into the guts of the full decoder. There is a decoder, presented as published C source from ETSI, that takes a 244-bit EFR frame, which can be either speech or SID, *plus* a BFI flag as input, and emits a block of 160 PCM samples as output - all Rx DTX logic is buried inside, intertwined with the actual speech decoder operation, which is naturally quite complex. I've already spent a lot of time looking at the reference C implementation of EFR from ETSI - I kinda had to, as I did the rather substantial work of turning it into a usable function library, with state structures and a well-defined interface instead of global vars and namespace pollution - the result is Themyscira libgsmefr - but I am still nowhere closer to being able to implement C3211 functionality for this codec. The problem is this: starting with a EFR SID frame and previous history of a few speech frames (the hangover period), how would one produce output EFR speech frames (not SID) that represent comfort noise, as C3211 says is required? We can all easily look at ETSI's original code that generates CN as part of the standard decoder: but that code generates linear PCM output, not secondary EFR speech frames that represent CN. There is the main body of the speech decoder, and there are conditions throughout that slightly modify this decoder logic in subtle ways for CN generation and/or for ECU-style substitution/muting - but no guidance for how one could construct "valid speech" EFR frames that would produce a similar result when fed to the standard decoder in the MS after crossing radio leg B. This is where I could really use some input from more senior and more knowledgeable GSM-ers: does anyone know how mainstream commercial GSM infra vendors (particularly "ancient" ones of pure T1/E1 TDM kind) have solved this problem? What do _they_ do in the scenario of call leg A with DTXu turning into call leg B without DTXd? Given that those specs were written in the happy and glorious days when everyone used 2G, when GSM operators had lots of spectrum, and when most networks operated large multi-ARFCN BTSes with frequency hopping, I figure that almost everyone probably ran with DTXd enabled when that spec section was written - hence if I wonder if the authors of the TFO spec failed to appreciate the magnitude of what they were asking implementors to do when they stipulated that a UL-to-DL mapping from DTXu-on to DTXd-off "shall" emit no-SID speech frames that represent TFO-TRAU-generated CN. And if I wonder if the actual implementors ignored that stipulation even Back In The Day... Here is one way how we might be able to "cheat" - what if we implement a sort of fake DTXd in OsmoBTS for times when real DTXd is not possible because we only have C0? Here is what I mean: suppose the stream of TCH frames about to be sent to the PHY layer (perhaps the output of my proposed, to-be-implemented UL-to-DL mapper) is the kind that would be intended for DTXd-enabled DL in the original GSM architecture, with all speech pauses filled with repeated SIDs, every 20 ms without fail. A traditional DTXd BTS is supposed to transmit only those SIDs that either immediately follow a speech frame or fall in the SACCH-aligned always-Tx position, and turn the Tx off at other times. We can't actually turn off Tx at those "other" times when we are C0 - but what if we create a "fake DTXd" effect by transmitting a dummy FACCH containing an L2 fill frame at exactly the same times when we would do real DTXd if we could? The end effect will be that the spec-based Rx DTX handler in the MS will "see" the same "thing" as with real DTXd: receiving FACCH in all those "empty" 20 ms frame windows will cause that spec-based Rx DTX handler to get BFI=1, exactly the same as if radio Tx were truly off and the MS were listening to radio noise. Anyway, I would love to hear other people's thoughts on these ideas, especially if someone happens to know how traditional GSM infra vendors handled those pesky requirements of TS 28.062 section C.3.2.1.1 for UL-to-DL mapping. Sincerely, Your GSM-obsessed Mother Mychaela