Themyscira Wireless update

[Mychaela Falconia]

Hi Rafael,

> btw, I got a PinePhone for me... of course far from being a FC,
> but at least I can load a free firmware in the Quectel EG-25.

My primary personal phone, running on T-Mobile's not-yet-shut-down GSM
network is still Pirelli DP-L10 - still a blob with no ability to fix
bugs or make functional improvements, but at least I can *see* what
it's doing by running rvtdump, and I can read out its file system with
fc-fsio (live) or fc-loadtool (while shut down), allowing me to
retrieve my received and sent SMS and transfer them to a real computer
for longer-term archival.

I still have my desire to build FC Venus board, but T-Mobile's
hostility to 2G forced me to take the long detour of setting up my own
GSM network first.

> I'm in +7 area,

Do you have a land line number in +7, or only mobile?  I ask because
call termination typically costs more to +7 mobile numbers than to +7
land lines.

> Could you send the link to the ETSI reference code?

I did not note down the original URL where I found it (somewhere under
etsi.org, but I had to dig), so I just reposted it on our own server:

ftp://ftp.freecalypso.org/pub/GSM/codecs/ts_146053v080000p0.zip

Further on the subject of EFR codec, I dug further and determined that
gapk support for EFR RTP encoding is completely broken.  Basically the
official RTP encodings for EFR and AMR are quite different: enough
though in terms of the actual codec EFR and the highest mode of AMR
are identical, their RTP specs were cast into stone at different times
by different people with different ideas.  For EFR we have ETSI
TS 101 318 (seems to be the original spec, referenced in RFC 3551),
and it prescribes that codec bits be packed into RTP bytes in the
"natural" order of codec parameters.  For AMR we have RFC 4867, based
on 3GPP TS 26.101, which prescribes a very different bit packing order
- the order of subjective importance.  Thus if you need to transcode
an RTP packet between EFR on one end and the highest mode of AMR on
the other end, you need to totally reshuffle the bit order.

What does it have to do with gapk EFR support?  gapk "supports" EFR by
way of opencore-amrnb library, calling exactly the same library
functions in exactly the same way for both AMR and EFR, but gapk
"format" layer for EFR RTP encoding is broken: it does a half-octet
shift, but no full reshuffling of bits, and the resulting bit order is
wrong because opencore-amrnb generates and takes in 3GPP TS 26.101 aka
AMR RTP bit order, not the one required for EFR RTP.

Of course one *could* use opencore-amrnb in production code for EFR
RTP, just add bit reshuffling functions - but it would be quite
inefficient because there is code deep inside opencore-amrnb that
picks individual bits for the order of subjective importance, and we
have to essentially reverse that built-in bit shuffling externally.
Right now I am leaning toward putting together my own library that
would work just like classic libgsm for 06.10, but for EFR, based on
the original EFR code from ETSI.

Looking at various parts of Osmocom code base that care about codecs -
not only gapk, but also codec code in libosmocore and osmo-bts - I get
a clear impression that Osmocom people don't really care much about
EFR.  They do a fairly decent job of supporting 06.10 and AMR, but
only bare minimum support for EFR.  But of course I and Osmocom are
driven by very different telos - my telos is giving utmost love and
attention to *vintage* mobile phones, while the driving telos of core
Osmocom people is something very different.  My reasoning for
prioritizing EFR over AMR goes like this:

1) There are many vintage phones that support EFR but not AMR, whereas
no one to my knowledge ever made a GSM phone that supports AMR but not
EFR.  There are also even older phones that support only 06.10 - and
once again no one to my knowledge ever made a phone that does not
support 06.10 - hence network support for 06.10 is also important.
However, I belong to North American telecom culture, in this culture
we use PCS1900 band for GSM, and there are very few (if any at all)
PCS1900 band phones that don't support EFR - I haven't found one yet.
Hence EFR should be considered the primary codec of choice for vintage
North American GSM phones.

2) In terms of learning curve and code complexity, EFR is undeniably
simpler than AMR.  Therefore, it makes sense to start with the simpler
option of EFR, make it work beautifully, and then possibly tackle AMR
at a later time, with lower priority.

In fact, the only reason why I care at all about AMR right now is
looking ahead to possible TFO or TrFO calls from/to other GSM networks.
Right now my outside voice interface is strictly G.711 only: any call
that enters or leaves Themyscira does so in G.711 64 kbps form, with
themwi-mgw transcoding between G.711 on the outside and my choice of
GSM codec on the inside.  Unfortunately this arrangement means that if
the other end of the call is also GSM, as opposed to a POTS land line
or some type of office phone, then we got two lossy codecs running in
tandem, which is worse than any one codec running end to end.  Hence
the desire to connect such calls in TFO or TrFO manner, if possible.

For as long as the legacy GSM network of T-Mobile USA still exists,
that network is the one of primary interest to me, in terms of TFO
efforts.  Obviously there is no way I could convince T-Mobile to set
up direct peering with me, bypassing regular PSTN routing, hence the
ultimate efficiency of TrFO won't be possible: whenever I receive a
call from TMO or route a call to them via regular PSTN, all I get is
G.711, with no option of negotiating GSM codecs for TrFO.  However, I
heard that TMO's GSM network runs on legacy TDM gear (not sure if it's
E1 or T1, given that it's GSM in USA) from Ericsson, not IP-based
gear, thus I suspect that whatever G.711 I get from them may be coming
from TRAUs that support old-style in-band TFO.  If so, then maybe I
could get tandem-free calls between TMO and ThemWi by doing in-band
TFO on G.711 in RTP!  I will very likely be the first and only person
on the planet doing that old-style TFO-in-G.711 *on RTP streams*, but
hey, that's what it means to think outside the box. :)

But if TFO from/to T-Mobile GSM really is possible, then I will
probably need to enable and support AMR in ThemWi, as I highly doubt
that TMO would reassign their call leg from TCH/H-AMR to TCH/F-EFR in
response to TFO negotiation messages asking them to do so.  With TMO
in their present state all TCH for AMR-capable phones are TCH/H, even
though the cells sit completely idle with zero call load except when I
or one of my family members with a GSM phone happen to make a call -
and whenever AMR mode is anything other than 12.2k, I don't see any
way to transcode it to EFR without going full tandem.

But all these thoughts regarding TFO from/to T-Mobile and/or TrFO
from/to other GSM networks around the world, perhaps run by more
cooperative people who would be open to direct peering with ThemWi,
are for a much later phase in the project - right now I still need to
get classic G.711 mode working in a rock solid manner, even if it
means tandem operation of two lossy codecs when the far end is also
GSM.  And if there is G.711 in the middle of the call, then it doesn't
matter which GSM codec I choose on my end, hence I go for EFR.

The next step for me is to make some changes to osmo-bts to make its
outgoing RTP stream continuous, without breaks or pauses at times of
GSM uplink DTX or frame loss.  I have surveyed the current state of
osmo-bts code, and here is what I see:

* osmo-bts-trx already integrates libosmocore ECU infrastructure so
that if the codec is 06.10 and a frame is lost for reasons other than
intentional DTX in the uplink, there will be no break in the RTP
stream - instead an ECU frame will be sent out.  However, there is
explicit code that suppresses this ECU sending when the uplink is in
the state of intentional DTX, as opposed to unplanned radio loss - I
reason that most people in Osmocom probably *like* the behavior of RTP
stream pausing during speech pauses, the opposite of my requirement.

* osmo-bts-sysmo (the version I need to run, given the hw I was able
to get at a steeply discount price - thanks Harald!) has no ECU
integration at all, so it will always produce a break in the RTP
stream any time voice frames are missing for any reason, including
FACCH stealing.

Given that different people have different needs (apparently most
people prefer the behavior of RTP stream stopping during speech pauses
and uplink DTX, whereas my application requires a continuous RTP stream
no matter what), the correct solution would be to add a configurable
option, which would probably need to be sent to the BTS from OsmoBSC.
However, I am not smart enough to implement such a feat, so I am going
to make a (presumably) non-mainlinable patch to osmo-bts that changes
the behavior to what I need, also implementing the logic in
osmo-bts-sysmo version at the same time.

To make this whole arrangement work with EFR in addition to 06.10, I
will also need to add an EFR ECU implementation to libosmocore - at
least this patch, when I get around to implementing it, should stand
some chance of being mainlinable...

Hasta la Victoria, Siempre,
Mychaela aka The Mother