FreeCalypso update: progress with DSP patches and voice calls

[Mychaela Falconia]

Hi DS,

> I redid the tests just to be sure; recompiled and reflashed gsm-fw,
> then had the GTA02 and another phone register and made a call to the
> GTA02 handset. I can confirm the voice was fine. I'm using a version of
> gsm-fw from 2015-10-19 "static DSP patch download enabled by default".

OK, at least we know now that under *some* conditions (though we don't
know exactly what they are), our gsm-fw is able to make successful
voice calls after all.  But you still haven't had any luck with
connecting to real GSM networks in your part of the world with this
fw, have you?

> Please note I don't issue AT commands by hand, instead I simply launch
> the qt-moko GUI.

Well, it just means that the AT command interface presented by our new
version of ACI (taken from LoCosto along with the rest of G23M) is
good enough to make QtMoko happy - good to know. :)

> Perhaps the issue you're seeing is not due to the codec, but another
> quirk of the DSP. For instance, OpenBTS does not enable encryption by
> default (it only supports A5/3 anyway, which is too recent for Calypso).
> Although I don't know the internals of GSM that well, I remember other
> features of commercial networks, such as channel hopping, are not
> present either in OpenBTS.

Very good points indeed.  Evidently there is something different
between your test network on which gsm-fw is able to make successful
voice calls and Operator 310260's network on which it fails to do so,
but it could be any number of things.

The main point still stands though: our gcc-built gsm-fw is a great
feat, but it is still not a point where it can satisfy the needs of an
end user who needs working voice calls on real GSM networks.  TCS211,
on the other hand, works just fine in that department.  Exactly what
the problem is, we aren't able to narrow it down by conventional debug
techniques, as whatever goes south seems to happen inside the DSP,
which is a black box to us.  The finger of suspicion then points to
L1, the code responsible for controlling that black box, and we have
no real way to tell how our (presumably defective) LoCosto-based L1
compares to the working one from TCS211, as the latter is all binary
blobs.

On the tcs211-l1-reconst front, I made my way through l1_afunc.c and
l1_async.c: replacing original object blobs with recompiled versions
of these two modules did not change the behavior of the fw, at least
not that I could notice.  However, when I got to l1_cmplx.c (I was
going down alphabetically through l1_*.c core modules), I got this
module to compile and link in the TCS211 environment, but the fw
behavior became worse.  It was still able to make voice calls with
good audio, but some unreliability/instability was introduced: plenty
of DSP errors were seen, although all of them apparently non-fatal.
This erratic behavior was only seen in that tcs211-l1-reconst state of
l1_afunc, l1_async and l1_cmplx replaced with new versions while the
rest of L1 remained original objects - a very peculiar state indeed -
and was never seen in the gsm-fw environment.

My original idea with tcs211-l1-reconst was to massage *each individual
C file* in such a way that it would compile into a perfect equivalent
of the corresponding object blob from TCS211 - and therefore original
and recompiled objects could be mix-n-matched in any combination,
allowing the work to focus on just one C module at a time.  But as I
quickly discovered when I delved into it, this idea is *much* easier
said than done.  Just taking l1_cmplx.c as the current stumbling block,
the differences between the result of recompilation and the original
object are so great that it's difficult for me to even approach that
mess.

Seeing that this L1 reconstruction idea may be a year+ long project in
itself, possibly needing some very advanced tools like a custom
decompiler (highly tuned to TMS470 compiler output) to be developed
first, I decided to try taking our overall FC project in a rather
different direction...  So here's what I got to show now:

1. I made gsm-fw run on Compal targets in addition to the already
   supported Openmoko and Pirelli.  No UI yet of course, so the LCD
   stays dark and the buttons do nothing - you need to control it with
   AT commands via fc-shell - but as I expected, gsm-fw running on
   C1xx targets is no different from the same fw running on gtamodem
   and pirelli targets: it successfully registers with the network,
   SMS works, but voice calls exhibit exactly the same problems.  I
   got a C156 that has 2 MiB of XRAM, so I was able to run gsm-fw in
   the ramImage config without touching the flash, as well as a bunch
   of C139s - the latter require flashing, as the RAM is way too small
   to even consider running a GSM fw image out of it.

2. Much more promising, I made a port of the blob-laden but solidly
   working Wine-built TCS211 fw to Mot C139:

https://bitbucket.org/falconian/tcs211-c139

Right now tcs211-c139 is also an ACI-only fw (no UI, control via AT
commands via fc-shell), but in terms of GSM functionality it works
just as solidly as TCS211 on its native GTA02 hardware: it registers
with the network OK, deep sleep just works right out of the box (!),
SMS works, and voice calls work with perfectly good audio.

You can probably guess where I'm heading with this idea.  Yes, this
TCS211 fw is laden with blobs and requires a nasty Windows environment
to compile, but if we can get the UI layers in this TCS211 fw ported
over to C139 hw as well, we may be able to get a practically usable
(for end users, as in phones in pockets and purses!) fw for this
target.  It will still have blobs for the GSM protocol stack and L1,
and all development will have to be done in the unpleasant TCS211
Windows-tainted environment, but all UI code will be built from source
controlled by us.  While not as good as our (still-standing, of course)
more ambitious goal of a libre dumbphone whose fw is compiled from
full source with gcc, I argue that it would still be a significant
incremental improvement over the current state of affairs of us having
to use original manufs' firmwares on our dumbphones which we are
*completely* powerless to improve, even the non-radio UI layers.

For instance, consider what I currently have to do as an end user:
with my end user hat on, I carry a Pirelli DP-L10 phone in my purse.
It runs its original fully proprietary fw, as no end-user-usable libre
fw exists yet for *any* dumbphone.  And it is the UI design misfeatures
of this proprietary fw that drive me up the wall, not anything in the
GSM radio protocols.  Thus gaining the ability to improve the UI on a
dumbphone and make it follow what *I* (as opposed to Motorola, Nokia,
LG, Pirelli or whoever) consider good phone UI design is what I need
the most.  Liberating the GSM radio protocol stack underneath would
surely be nice, but for me personally it is a much lower priority.

Thus I feel that creating semi-libre firmware for Mot C139 in which
the UI code is fully liberated while the GSM radio protocol stack
components remain object blobs would be a worthwhile FreeCalypso
subproject.  In fact, from my perspective as an end user who very
badly wants a libre dumbphone, to me the biggest shortcoming of what I
propose is not the presence of binary blobs, but the extremely feeble
nature of Mot C139 hardware.  Switching from Pirelli DP-L10 to Mot C139,
even if it is accompanied by a firmware change for the better in terms
of freedom to make UI improvements, would be a major downgrade for me.
Going from a 128x128 pix LCD to 96x64 pix is a reduction of screen
real estate by a factor of more than 2.5, and I'm going to *really*
miss the hands-free loudspeaker - albeit on the Pirelli the usability
of this loudspeaker is greatly hampered by the stupid proprietary UI
firmware that refuses to turn it on until the call is answered on the
other end...

But the Pirelli is too laden with undocumented hardware to be much of
a viable target for libre fw - remember the spectacular failure when I
tried running a port of TCS211 on it.  Thus I still feel that building
our own hardware is still the *only* way to get a dumbphone that truly
Does Not Suck.  But semi-libre fw on Mot C139 seems like a reasonable
interim step to help those who can't wait for our own hw.  And once we
have this semi-libre fw on Mot C139, I would say building our own hw
should be a higher priority next step than reconstructing L1 to get fw
without any blobs...

Thoughts, ideas, comments?

Hasta la Victoria, Siempre,
Mychaela