FreeCalypso update: progress with DSP patches and voice calls

[Mychaela Falconia]

Hello everyone,

It has been almost a month and a half since my last update here, so I
thought I would let everyone know that I am still alive and kickin'.

I have just tracked down and fixed the bug that kept us from being
able to load the static DSP patch from TCS211, and thereby brought us
one step closer to having working voice calls in our gcc-built libre
GSM firmware.

Just to recap: between July and now, our gcc-built fw was able to
connect to GSM networks and send and receive SMS, but voice calls were
totally failing: a call would typically drop before it even finished
connecting.  We were then running without any DSP patches, whereas all
production firmwares known to us always apply some.  An experiment
showed that if we knock out both static and dynamic DSP patches in our
working TCS211/leo2moko reference, it also starts failing on voice
calls in a manner similar to our own work-in-progress fw, so it was a
reasonable conclusion that at least the static patch is necessary.
But when we enabled downloading of this static patch (with actual
patch code extracted from TCS211) in our gcc-built gsm-fw, other
things started breaking and the DSP was reporting a checksum different
from the expected value, indicating that the patch must be getting
corrupted somehow.

Tracking down the cause of this DSP patch corruption was not trivial
at all.  At first I thought something was wrong in the actual process
of downloading the patch (which comes as a char array) into the DSP,
but I couldn't find anything wrong there.  Then I decided to examine
the structure of this DSP patch: even though it comes as a raw char
array, the code that downloads it (leadapi.c, came as real source with
our copy of TCS211) understands enough of its structure to figure out
where one section ends and another begins, and where in the DSP's RAM
address space each section needs to be loaded.

Thankfully one of the docs we have for the Calypso chipset hardware
shows the memory map of the DSP - just a bird's eye view thereof, but
enough to figure out what the download section addresses in the patch
correspond to.  The DSP "thinks" in terms of 16-bit words, not bytes,
and its addresses are word addresses, not byte: an increment of 1 in
the DSP address is an advance of 16 bits, not 8.  Think of it like the
good old PDP-11 (yes, I'm showing my mental age here) where memory
sizes were traditionally measured in words instead of bytes.

The Calypso DSP has a total of 28 Kwords of RAM (much less RAM than
mask ROM: it has 128 Kwords of the latter), and 8 Kwords out of that
constitute dual-ported API RAM, accessible to both the DSP and the ARM
part of the Calypso (the MCU as TI called it).  DSP word addresses
from 0 to 0x6FFF map to the RAM (28 Kwords in total), and out of that
total RAM address range, DSP word addresses 0x800 through 0x27FF cover
the 8 Kword portion that is also visible from the ARM side as the API
RAM.  On the ARM side this API RAM appears at 0xFFD00000, and 8 Kwords
become 16 Kbytes.

When I examined the structure of the "base" (statically loaded before
any of the dynamic ones) DSP patch from TCS211, I saw that all code
download sections except one go into non-API RAM, i.e., into DSP RAM
areas which only the DSP can access - this part was as expected.  But
there is one code section that gets loaded into an address range in
the API RAM, and appears to stay there.  That section naturally drew
my attention: being in a RAM that is accessible to both DSP and ARM
cores makes it more susceptible to corruption.

Then I wrote a utility that reads and dumps the content of the DSP API
RAM in a Calypso GSM device as the regular fw is running, using the
ETM memory read command in 16-bit mode.  This utility has been added
to the FreeCalypso host tools suite as fc-dspapidump.  I ran this
utility against our "golden" TCS211/leo2moko reference, and indeed the
content of those API RAM words was exactly the same as in the patch,
confirming my assumption that these API RAM words are reserved for
that part of the patch code and not reused for something else.

Then I did the same fc-dspapidump from our failing fw, after it
downloaded the patch and got the wrong checksum back (0x15C8 instead
of the expected 0xD278).  Observed result: one word in the API RAM
range in question was different.  API RAM word 0xEF9 (0x16F9 as
addressed by the DSP, 0xFD001DF2 as addressed by the ARM) contains
0x4350 in the working TCS211 case, as loaded from the patch code array,
but in the case of our failing fw it contained 0x0000.  A quick bit of
hex math confirmed that this word must be responsible for the checksum
mismatch: 0xD278 + 0x4350 = 0x15C8 with 16-bit word overflow.  (The
checksum function must be adding all words together as a simple ripple
carry sum and then taking either a 1's or a 2's complement.)

Then I went over our L1 code, looking to see if I could find any
references to the DSP API word in question, and found the culprit.  It
was a LoCosto-ism that slipped through the cracks.

As a reminder, most of the difficulties we are having in our FC project
in general stem from the destructive nature of Man: many of TI's
sources and docs from the Calypso era apparently have not survived the
mainstream society's reclassification of the Calypso as "obsolete junk"
followed by the closure of the TI division in question, and we are left
with just bits and pieces to work with.  A source for a Calypso version
of L1 that would correspond closely to the "golden" version we've got
in binary object form is one of those things which we lack.  Aside
from the TSM30 version which has other problems, the only TI L1 source
we have comes from their LoCosto program, not Calypso, and this LoCosto
L1 source formed the basis for the L1 code in our current FC GSM fw.

TI did not follow what I would consider to be good software development
practices.  If you've got an older chip and working firmware for it,
and now you've got a newer chip and you need to make the fw run on the
latter, usually the responsible course of action would be to add
support for the newer chip without breaking the old one, inserting
conditional compilation directives as necessary.  Then you would have
just one source to maintain, building different binaries for different
hw targets, with features that make sense only on some (newer) targets
cleanly disabled on those hw targets where they can't work, while
target hw-independent improvements automatically benefit the older hw
as well.  You would regression-test your fw on the older targets to
make sure you didn't break them.  This is the approach taken by the
Linux kernel, for example.

But TI failed to do the above with their L1 code.  When they ported L1
from Calypso to LoCosto, they only did lip service to keeping it
backward-compatible with the Calypso.  There are plenty of C
preprocessor conditionals on CHIPSET, DSP, ANLG_FAM and RF_FAM
(preprocessor-defined magic numbers identifying different hw targets),
but they did not regression-test this new L1 code on the Calypso, and
in actuality the Calypso support in the LoCosto version of L1 is
broken: if one tries to compile that LoCosto L1 source with the
preprocessor symbols (CHIPSET, DSP etc) set to their Calypso values,
the code doesn't even compile.  I had to fix all those compilation
failures in the process of integrating that code into our FC gsm-fw.

Most of the trouble spots in the LoCosto version L1 take the form of
TI having added some LoCosto-ism without taking care to put it under
conditional compilation, thereby breaking the Calypso configuration.
Most of these were caught and fixed in the process of getting the code
to compile, as the offending LoCosto-ism would usually break
compilation by referencing something that doesn't exist in the Calypso
configuration.  But some slipped through, and one of these crawling
critters was the cause of our DSP patch corruption.

One of the new features added with LoCosto is measurement of the load
on the DSP CPU.  Apparently it requires the new DSP ROM version in the
LoCosto, as in several places the code in question is conditionalized
on #if (DSP >= 38).  Our Calypso DSP is 36.  But there were some spots
where code was added for the DSP CPU load measurement feature without
being conditionalized thus.  I fixed most of these back in the summer
of 2014 when I was integrating this L1 into our gsm-fw, as they were
breaking compilation.  But two spots remained: the definition of the
DSP API RAM addresses for this feature, and the writing of one of
those words from the ARM side.

Yup, you probably guessed it by now: the DSP API words used by the DSP
CPU load measurement feature on LoCosto (0xEF0-0xEF9) fall into the
range used by our DSP patch from TCS211 on the Calypso.  Removing the
offending write into the 0xEF9 word from the l1ddsp_end_scenario()
function in l1_drive.c (putting the #if (DSP >= 38) conditional where
TI neglected to do so) made things quite a bit better.

The two most recent commits to the freecalypso-sw Hg tree fix this
LoCosto creepage bug and enable downloading of the static DSP patch
from TCS211.  With these changes applied, we are now considerably
closer to working voice calls, although we still aren't quite there
yet.  With non-broken static DSP patch loading enabled, voice calls
now connect reliably, whereas previously connection establishment was
failing.  However, the downlink audio (heard in the earpiece of the
GTA02 with the right ALSA scenario file loaded from the AP side)
consists of nothing but noise.  The uplink audio transmitted toward
the far end of the call seems to be OK, though.

I was doing all of my tests in this series on the GTA02 target,
connecting to the live GSM network of Operator 310260.  The debug
trace output I get seems to indicate that the codec selected by the
network was AMR.  I invite DS to test this latest fw fix against his
lab network with the plain FR codec.

It is worth noting that our current state of voice calls connecting,
uplink audio being apparently good, but total noise on the downlink is
exactly the same as what Akib Sayyed reported on the OsmocomBB mailing
list almost exactly 2 y ago when he tried enabling AMR in OsmocomBB
without applying any DSP patches.

If we are to continue on the path of trying to make LoCosto-source-based
L1 work on Calypso GSM devices in our current gsm-fw framework, the
logical next step would be to try integrating the dynamic DSP patch
downloading code - as discussed on this list back in July, our TCS211
"golden reference" (leo2moko) loads some dynamic DSP patches on top of
the static one, and we've only got the latter in our current gsm-fw.

I should also note that for some reason our "golden reference" TCS211
fw seems much more resilient to the lack of DSP patches than our
gcc-built one.  When I knocked the dynamic patch downloading out in
leo2moko while keeping the static one, not only was it still able to
make calls, but even the audio still sounded OK to my not-too-discerning
ear.  It may have been slightly degraded compared to the fully patched
version (some artifacts *may* have been introduced, or maybe not - my
ears aren't discerning enough to tell reliably), but it was a far cry
from what we are getting with our current gsm-fw with static patch
only: nothing but noise heard in the earpiece.  And when I knocked the
static patch out as well, voice calls would *sometimes* fail the same
way they failed in our gcc-built fw w/o DSP patches - but other times
they would connect just fine, and the audio sounded fine too!  In
contrast, I never succeeded in dialing a MO call from sans-patches
gcc-built fw, and I only got MT calls to succeed once or twice - and I
didn't have the ability to check the audio at that time.

Getting the dynamic patch download mechanism into our gsm-fw won't be
a trivial "just try it" late-one-evening kind of task.  The l1_dyn_dwl
code in the LoCosto L1 source takes up 6 C modules with a total of
2819 lines of code.  And it won't be as simple as just plopping these
6 C files into our gsm-fw tree and fixing a few compilation hitches:
the version in the LoCosto source supports the dynamic patches needed
for Calypso+ and LoCosto, but not the ones featured in our TCS211
binary libs version.  So once again the binary libs will need to be
disassembled, the logic contained therein will need to be reverse-
engineered, and then reapplied to the available source.

When I am ready to tackle this L1 firmware mess once again (see below),
I am wondering if perhaps I should take a different approach with the
l1_dyn_dwl code.  My previous approach has been to plop bits of code
from the LoCosto source into gsm-fw, get them to compile with gcc, and
have no way of knowing whether or not I got it right - just because
something compiles doesn't mean that it doesn't contain a bad porting
artifact, as we learned painfully with the static patch corruption bug.
I wonder if it might make more sense to take a different approach:
instead of plopping bits of code from LoCosto source into gsm-fw, plop
them into a TCS211 tree instead, and try to use that nasty build system
(which does work under Wine) to compile them with TI's proprietary
compiler.  Tackle just one C module at a time (don't try to port the
whole fw before having any idea if I got it right or not), use the
original TCS211 header files (which we do have), and try to get each
individual C module in question to compile (with TI's original
compiler) into an exact match of the corresponding blob from the
semi-src deliverable we got.

Before trying to recreate the full TCS211 L1 in that manner, try doing
it just for the 6 l1_dyn_dwl_*.c modules.  If we can reconstruct TCS211
source for these 6 modules, then we can try integrating them into our
gcc-built gsm-fw, and see what happens.  If adding this dynamic patch
downloading makes voice calls work beautifully, great.  If we continue
having seemingly-unsolvable problems with this L1, then try
reconstructing the rest of TCS211 L1 in the one C module at a time
manner.  Then try integrating that reconstructed version of L1 into
our gcc-built gsm-fw.  Yes, a slow and painful way, but we may not
have a better alternative.

But I am also unsure if doing the above is what I should do next.  You
see, because there is only one of me, for as long as I am pecking away
at firmware reconstruction and reintegration issues, no progress is
being made at all on the hardware subproject of FreeCalypso.  I realize
that many people here probably want to see libre fw running on Mot C1xx
phones (or on the Pirelli DP-L10) more than anything else, but I have
a broader vision.

The perspective of a developer is different from that of a (potential)
user.  On the one hand, it is reasonable to expect that when we get
some libre fw working on one of Motorola or Pirelli targets, non-
developer users will likely be content with using just those phones
and nothing else.  But the needs of a developer are different.  The
need to be able to play with TCS211 fw in addition to our own will NOT
go away any time soon.  And for the purpose of running TCS211, Mot and
Pirelli phones will *never* be an acceptable replacement for the GTA02
modem.  The only thing that can be a fully satisfactory replacement
for Openmoko devices for the purposes of FreeCalypso *development* (as
opposed to the needs of non-developer users) is our own hardware, the
kind I have been proposing for months now.

Right now our project is critically dependent on Openmoko devices for
development, and because these devices have fully reached unobtainium
status, we have no possibility of attracting any new developers.  Once
again, Mot C1xx or Pirelli DP-L10 may be good enough for a user, but
they can never replace the GTA02 modem for developer purposes.  And on
top of that, this one and only suitable development platform we have
is inconvenient as all hell.  Our own proposed FC GSM development
board will be far more convenient in every way.

Thus I feel a need to spend more time and effort on the hardware
subproject of FreeCalypso.  The current status with the proposed
FCDEV3B board (FreeCalypso development board, triband, based on a
verbatim reuse of Openmoko's modem section at the physical layout
level) is that I got a PADS ECO file made along with the necessary
libraries, allowing a PADS-using and PADS-experienced PCB layout
engineer to finish the job of creating the FCDEV3B PCB from the GTA02.
However, I have not succeeded in finding anyone who would be willing
to do this job for a price I can afford.  The only ones who were
willing to even take a serious look into it were San Diego PCB
(www.sdpcb.com), a fully commercial PCB layout shop charging fully
commercial labor rates.  They said it would take about 40 hours of
work at $95/hour.  In other words, about 4 kUSD.  And that cost is NOT
for physically making the board - it is just for finishing the PCB
design in PADS, i.e., pure desk work, nothing physical.  Whatever it
will cost to actually produce the PCB, that will be an entirely
separate cost afterward.  Needless to say, neither I nor probably our
community as a whole can afford to pay 4 kUSD just for the desk work
of massaging an already existing PCB design into our desired shape.

Thus we need a different approach.  When I first got a hold of the
GTA02 PCB design in PADS ASCII format (as opposed to PADS binary), my
first thought was to write an automated lossless converter from this
PADS ASCII format into some FLOSS PCB format.  The problem, however,
is that the only FLOSS PCB program I am comfortable with (GNU PCB,
more commonly referred to as gEDA/PCB) currently lacks support for
blind & buried vias and a few other common PCB layout features which
are used by the GTA02 design and all other GSM device PCB designs I've
seen.  It should be obvious that an automated lossless translation
from format A into format B can only be possible if B supports all
features of A, or at least all features of A which are actually used
by the source file to be converted.

Thus before we can use a FLOSS PCB layout program to finish the design
of our desired FCDEV3B via translation/import from PADS, we will first
need to *develop* a FLOSS PCB layout tool that has enough PADS-like
capabilities (blind & buried vias, copper pours, pad stacks, arbitrary
drawing layers) in order to be able to import a reuse of a commercial
GSM device PCB layout that was done in PADS.  Thus I feel that the
right course of action for me at this point is to embark on a project
of creating an improved version of GNU PCB with the just-listed PADS-
like capabilities.  Comments and suggestions are welcome.

Happy hacking,
Mychaela aka Space Falcon