Firmware bring-up status

[Spacefalcon the Outlaw]

Background (David and DS already know it, but I repeat it here for the
newly joined listmembers and for the archive): within this past week
the gsm-fw subproject advanced to the point of having a firmware image
for the gtamodem target that contains everything that should be needed
to bring up GSM functionality, i.e., the GSM protocol stack, the AT
command interface and all of their underlying dependencies.  But
unlike leo2moko from 2013-10, this one is built with gcc from full
source, and the versions of the G23M protocol stack and TI Layer 1 are
ones which never ran on the Freerunner or on any other known Calypso
device before, taken from TI's LoCosto aka TCS3.2 source.

Having this GSM firmware image *built* (as in link passing with all
dependencies satisfied) is the culmination of the past year and a half
of work (measured from the fall of 2013 when I obtained the last
required piece of starting-point material), but unfortunately getting
the fw image built is just the first half of the job, and now we enter
the second stage: getting it to actually work.

Right now one can download the current freecalypso-sw source from
bitbucket, compile gsm-fw for the gtamodem-gsm configuration
(cd gsm-fw; cp configs/gtamodem-gsm build.conf; make) and flash the
resulting image (gsm-fw/finlink/flashImage.bin) into a GTA02 modem
with fc-loadtool.  However, this firmware is currently broken in
(at least) two ways:

1. There is some weird interaction happening with some hardware issue,
   it seems.  If you power the modem off after flashing this fw, then
   take a bathroom or coffee break, and then power it back on, the fw
   boots fine and presents a live AT command interface on the MODEM
   UART (the one connected to the Freerunner's Linux AP).  But if you
   power the modem on with this experimental fw running, then power it
   off, then power it back on without inserting an unnaturally long
   delay, the firmware crashes on boot!

   This non-deterministic behavior that depends on what the modem was
   doing *before a power cycle* is utterly mind-blowing.  It definitely
   seems like some kind of hardware issue (temperature or stored
   electrical charge or somesuch - what else can persist across a
   power cycle?), but the firmware has to be at least partially at
   fault too, as none of the Windows-built firmwares (neither mokoN
   nor leo2moko) exhibit this behavior.

   If anyone here would like to see this behavior themselves, you
   would need 3 things:

   * A Freerunner you can play with;
   * A headset jack serial cable: you can flash fw images from inside
     the FR without this cable, but you'll need it to see the trace
     output from the fw as it either boots successfully or crashes;
   * At least a basic understanding of the rvtdump output, like what
     is normal and what isn't.

   If anyone here satisfies the above criteria and would like to join
   in the debug investigation, I'll provide more detailed instructions.

   Oh, and I wrote earlier (to David and DS before we got this mailing
   list up) that the firmware was crashing on boot when I enabled
   access to the real FFS (flash file system) in the modem's flash
   instead of using fake FFS in RAM.  That report was erroneous: it
   just so happened that I first observed the weird "crash on boot
   when the modem is hot" behavior when I tried booting an image built
   with feature mokoffs enabled.  Subsequent experiments revealed that
   whether the fw will boot successfully or crash on boot depends on
   the power cycling as described above, and not at all on whether the
   fw image was built to use the real FFS or a fake one.

2. When the experimental fw does boot without crashing, the AT command
   interface is live, but I can't get far enough to read things from
   the SIM or turn the radio on.  With the stable TCS211 fw (leo2moko),
   one begins a session with AT+CFUN=1; that command comes back with
   an OK response, and then you can issue AT+CNUM and see your own
   phone number, or issue AT+CIMI and see your SIM's IMSI.

   But with our experimental fw both AT+CFUN=1 and AT+CFUN=4 respond
   with ERROR, and no further progress can be made toward bringing the
   GSM functionality up.

Right now I'm investigating problem #2 (AT+CFUN not working when the
boot cycle is successful) as it seems a little less daunting than
problem #1 (mind-boggling crashes when the modem is "hot" from a
previous power cycle).

Hasta la Victoria, Siempre,
SF