changeset 373:9399a83cb394

first round of documentation updates for 2018
author Mychaela Falconia <falcon@freecalypso.org>
date Thu, 11 Jan 2018 22:44:09 +0000
parents c389d938a50e
children 3f2dce15278c
files README doc/Compiling doc/Freerunner-Howto
diffstat 3 files changed, 103 insertions(+), 104 deletions(-) [+]
line wrap: on
line diff
--- a/README	Mon Jan 01 19:06:34 2018 +0000
+++ b/README	Thu Jan 11 22:44:09 2018 +0000
@@ -1,31 +1,27 @@
 FreeCalypso Magnetite firmware project
 ======================================
 
-This source tree contains yet another firmware offering created under the
-FreeCalypso umbrella.  The key qualities of this firmware offering are:
-
-Negatives:
+As of A.D. 2018, FC Magnetite is the primary official Calypso firmware source
+tree for the FreeCalypso family of projects.  This maintained and evolving
+source can be built in many different configurations for several different
+hardware targets, and is currently used in the following ways:
 
-  * Builds with TI's proprietary TMS470 compiler and thus requires Wine;
-  * Some of the fw components are still in the form of binary blobs -
-    but see below on the deblobbing progress.
-
-Positives:
+* The official production firmwares for our fully supported GSM+GPRS modem
+  products (FCDEV3B and the modem part of Openmoko devices) are built from
+  this source tree.  See doc/Modem-configs for this mode of usage.
 
-  * Officially supports our own FCDEV3B and Openmoko GTA01/02 modem targets,
-    as well as "hack-only" support for Motorola C139 and Pirelli DP-L10,
-    all from the same source tree;
+* The work in progress toward a future complete FreeCalypso phone handset
+  (also known as a Libre Dumbphone) is being done in this source tree.
+  See doc/Handset-goal for more info about this project direction.
 
-  * Works as solidly as the TCS211 "golden" reference from TI, on all of the
-    supported targets - deep sleep works (on non-broken hardware), voice calls
-    work in all codec modes including AMR, the DSP dynamic download mechanism
-    does its magic, the call audio passes reliably in both directions.
-
-The present FC Magnetite firmware is built on the principle of starting with
-the known working TCS211 code base, without any major restructuring, and making
-small incremental evolutionary changes, testing at every step to ensure that
-nothing breaks.  It is the direct opposite of the "rebuild from the ground up"
-approach taken with our previous Citrine firmware aka "gsm-fw".
+* One can play with FreeCalypso and get a taste for it using certain hardware
+  that used to be readily available and which most people in the phone hacking
+  scene already have: Motorola C139 and Pirelli DP-L10.  Firmware images of the
+  "hacking toy" sort can be built for both of these models from the present
+  source tree, and one can exercise most of the functions and capabilities of
+  FreeCalypso (with the notable exception of CSD and GPRS) on this historical
+  but available-to-most-people hardware.  See doc/C139-Howto, doc/Pirelli-Howto
+  and doc/Voice-pseudo-modem for more information.
 
 Functionality
 =============
@@ -36,40 +32,25 @@
 two mechanisms can coexist.)  The code we got from TI (TCS211) is very solid
 and mature in the modem configuration (control via AT commands only, no UI, no
 battery management, no traditional handset on/off control), but the additional
-code layers that are needed for handset products but not for modems are in a
+code layers that are needed for handset products but not for modems came in a
 very rough "proof of concept" condition, nowhere close to a usable product.
 
-The same situation currently exists in FreeCalypso Magnetite.  As explained
-below in the Build system section, a key feature of FC Magnetite is that many
-different firmware configurations can be built from the same source tree.  One
-of the configuration choices is that you can build the fw either with or without
-the phone UI layers.  The work we have done earlier in the tcs211-c139 side
-project (late 2015) has been integrated into Magnetite, properly conditionalized
-so that TI's original configuration is fully preserved when the target != c139.
-However, if you build the UI-enabled configuration for any target other than
-Mot C139, it will compile, but it won't do anything useful: it will try to
-display TI's 176x220 pixel color UI on the D-Sample LCD on Calypso chip select
-nCS3, but this hardware doesn't exist on any of our supported targets.  (The
-Pirelli in particular has flash on that chip select instead, so it may interfere
-with FFS operation.)
+In FC Magnetite we further maintain and support the solid code base we got for
+the modem functionality, and we are also working to improve the support for
+handset products and bring it into a practically usable state.  We have already
+implemented an entirely new battery charging and discharge monitoring driver
+that actually works on our target hw (of the two we got from TI, one was
+bitrotten and the other was designed for charging hardware that is quite
+different from what we are working with), and we shall hopefully start working
+on the UI soon - see doc/Handset-goal for more info.
 
-Further work on the handset firmware configuration (UI, battery management and
-other currently missing functionality required for a usable phone) will have to
-wait until we build our own Handset Motherboard Prototype (HSMBP) with a 176x220
-pixel 16-bit color LCD, replicating TI's D-Sample - I personally am not too
-interested in doing this handset fw work on the very crippled Mot C139 or on
-the Pirelli with its own host of issues, although others in the community are
-more than welcome to take a shot at it if someone is interested.
-
-Until then, the primary current focus of the FC Magnetite project is the
-AT-command-controlled modem configuration.  Both TI's original modem fw (TCS211)
-and our recreation thereof in this Magnetite project support not only voice
-calls and SMS, but also CSD, fax and GPRS.  This advanced functionality is
-fully supported on our own GSM MS development board (FCDEV3B), where both
-Calypso UARTs are presented directly to the developer, as well as on the
-embedded Calypso modem in Openmoko GTA01/02 smartphones, where the AT command
-channel with CSD, fax and GPRS capabilities is connected to the phone's
-application processor.
+Both TI's original modem fw (TCS211) and our recreation thereof in this
+Magnetite project support not only voice calls and SMS, but also CSD, fax and
+GPRS.  This advanced functionality is fully supported on our own GSM MS
+development board (FCDEV3B), where both Calypso UARTs are presented directly to
+the developer, as well as on the embedded Calypso modem in Openmoko GTA01/02
+smartphones, where the AT command channel with CSD, fax and GPRS capabilities
+is connected to the phone's application processor.
 
 Build system
 ============
@@ -117,35 +98,62 @@
 As yet another defenestration measure, all source files are checked into this
 tree with Unix line endings.
 
-Blob status
+Two versions of the G23M protocol stack
+=======================================
+
+A major component of all functional firmware configurations is the mass of code
+that implements layers 2 and 3 of the GSM+GPRS protocol stack, called G23M.  In
+FC Magnetite we have the option of using one of two different versions of this
+key firmware component:
+
+* The original G23M version from Openmoko: this version is believed to be very
+  stable because it has been used (successfully to the best of our knowledge)
+  by Openmoko, and it came from TI's TCS211 program that officially targeted
+  the Calypso chipset - but it exists only as binary object libraries with no
+  corresponding source.  The lack of source means no ability to change the
+  feature configuration (what is enabled and what is disabled), very difficult
+  to debug if something does go wrong - all of the usual problems of software
+  without source.
+
+* A newer version from TI's TCS3/LoCosto program.  This version came to us in
+  the form of full C source, and because this fw component is chipset-
+  independent (unlike L1), we have successfully produced a TCS2/TCS3 hybrid in
+  which the new G23M code from TCS3.2 is grafted onto the chipsetsw foundation
+  from TCS211, which we have reconstructed into full C source form as well.
+
+Each of the two G23M PS versions also has its own version of the Application
+Control Interface (ACI) layer to go with it, which we call aci2 and aci3 -
+except that in the case of ACI, we have the full source for both versions.
+
+The new TCS2/TCS3 hybrid config is the way forward, as our goal is to have no
+blobs in our firmware - having the full source is a prerequisite for maintaining
+a software product.  FC Magnetite supports building both configurations in order
+to facilitate the transition, and as of this writing, we still need to add a few
+bits to the new version of ACI in order to achieve 100% feature parity with
+Openmoko.
+
+Other blobs
 ===========
 
-A long-term FreeCalypso goal is to have our phone/modem firmware rebuild fully
-from source without any blobs, but this goal has not been achieved yet.  While
-we do have what *seems* to be a suitable replacement source (or feasible ability
-to reconstruct such) for every piece of TCS211 fw that came in binary-only form,
-actually making this replacement without breaking functionality is a very
-non-trivial endeavor.  Our previous attempt to rebuild the firmware from the
-ground up, using source pieces lifted from different available leaks and
-building with gcc so that no TMS470 COFF blobs could be used - see FreeCalypso
-Citrine - has produced only a very limited subset of the original functionality,
-and until very recently even the most basic function of voice calls did not work
-reliably.
+The TCS2/TCS3 hybrid firmware is built almost entirely from source; the only
+components which are linked in the form of prebuilt libraries are GPF, Nucleus
+and the TMS470 compiler's equivalent of libc/libgcc.  It needs to be noted that
+these components are so stable and configuration-independent that they were
+used mostly in prebuilt library form even inside TI.  In the case of GPF we've
+got the corresponding source for most of the modules within those libs; for the
+few modules for which the original source has been lost, we've already done a
+rough first-draft reconstruction as part of our previous gcc-built GSM fw
+efforts, and we only need to polish this reconstruction in order to have
+deblobbed GPF in our mainline production fw.  In the case of Nucleus we likewise
+have another version of it in full source form which has been proven good in
+other FreeCalypso firmware projects.
 
-Instead the new FreeCalypso firmware approach implemented in FC Magnetite is to
-approach the blob-free goal incrementally.  The new Magnetite build system is
-specifically designed to enable the transition from the use of blobs to
-recompilation from source to be made with very fine granularity, down to the
-level of individual object modules within libs if necessary.  We tackle one
-binary-only component at a time, either reconstructing the missing source from
-disassembly or adapting the source from a different version as works best in
-each individual case, and we make a test build of the firmware using the
-reconstructed or fitted component instead of the original blob.  If the firmware
-still works (doesn't break), we make this deblobbing transition permanent and
-move on to the next component.
-
-See doc/Modem-configs for the current status of the deblobbing effort and for
-the description of the currently available configurations.
+The planned successor to FC Magnetite (tentative planned name Selenite) will
+use the hybrid config for the G23M PS, deblobbed GPF and the source-enabled
+version of Nucleus, resulting in blob-free firmware except for the proprietary
+TMS470 compiler and its equivalent of libc/libgcc.  Transition to building with
+gcc (like in FreeCalypso Citrine) will follow afterward, completely exiting the
+land of blobs and proprietary build tools.
 
 Further reading
 ===============
--- a/doc/Compiling	Mon Jan 01 19:06:34 2018 +0000
+++ b/doc/Compiling	Thu Jan 11 22:44:09 2018 +0000
@@ -4,7 +4,7 @@
 In order to compile our FreeCalypso Magnetite firmware, you will need a
 Unix/Linux system.  Even though we are using a compiler which we got in the
 form of Windows .exe binaries and thus have to use Wine (see below), everything
-that we have built on top of it is Unix-based.  This author uses Slackware Linux
+that we have built on top of it is Unix-based.  The Mother uses Slackware Linux
 version 13.37, 32-bit.
 
 You will need to install the following four pieces of software on whatever
@@ -71,8 +71,7 @@
 pirelli		Pirelli DP-L10
 
 For the available configurations (the second required argument to the configure
-script), look in the configs directory and read the Handset-configs and
-Modem-configs write-ups.
+script), look in the configs directory and read the various write-ups under doc.
 
 Each configuration is built in its own directory; by default the build directory
 is named build-$TARGET-$CONFIG, i.e., for the example configure.sh line above,
--- a/doc/Freerunner-Howto	Mon Jan 01 19:06:34 2018 +0000
+++ b/doc/Freerunner-Howto	Thu Jan 11 22:44:09 2018 +0000
@@ -1,27 +1,19 @@
-Running FreeCalypso Magnetite firmware on the Neo Freerunner
-============================================================
-
-As explained in the Modem-configs article, we currently have 3 different modem
-fw configurations which differ in the degree of deblobbing.  The classic
-configuration is exactly the same as leo2moko from 2013; the l1reconst config
-rebuilds most of L1 from source, but our L1 reconstruction is so accurate that
-there will be no observable difference in behaviour.  The hybrid configuration,
-however, is quite interesting in that it is a new major development, as major
-parts of the fw have been replaced with an entirely new version from another TI
-program.
+The latest official firmware for Openmoko Neo1973 and Neo FreeRunner Calypso
+modems is moko13 as of this writing; moko13 is FreeCalypso Magnetite built for
+the gtamodem target in the l1reconst configuration.  There have not been any
+changes in FC Magnetite since the moko13 release that affect the l1reconst
+config on the gtamodem target in any noticeable way, hence there is no need for
+a new release currently.
 
-The deblobbed TCS2/TCS3 hybrid configuration is intended to be our direction
-going forward, but it will need extensive testing and debugging before it can
-replace the classic/l1reconst configs (or older mokoN/leo2moko firmwares) for
-production use.  The principal developer's intent is to build our desired
-FCDEV3B hardware and do this testing and debugging on that board, but it can
-also be done on Openmoko devices.
+However, we also have the new TCS2/TCS3 hybrid config in which the old version
+of the G23M protocol stack from Openmoko (binary libs only, no source) has been
+replaced with a newer version from TI's TCS3/LoCosto program, and this new
+version is full source.  This hybrid firmware has not yet reached mokoN release
+status because some obscure features of the old ACI have not yet been
+reimplemented in the new one, but in every other way this hybrid fw is ready to
+be tested and exercised in real usage.
 
-Given the current state of the remains of what once was the Openmoko community,
-I (Mychaela) do not feel like doing any significant work on this platform;
-instead I would rather build our own FCDEV3B and leave Openmoko behind.
-However, if anyone else feels otherwise and would like to play with our
-TCS2/TCS3 hybrid modem fw on Openmoko hardware, you can build it like this:
+The new hybrid fw for the gtamodem target can be built as follows:
 
 ./configure.sh gtamodem hybrid
 cd build-gtamodem-hybrid; make