--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/RVTMUX	Sat Jun 11 00:13:35 2016 +0000
@@ -0,0 +1,236 @@
+TI's Calypso GSM/GPRS baseband processor chip has not one but two UART serial
+ports, called "MODEM" and "IrDA" in the hardware documentation.  In hardware
+terms, both support basic data-leads-only UART operation at a fixed baud rate,
+but their extended capabilities differ: the IrDA UART adds IrDA capability (no
+surprise), whereas the MODEM UART adds hardware flow control and autobaud.  If
+one is not implementing an actual IrDA interface, then the so-called "IrDA"
+UART becomes a strict subset of the MODEM one in terms of hw capabilities -
+just an extra UART, but a somewhat less capable one.
+
+In a classic modem design such as that present in the GTA0x smartphones made by
+FIC/Openmoko, the Calypso presents a standard AT command interface on its MODEM
+UART port.  (In the case of GTA0x phones this serial channel is wired to the
+phone's application processor; in a standalone modem it would be wired to a
+USB-serial chip or even to a classic RS-232 DB25 port.)  However, what is less
+known is that the standard firmware for such modems simultaneously presents an
+entirely different interface on the IrDA UART - an interface intended for
+development, debugging and factory production testing (which includes RF
+calibration and IMEI etc programming), rather than for "normal" end users.
+
+Normally this debug/development serial interface (called RVTMUX as will be
+explained momentarily) is hidden from "ordinary" users - for example, on FIC
+GTA0x phones it is wired to the analog headset jack through a hardware switch
+which needs to be enabled through a GPIO signal from the AP.  On Mot C139 and
+its siblings the situation is similar: one needs to key in the secret magic
+sequence **16379#, and then the firmware presents a hidden menu for switching
+the analog headset jack between its "normal" function and the UART carrying
+RVTMUX.
+
+But there also exist some oddball devices on which the RVTMUX interface is
+presented "in your face".  The Pirelli DP-L10 phone has a USB charging port
+which is also wired (through a CP2102 USB-serial chip) to the IrDA UART on the
+Calypso - that's right, IrDA, not MODEM - a design decision with which this
+hacker strongly disagrees.  (It'll definitely be wired to the MODEM UART
+instead on our own semi-clone of this phone, but I digress.)  Apparently Foxconn
+(the designers of this phone) had no desire to provide a standard AT command
+interface, and instead the only "official" way to use the "data" function of
+their USB port (rather than the charging function) is for their "PC sync"
+feature, i.e., their proprietary Weendoze software.  And guess what, their
+proprietary "PC sync" feature works over TI's RVTMUX interface, as that is
+what's presented on Calypso's IrDA UART behind the CP2102!
+
+OK, so what is this RVTMUX?  RV stands for RiViera, an application framework
+which TI added to their GSM firmware suite in the early 2000s, T stands for
+trace, and MUX stands for multiplexor.  It's a binary packet interface, although
+many of these packets contain ASCII debug messages inside.  The framing format
+is the same in both directions: each packet begins and ends with an STX (0x02)
+byte, all payload bytes except 0x02 and 0x10 are sent literally, and there is a
+DLE (0x10) byte prepended before any 0x02 or 0x10 in the payload.  It's the same
+general principle as asynchronous HDLC (RFC 1662): packets can contain any
+binary data, and the framing provides packet boundaries - although TI's version
+is a little less robust than async-HDLC when it comes to recovering after lost
+synchronization.
+
+The firmware suite component responsible for actually sending and receiving
+these packets over the assigned UART port (usually IrDA, but can be MODEM too,
+as on Compal phones for example) is called RVT (RiViera Trace), and it
+implements a MUX function.  There are several logical channels multiplexed over
+one physical serial port, and the first byte of every packet indicates which
+logical channel it belongs to.  Any component within the GSM firmware suite can
+send packets to RVT for transmission on this serial interface, and can also
+register to receive packets beginning with a particular type ID byte.
+
+Debug trace output
+==================
+
+All GSM device firmwares that are based on TI's Calypso chipset reference fw
+continuously emit quite voluminous debug trace output on their RVTMUX serial
+port, whether it is hidden or exposed on a given device.  Like all RVTMUX
+traffic, this debug trace output takes the form of binary packets as explained
+above, but the content of these packets is mostly ASCII with some binary header
+bytes prepended.  FreeCalypso host utility rvtdump captures all serial output
+from a GSM device's RVTMUX port, parses the packet structure and displays this
+output in line-oriented pure ASCII with all binary parts decoded.
+
+Test Mode commands
+==================
+
+The other major use of the RVTMUX interface is sending so-called Test Mode
+commands from an external host to a running GSM device.  Depending on the
+firmware version, a GSM device can be commanded to do any of the following
+things through this mechanism:
+
+* Exercise RF test modes, e.g., transmit continuously at a set frequency and
+  power level;
+* Read and write arbitrary memory locations in the Calypso ARM7 address space;
+* Read and write ABB chip registers;
+* Reboot or power off;
+* Access and manipulate the device's flash file system (FFS).
+
+In the segment of history of interest to us TI has produced two different
+target firmware components that can receive, interpret and act upon Test Mode
+command packets:
+
+* The original Test Mode component of Layer 1, called L1TM or TML1: this
+  component handles all RF test modes (needed for RF calibration on device
+  production lines), and originally it also implemented memory and ABB register
+  read and write commands, and provided access to TMFFS1 (see below).  In the
+  original implementation this component registered itself as the handler for
+  the "TM" RVTMUX channel (RVT packet type 0x14), so it would receive all TM
+  packets sent to the device.
+
+* Enhanced Test Mode (ETM) is a later invention.  It registers itself (instead
+  of the old TM in L1) with RVT as the handler for the "TM" RVTMUX channel, and
+  then provides a registration service of its own, such that various components
+  in the fw suite can register to receive external command packets passing
+  first through RVT, then through ETM, and can send responses passing through
+  ETM, then through RVT back to the external host.  If a given fw version
+  contains both ETM and L1TM, then L1TM registers itself with ETM; an external
+  host would send exactly the same binary command packets to exercise RF test
+  modes, but inside the firmware they now pass through ETM on their way to L1TM.
+
+The ETM_CORE module contained within ETM itself provides some low-level debug
+commands: by sending the right binary command packets to the GSM device via the
+RVTMUX serial channel, an external host can examine or modify any memory
+location and any hardware register, cause the device to reset, etc.  Prior to
+ETM some of these functions (but not all) could be exercised through older TM3
+commands, but in FreeCalypso we became familiar with the ETM versions of these
+commands long before the older ones because we got the ETM component in full
+source form, whereas our copy of TCS211 (TI's reference fw) has L1TM in a
+binary library.
+
+Our TCS211/leo2moko reference fw has both ETM and L1TM, thus it accepts both
+ETM and TM3 command packets.  ETM commands (including TMFFS2, see below) work
+on Pirelli's fw, but Mot/Compal's original fw for the C139 has only the
+original non-enhanced Test Mode, not ETM.
+
+FFS access via TM/ETM
+=====================
+
+One of the essential facilities provided in one form or another in all known
+incarnations of the Test Mode mechanism is the ability to access and manipulate
+the GSM device's flash file system (FFS).  See TIFFS-Overview for a description
+of this file system.  TI's TMFFS1 and TMFFS2 protocols provide a command and
+response packet interface to the FFS API functions inside the fw, and enable an
+external host connected to the GSM device via the RVTMUX channel to perform
+arbitrary read and write operations on the device file system.
+
+In the segment of history of interest to us TI has produced two different
+and entirely incompatible versions of the TMFFS protocol: TMFFS1 and TMFFS2.
+Or rather, what is now called TMFFS1 was originally just TMFFS, and then came
+TMFFS2.  TMFFS2 works only through ETM, whereas TMFFS1 predates ETM: in the
+original implementation the tm_ffs() function in the FFS code was called from
+L1TM code.
+
+Our copy of TCS211 reference fw includes the source for both TMFFS1 and TMFFS2;
+it is theoretically possible to build a firmware image that includes both TMFFS
+versions (they won't conflict because they respond to different command
+packets), but it is pretty clear that TI never intended to have both enabled
+at the same time.  Our copy of TCS211 came with TMFFS1 enabled and we didn't
+change it when we made the moko12 (leo2moko-r1) fw release for the Openmoko
+community (the previous proprietary mokoN firmwares also implement TMFFS1),
+but we have subsequently switched to TMFFS2 for our current TCS211-based work.
+
+Pirelli's fw implements TMFFS2: we don't have any source for this fw, but our
+FreeCalypso host utilities written to talk the TMFFS2 protocol based on our
+available TCS211 source work beautifully when run against Pirelli's fw.
+
+Use in FreeCalypso
+==================
+
+The FreeCalypso project has adopted the same general firmware architecture as
+that exhibited by TI's standard firmwares from the Moko/Pirelli time frame.  We
+use TI's RiViera framework lifted directly out of the TCS211 reference fw, and
+that includes the RVT module and the RVTMUX interface it presents.  Our GSM fw
+emits the same 3 kinds of debug traces (RV, L1 and GPF) as the pre-existing
+firmwares with which we are seeking functional parity, and for Test Mode
+functionality we have the option of including ETM, TMFFS1 and/or TMFFS2 in our
+firmware builds.  (Both TMFFS versions require ETM in our implementation, and
+it is possible to build a firmware image with both included.)
+
+We have adopted ETM and TMFFS2 as the standard combination for FreeCalypso,
+i.e., ETM_CORE for memory and ABB register reads and writes and TMFFS2 for
+external FFS access.  We needed to develop our own host tools for operating on
+GSM device FFS via one of the two TMFFS protocols, and after studying the fw
+source implementing both, I (Space Falcon) came to the conclusion that TMFFS2
+is both more capable and more reliable; my guess is that TMFFS1 was likely kept
+around only because some of TI's crappy Weendoze host software depended on it.
+(See gsm-fw/services/ffs/tmffs.c if you would like to judge for yourself.)
+
+We have the following host tools for communicating with TI-based GSM firmwares
+(both our own and some of the existing proprietary ones):
+
+rvtdump		This tool produces a human-readable dump of all output emitted
+		by a TI-based GSM fw in the form of RVTMUX binary packets.  It
+		can also log this dump to a file.
+
+rvinterf	This tool is a superset of rvtdump: it not only dumps and/or
+		logs all output from the GSM fw, but also provides a mechanism
+		for sending command packets to it.
+
+Rvinterf is the engine behind the following host tools that send Test Mode
+commands to a target:
+
+fc-tmsh		This is our basic tool for sending Test Mode commands to a
+		running GSM fw.  It is strictly asynchronous in that commands
+		entered by the operator get sent to the target, and any response
+		packets received from the target are displayed as they come in.
+		The tool has no knowledge of any correspondence between commands
+		being sent and whatever responses they should elicit, i.e., it
+		is perfectly suited for experimental discovery of firmware
+		behaviour in response to Test Mode commands.
+
+		This tool was written before we realized that there was/is an
+		older, more basic Test Mode predating ETM, hence in many place
+		we say "ETM" when we really should have said "TM".  Oh well...
+
+fc-fsio		This tool speaks the TMFFS2 protocol and allows a user or
+		developer to perform a wide range of operations on the file
+		system of a GSM device.  It operates synchronously, i.e., it
+		sends ETM/TMFFS2 commands and expects responses in strict
+		lock-step; a single user command may translate into a large
+		number of ETM/TMFFS2 command packet exchanges.
+
+AT commands over RVTMUX
+=======================
+
+There is one more use to which we put the RVTMUX debug serial interface that is
+an original FreeCalypso invention: communicating with the AT command interpreter
+(ATI).  TI's original architecture assumes that if a product is to offer a
+standard AT command interface (the product is either a GSM/GPRS modem for which
+this AT command interface is the sole mode of usage or a feature phone that
+offers a data port as one of its features), then it will be presented on a
+dedicated UART separate from RVTMUX.
+
+However, many of our target devices have only one UART practically accessible,
+and even when we use Openmoko's modem as our development platform, the RVTMUX
+interface is more convenient because it connects externally, whereas the MODEM
+UART is connected to the application processor of the smartphone.  Therefore,
+we developed a way to pass AT commands over RVTMUX.  We created a new RVTMUX
+channel for this interface and assigned it RVT packet type 0x1A.  Packets sent
+from an external host to the GSM device carry AT commands and SMS string input,
+whereas packets flowing the other way carry ATI's responses to commands and
+asynchronous notifications such as incoming calls.
+
+The host utility for talking AT commands to a FreeCalypso GSM device via RVTMUX
+is fc-shell; it works via rvinterf just like fc-fsio and fc-tmsh.