# HG changeset patch # User Mychaela Falconia # Date 1447551770 0 # Node ID 285505f9801328f6361b06338b893e74a394810d # Parent 6de5661d1fe1dfbc741db8a0b984fb01eebcfedb doc/RVTMUX: major updates triggered by new understanding of TM/ETM relationship diff -r 6de5661d1fe1 -r 285505f98013 doc/RVTMUX --- a/doc/RVTMUX Fri Nov 13 19:29:16 2015 +0000 +++ b/doc/RVTMUX Sun Nov 15 01:42:50 2015 +0000 @@ -60,83 +60,177 @@ send packets to RVT for transmission on this serial interface, and can also register to receive packets beginning with a particular type ID byte. +Debug trace output +================== + +All GSM device firmwares that are based on TI's Calypso chipset reference fw +continuously emit quite voluminous debug trace output on their RVTMUX serial +port, whether it is hidden or exposed on a given device. Like all RVTMUX +traffic, this debug trace output takes the form of binary packets as explained +above, but the content of these packets is mostly ASCII with some binary header +bytes prepended. FreeCalypso host utility rvtdump captures all serial output +from a GSM device's RVTMUX port, parses the packet structure and displays this +output in line-oriented pure ASCII with all binary parts decoded. + +Test Mode commands +================== + +The other major use of the RVTMUX interface is sending so-called Test Mode +commands from an external host to a running GSM device. Depending on the +firmware version, a GSM device can be commanded to do any of the following +things through this mechanism: + +* Exercise RF test modes, e.g., transmit continuously at a set frequency and + power level; +* Read and write arbitrary memory locations in the Calypso ARM7 address space; +* Read and write ABB chip registers; +* Reboot or power off; +* Access and manipulate the device's flash file system (FFS). + +In the segment of history of interest to us TI has produced two different +target firmware components that can receive, interpret and act upon Test Mode +command packets: + +* The original Test Mode component of Layer 1, called L1TM or TML1: this + component handles all RF test modes (needed for RF calibration on device + production lines), and originally it also implemented memory and ABB register + read and write commands, and provided access to TMFFS1 (see below). In the + original implementation this component registered itself as the handler for + the "TM" RVTMUX channel (RVT packet type 0x14), so it would receive all TM + packets sent to the device. + +* Enhanced Test Mode (ETM) is a later invention. It registers itself (instead + of the old TM in L1) with RVT as the handler for the "TM" RVTMUX channel, and + then provides a registration service of its own, such that various components + in the fw suite can register to receive external command packets passing + first through RVT, then through ETM, and can send responses passing through + ETM, then through RVT back to the external host. If a given fw version + contains both ETM and L1TM, then L1TM registers itself with ETM; an external + host would send exactly the same binary command packets to exercise RF test + modes, but inside the firmware they now pass through ETM on their way to L1TM. + +The ETM_CORE module contained within ETM itself provides some low-level debug +commands: by sending the right binary command packets to the GSM device via the +RVTMUX serial channel, an external host can examine or modify any memory +location and any hardware register, cause the device to reset, etc. Prior to +ETM some of these functions (but not all) could be exercised through older TM3 +commands, but in FreeCalypso we became familiar with the ETM versions of these +commands long before the older ones because we got the ETM component in full +source form, whereas our copy of TCS211 (TI's reference fw) has L1TM in a +binary library. + +Our TCS211/leo2moko reference fw has both ETM and L1TM, thus it accepts both +ETM and TM3 command packets. ETM commands (including TMFFS2, see below) work +on Pirelli's fw, but Mot/Compal's original fw for the C139 has only the +original non-enhanced Test Mode, not ETM. + +FFS access via TM/ETM +===================== + +One of the essential facilities provided in one form or another in all known +incarnations of the Test Mode mechanism is the ability to access and manipulate +the GSM device's flash file system (FFS). See TIFFS-Overview for a description +of this file system. TI's TMFFS1 and TMFFS2 protocols provide a command and +response packet interface to the FFS API functions inside the fw, and enable an +external host connected to the GSM device via the RVTMUX channel to perform +arbitrary read and write operations on the device file system. + +In the segment of history of interest to us TI has produced two different +and entirely incompatible versions of the TMFFS protocol: TMFFS1 and TMFFS2. +Or rather, what is now called TMFFS1 was originally just TMFFS, and then came +TMFFS2. TMFFS2 works only through ETM, whereas TMFFS1 predates ETM: in the +original implementation the tm_ffs() function in the FFS code was called from +L1TM code. + +Our copy of TCS211 reference fw includes the source for both TMFFS1 and TMFFS2; +it is theoretically possible to build a firmware image that includes both TMFFS +versions (they won't conflict because they respond to different command +packets), but it is pretty clear that TI never intended to have both enabled +at the same time. Our copy of TCS211 came with TMFFS1 enabled and we didn't +change it when we made the moko12 (leo2moko-r1) fw release for the Openmoko +community (the previous proprietary mokoN firmwares also implement TMFFS1), +but we have subsequently switched to TMFFS2 for our current TCS211-based work. + +Pirelli's fw implements TMFFS2: we don't have any source for this fw, but our +FreeCalypso host utilities written to talk the TMFFS2 protocol based on our +available TCS211 source work beautifully when run against Pirelli's fw. + Use in FreeCalypso ================== The FreeCalypso project has adopted the same general firmware architecture as that exhibited by TI's standard firmwares from the Moko/Pirelli time frame. We use TI's RiViera framework lifted directly out of the TCS211 reference fw, and -that includes the RVT module and the RVTMUX interface it presents. At the -present time (early development stage, none of the actual GSM functionality has -been integrated yet) this RVTMUX interface is put to the following uses in our -own gsm-fw: - -* Debug trace output from various components sent via the rvf_send_trace() - function - it is the RiViera Trace output in the proper sense; - -* The ETM module and the associated FFS access protocol described below. +that includes the RVT module and the RVTMUX interface it presents. Our GSM fw +emits the same 3 kinds of debug traces (RV, L1 and GPF) as the pre-existing +firmwares with which we are seeking functional parity, and for Test Mode +functionality we have the option of including ETM, TMFFS1 and/or TMFFS2 in our +firmware builds. (Both TMFFS versions require ETM in our implementation, and +it is possible to build a firmware image with both included.) -In the existing proprietary firmwares which serve as our reference, the RVTMUX -serial channel is continuously spewing very voluminous debug output. This debug -output exhibits 3 different packet types: RV traces described above, and also -L1 and G23 traces, each in its own format. We expect that our own gsm-fw will -become just like these reference versions in this regard, once we integrate -those code layers. +We have adopted ETM and TMFFS2 as the standard combination for FreeCalypso, +i.e., ETM_CORE for memory and ABB register reads and writes and TMFFS2 for +external FFS access. We needed to develop our own host tools for operating on +GSM device FFS via one of the two TMFFS protocols, and after studying the fw +source implementing both, I (Space Falcon) came to the conclusion that TMFFS2 +is both more capable and more reliable; my guess is that TMFFS1 was likely kept +around only because some of TI's crappy Weendoze host software depended on it. +(See gsm-fw/services/ffs/tmffs.c if you would like to judge for yourself.) -ETM and FFS access -================== +We have the following host tools for communicating with TI-based GSM firmwares +(both our own and some of the existing proprietary ones): + +rvtdump This tool produces a human-readable dump of all output emitted + by a TI-based GSM fw in the form of RVTMUX binary packets. It + can also log this dump to a file. -Another component which we have lifted out of the TCS211 reference fw is ETM, -which stands for Enhanced Test Mode. This module registers its own "top level" -protocol over RVTMUX, and provides a registration service of its own, such that -various components in the fw suite can register to receive external command -packets passing first through RVT, then through ETM, and can send responses -passing through ETM, then through RVT back to the external host. +rvinterf This tool is a superset of rvtdump: it not only dumps and/or + logs all output from the GSM fw, but also provides a mechanism + for sending command packets to it. + +Rvinterf is the engine behind the following host tools that send Test Mode +commands to a target: -The ETM_CORE module contained within ETM itself provides some low-level debug -commands: by sending the right binary command packets to the GSM device via the -RVTMUX serial channel, an external host can examine or modify any memory -location and any hardware register, cause the device to reset, etc. +fc-tmsh This is our basic tool for sending Test Mode commands to a + running GSM fw. It is strictly asynchronous in that commands + entered by the operator get sent to the target, and any response + packets received from the target are displayed as they come in. + The tool has no knowledge of any correspondence between commands + being sent and whatever responses they should elicit, i.e., it + is perfectly suited for experimental discovery of firmware + behaviour in response to Test Mode commands. -The only other ETM-based functionality currently integrated in our gsm-fw -besides ETM_CORE is TMFFS (Test Mode for FFS), which is the external access -channel to the device file system - see TIFFS-Overview. The TMFFS1 and TMFFS2 -protocols provide a command/response packet interface to the FFS API functions -inside the fw, and enable an external host connected to the GSM device via the -RVTMUX channel to perform arbitrary read and write operations on the device -file system. + This tool was written before we realized that there was/is an + older, more basic Test Mode predating ETM, hence in many place + we say "ETM" when we really should have said "TM". Oh well... -TMFFS protocol versions +fc-fsio This tool speaks the TMFFS2 protocol and allows a user or + developer to perform a wide range of operations on the file + system of a GSM device. It operates synchronously, i.e., it + sends ETM/TMFFS2 commands and expects responses in strict + lock-step; a single user command may translate into a large + number of ETM/TMFFS2 command packet exchanges. + +AT commands over RVTMUX ======================= -TI made two different and entirely incompatible versions of the TMFFS protocol -for accessing a device's FFS via RVT/ETM: TMFFS1 and TMFFS2. The fw sources -available to us contain implementations of both versions, so we have the freedom -to use whichever we like better for FreeCalypso. After studying the fw source -implementing the two TMFFS protocols, I (Space Falcon) came to the conclusion -that TMFFS2 is both more capable and more reliable; my guess is that TMFFS1 was -likely kept around only because some of TI's crappy Weendoze host software -depended on it. (See gsm-fw/services/ffs/tmffs.c if you would like to judge -for yourself.) Thus TMFFS2 is currently the "officially adopted" version for -FreeCalypso. +There is one more use to which we put the RVTMUX debug serial interface that is +an original FreeCalypso invention: communicating with the AT command interpreter +(ATI). TI's original architecture assumes that if a product is to offer a +standard AT command interface (the product is either a GSM/GPRS modem for which +this AT command interface is the sole mode of usage or a feature phone that +offers a data port as one of its features), then it will be presented on a +dedicated UART separate from RVTMUX. -Our fc-tmsh utility (see below and ../rvinterf/README) allows a developer- -operator to send TMFFS "get version" queries to a running GSM fw in both -ETM_FFS1 and ETM_FFS2 formats; this capability allows us to determine -experimentally which protocol (if any) is implemented by a given proprietary -firmware version. Experiments reveal that Openmoko's moko11 firmware -implements TMFFS1, whereas Pirelli's fw implements TMFFS2. +However, many of our target devices have only one UART practically accessible, +and even when we use Openmoko's modem as our development platform, the RVTMUX +interface is more convenient because it connects externally, whereas the MODEM +UART is connected to the application processor of the smartphone. Therefore, +we developed a way to pass AT commands over RVTMUX. We created a new RVTMUX +channel for this interface and assigned it RVT packet type 0x1A. Packets sent +from an external host to the GSM device carry AT commands and SMS string input, +whereas packets flowing the other way carry ATI's responses to commands and +asynchronous notifications such as incoming calls. -The leo2moko-r1 firmware produced by the FreeCalypso project in 2013-10 -implements TMFFS1, simply because that was the selected configuration in the -found Leonardo source that transitional fw is based on, and that release was -made before I learned RVTMUX, FFS, ETM and TMFFS properly. All future -FreeCalypso firmwares will use TMFFS2, or at least that's the current plan. - -Host utility support -==================== - -As one would naturally expect, the FreeCalypso project has developed some host -tools that allow a PC running GNU/Linux (or other Unix systems) to interface to -running firmwares on GSM devices via RVTMUX. See the rvinterf subtree of -freecalypso-sw for the source and documentation. +The host utility for talking AT commands to a FreeCalypso GSM device via RVTMUX +is fc-shell; it works via rvinterf just like fc-fsio and fc-tmsh.