# HG changeset patch # User Mychaela Falconia # Date 1465888426 0 # Node ID 2748f257312b899297690162672c1052071c2878 # Parent 08804864172ae990b0b2e015a23412fc8cb6f9fa doc/TCH-bit-access: new write-up diff -r 08804864172a -r 2748f257312b doc/TCH-bit-access --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/TCH-bit-access Tue Jun 14 07:13:46 2016 +0000 @@ -0,0 +1,238 @@ +It has been discovered that the DSP ROM in the Calypso GSM baseband processor +implements one nifty feature which is not used at all in standard phone or modem +operation, but which can be used for all kinds of interesting hacks: the traffic +channel (TCH) bits coming out of the GSM 05.03 channel decoder in the downlink +direction (to be fed to one of several speech decoders) can be read out of the +DSP's API RAM in real time, and in the uplink direction the user can feed her +own bits to the input of the GSM 05.03 channel encoder, effectively suppressing +the output of the internal vocoder. + +The DSP mechanism in question is known to work in TCH/FS and TCH/EFS channel +modes, corresponding to the FR and EFR codecs; it is not currently known if it +can also be used in TCH/HS, data traffic or AMR modes. I am aware though of an +anecdotal report that someone tried to make this feature work with AMR, but was +unsuccessful - hence we should be prepared for the possibility that the hack +is not possible with AMR. + +In order to make use of this TCH bit access feature, one needs 3 things: + +1) Firmware on the Calypso (the ARM part) that reads downlink bits from the DSP + and writes uplink bits into it while doing everything else that a GSM fw must + do in order to operate the MS; + +2) Some protocol for passing these TCH bits into and out of the Calypso device; + +3) A source for TCH UL bits and a sink for TCH DL bits on the external host. + +In the case of FreeCalypso, we have defined our own protocol for passing TCH +bits into and out of Calypso GSM devices running one of our firmwares in the +form of an extension to TI's RVTMUX interface, i.e., we have defined a new +RVTMUX channel for this TCH interface and defined the packet types and formats +to be sent over the wire. On the Calypso side this interface is implemented in +FreeCalypso GSM firmwares (currently only in Citrine, but we can also make a +special version of TCS211 with this feature added if the need arises), and on +the host tools side there is support in rvinterf and fc-shell. + +The TCH bit access mechanism in FreeCalypso has been designed with an objective +of presenting to the user exactly what TI's DSP presents to us, i.e., standing +out of the way as much as possible. TI's DSP presents TCH downlink bits and +accepts TCH uplink bits in the form of an array of 16-bit words; the bit order +within these words corresponds to the GSM 05.03 channel encoder bit order (with +a couple of TI-specific quirks documented below) and NOT that of the GSM 06.10 +or EFR codecs. On the RVTMUX serial interface between the Calypso device and +the external host we transfer each TCH frame as a block of 33 bytes; our Calypso +firmwares translate between these bytes and the DSP's 16-bit words, but do not +reorder or change the bits in any way. + +On the host tools side our fc-shell utility provides user commands to save TCH +DL bits into a file and to play TCH UL bits from a file; in the present version +these files are written and read in an ASCII-based hex format. In these ASCII +files each TCH frame is represented as a string of 66 hexadecimal digits, and +these hex digits correspond directly to the 33 bytes being read out of or +written into DSP API words. Therefore, in order to generate and/or interpret +these hexadecimal strings correctly, you (the user) need to understand the bit +order and mapping used by TI's implementation of the GSM 05.03 channel encoder. + +Recall from the GSM specs that the 260 bits which comprise one speech frame are +not all treated equally, instead they are divided into 182 class 1 bits which +are protected by a convolutional encoder and 78 class 2 bits which are +transmitted without any forward error correction. Furthermore, the first 50 of +the class 1 bits are also protected by a CRC. The order in which the bits +appear on TI's DSP interface corresponds to this division, known as the order of +subjective importance. + +Now let's look at the actual bit order and mapping which you need to understand +in order to make sense of the hex strings in tch record and tch play files. +The bit numbering is from the most significant bit to the least significant bit, +i.e., in our string of 66 hex digits the most significant bit of the leftmost +digit is bit 0 and the least significant bit of the rightmost digit is bit 263. +TI's DSP assigns these bits as follows: + +* Bits 0 through 181 correspond to the 182 protected (class 1) bits in the + standard GSM 05.03 order; + +* Bits 182 through 185 are unused - put zeros there if you are generating them + yourself; + +* Bits 186 through 263 correspond to the 78 unprotected (class 2) bits in the + standard GSM 05.03 order. + +Uplink testing +============== + +The uplink sending mechanism can be exercised as follows: + +1. Record a speech sample in the GSM 06.10 codec format using any Unix/Linux + audio tool that can write the de facto standard libgsm format. For example, + using SoX: + + rec -c1 recording.gsm + + SoX will write the recording in the GSM 06.10 libgsm format based on the + .gsm suffix at the end of the recording file name; the -c1 option is needed + to disable stereo, otherwise the recording will be slowed down 2x. + +2. Convert it from libgsm standard format into our ad hoc hex strings for + playing into the TCH uplink: + + fc-fr2tch recording.gsm recording.tch-ul + +3. In fc-shell, when you are in an established voice call, issue this command: + + tch play recording.tch-ul + +You should now hear the speech sample you recorded in step 1 above on the other +end of the GSM call. Please note, though, that for this example to work, the +call must be connected in the FR codec mode, not EFR. If you try it on an EFR +call, the vocoder on the other end of the call will try to interpret your FR +codec bits per EFR, resulting in garbage. In order to make this procedure work +properly with EFR, you would need to generate a speech sample in EFR format and +then put it in the correct hex string form with the correct bit order for +feeding to TI's GSM 05.03 channel encoder implementation, and at the present +moment the tools to do this feat are lacking. + +The fc-fr2tch step above is new with the current version of FreeCalypso host +tools. My original implementation of the tch play command in fc-shell read +libgsm GSM 06.10 speech files directly, but I changed it to read hex strings +which correspond literally to what goes into TI's implementation of the 05.03 +channel encoder to make it more general: + +1) to support playing EFR speech bits into TCH/EFS channels; +2) to support people who may want to pass totally non-speech data over traffic + channels in TFO mode - see below. + +Downlink testing +================ + +When you are in an established voice call in fc-shell, you can record the +downlink TCH bits as follows: + +tch record + +If you would like to record an entire call from the beginning, issue the +tch record command as above before the ATD or ATA command that dials or answers +the call. Either way, whether you are recording a call from the beginning or +from the middle, you need to eventually stop your recording with this command: + +tch record stop + +You can issue this stop command before or after the call is terminated, but +until you issue this tch record stop command, the output file is not closed and +thus may not be written to the file system. + +The recording is written in an ASCII format that is similar but not identical +to what the tch play command takes as its input. Here is an example of what +you might get from tch record: + +C214 281D 0063 81C008000008200046000000000000000000000000000007056608060B0A010B09 +C204 0540 003A 9391480D3051F4BD81DF5EB35069BEBC4AEDF756351C4C19689BB1CA4EA5D4F5F5 +C204 05A2 0047 65B80F9E690F7C8CE4ED80DEF69DA6436518AB99ABCE5815E6B562C5CE4EAC5DC5 +C204 04CA 0044 A4483744B04371ED0334ECB350AF28C639B7F095519EF0242D299B6405124F77A5 +C214 2544 0066 83800400000C1000E80000000000000000000000000000300674A07070F080D0B2 +C200 4E8A 0000 07071DF0F83B9521EE61CFF095AA8C0E560300F6A5573C31F3E00601ED4AAE7E2F +C200 4C94 0000 077385ADE20450B2E410961D6C5B0A173ACF9E2D38D77C28CED8495D88AA4DE72E +C200 4ABC 0000 077F0CF86004132DA5C0A6D5A4B0BD4B28159A07D8F4282DC6AAAB27503BC02701 + +The example above is quoted from an actual recording made from a call to WWV at ++1-303-499-7111, a time of day service. This example shows 5 garbled frames +followed by 3 good ones. + +In each line the first 3 space-separated 16-bit words are status words from the +DSP, and the rest is a hex string of 66 digits giving the 260-bit frame with 4 +extra dummy bits at the boundary between the protected and unprotected portions. + +Now here comes the unpleasant part: radio systems are inherently error-prone, +hence if you are naively expecting every received downlink frame to be a valid +speech frame that matches exactly what the other end transmitted, you will be +disappointed. The downlink frame stream will always consist of both valid and +invalid frames, and in the standard processing chain inside TI's DSP the speech +decoder block that receives these frames from the channel decoder will look at +the status words written by the latter in order to decide what to do with each +frame. But unfortunately I don't know the details, as the DSP is a mostly +undocumented black box. + +Our FreeCalypso firmwares forward the downlink frame status words from the DSP +to the external host and my current implementation of the tch record command +writes them into the file because it is valuable information which should not +be discarded, but unfortunately I don't know their meaning. The first status +word consists of bit flags, the last one seems to be some kind of error count +(what kind of errors? I don't know), and the middle one is totally unknown. + +There is also an fc-tch2fr utility that does sort of an inverse of fc-fr2tch, +except that fc-tch2fr expects files written by tch record which contain the +status words at the beginning - the files written by fc-fr2tch and read by +tch play don't have these status words; they just have 66 hex digits per line +giving the bits to be fed to the GSM 05.03 block in the DSP. The fc-tch2fr +utility works by disregarding the status words; the result will be a .gsm file +which you can play with SoX etc, but it will produce horrible sounds wherever +some garbled frames were received on the call downlink. + +Passing non-speech data over TCH +================================ + +If you are incredibly lucky and you happen to live in a part of the world where +the local GSM network implements TFO (tandem-free operation, see GSM 02.53), +i.e., if your GSM network transparently passes codec frames from one end of the +call to the other without transcoding back and forth in the middle, you might +be able to send arbitrary non-speech data bits over this TCH connection. If +you are going to attempt such a feat, you are mostly on your own as the GSM +network in my part of the world does not support TFO, but here are some general +ideas: + +* Heed the difference between the 182 protected bits and the 78 unprotected + bits of each 260-bit frame. The bits in the first class are less likely to + be corrupted by radio errors. Also the first 50 out of these class 1 bits + are protected by a CRC, so if they get corrupted, there will be a flag bit + somewhere in the DSP status words - but I don't know which bit it is... + +* Given the lack of detailed information about the exact meaning of the DSP + status words, implement some error checking of your own, e.g., an in-band + CRC within your frame payload so you can check on the receiving end if you + got an uncorrupted frame from your data source or not. + +* Expect to lose some frames - even if your radio environment is pristine and + error-free, some frames will still be lost because their timeslots had to be + stolen by FACCH on either end of the call. You will need to implement + acknowledgements, retransmissions and retries - the usual stuff. + +EFR differences +=============== + +TCH/FS and TCH/EFS channel modes corresponding to the FR and EFR codecs, +respectively, share the same channel encoder that takes 260-bit frames as input. +An EFR speech frame is only 244 bits, but as the GSM 05.03 spec explains, each +EFR frame gets expanded to 260 bits by adding some CRC and repetition bits. +One would thus expect that the behaviour of the channel encoder block should be +strictly identical between TCH/FS and TCH/EFS channel modes, but it appears that +TI's implementation exhibits one difference in behaviour between the two modes. + +It appears that TI's implementation of the GSM 05.03 channel encoder computes +the EFR-specific CRC bits internally, such that bits [65,72] of each uplink +frame fed to the DSP in the TCH/EFS mode need to be 0 and will be replaced by +the DSP with CRC bits computed per the standard. It also appears that their +implementation of the channel decoder verifies this CRC, sets a status bit +indicating whether the check succeeded or failed, and then zeroes these bits +out, i.e., the original bits in these CRC bit positions are lost. No problem +if what you are passing over TCH/EFS is indeed EFR speech frames, but it will +be a problem if you are passing non-speech data over your TCH and put something +else in those bits which the spec allocates to CRC.