# HG changeset patch # User Mychaela Falconia # Date 1671961682 0 # Node ID 486e19acc66fd6f44f4c35962581d8b856135c64 # Parent 80c93ef82a515b6f41633e1bd5c5537862cb9f42 doc/Calypso-TCH-downlink: document new utilities diff -r 80c93ef82a51 -r 486e19acc66f doc/Calypso-TCH-downlink --- a/doc/Calypso-TCH-downlink Fri Dec 16 08:38:36 2022 +0000 +++ b/doc/Calypso-TCH-downlink Sun Dec 25 09:48:02 2022 +0000 @@ -1,11 +1,10 @@ -It has been discovered that the DSP ROM in the Calypso GSM baseband processor -implements one nifty feature which is not used at all in standard phone or modem -operation, but which can be used for all kinds of interesting hacks: the traffic -channel (TCH) bits coming out of the GSM 05.03 channel decoder in the downlink -direction (to be fed to one of several speech decoders) can be read out of the -DSP's API RAM in real time, and in the uplink direction the user can feed her -own bits to the input of the GSM 05.03 channel encoder, effectively suppressing -the output of the internal vocoder. +It has been discovered that the implementation of standard signal processing +chains for speech TCH downlink and uplink in the DSP ROM in the Calypso GSM +baseband processor allows these signal processing chains to be tapped at certain +points, as detailed in the TCH-tap-modes article in our freecalypso-docs Hg +repository. There is a mechanism to capture the stream of received traffic +frames on TCH DL, and there is another mechanism by which an externally supplied +stream can be "played" into TCH UL. I (Mother Mychaela) previously played with this functionality back in 2016, and it's been mostly shelved since then. This functionality is now getting renewed @@ -14,31 +13,99 @@ can take another shot at decoding TCH downlink captures taken from Calypso GSM MS. -For reasons of project resource planning, the work to study Calypso DSP downlink -bits in EFR mode is currently postponed until after we build our long-overdue FC -Venus board, but in the case of FR1 codec, we have a TCH downlink recording that -was captured back in 2016, and by analyzing this recording, we can make the -following observations: +The overall functionality is described in the TCH-tap-modes article in +freecalypso-docs; the mechanism for capturing TCH DL bits from Calypso DSP is +split between FreeCalypso GSM MS firmware (added to FC Tourmaline as of +2022-12-23) and the fc-shell utility in the FC host tools package - the updated +version (required for use with current Tourmaline fw) will be released as part +of fc-host-tools-r18. There is also a set of utilities included in the present +GSM codec libraries & utilities package for parsing and decoding these Calypso +TCH DL captures; the present document describes these utilities. + +As explained in the TCH-tap-modes article in freecalypso-docs, the mechanism +for capturing TCH DL is currently implemented for TCH/FS, TCH/HS and TCH/EFS, +corresponding to FR1, HR1 and EFR codecs. However, further parsing and decoding +support has only been implemented for FR1 and EFR codecs in the present package, +in the form of the following utilities: -* DSP status word 0 contains various flags; we already knew some of them, but - now we got additional knowledge: +gsmfr-dlcap-parse This program reads a TCH/FS DL capture file and parses + it for human analysis. All input fields are passed + through to the output, but the program also computes + the ternary SID flag of GSM 06.31 section 6.1.1 from + the payload bits (for comparison against what the DSP + wrote in its status word 0) and prints all broken-down + parameter fields of each GSM 06.10 FR1 codec frame. - bit 2 is BFI - bits 4:3 give a 2-bit SID flag per GSM 06.31 section 6.1.1 +gsmfr-dlcap-gsmx This program reads a TCH/FS DL capture file and converts + it into an extended-libgsm (gsmx) file containing a mix + of FR1 codec frames and Themyscira BFI markers. The + latter BFI markers will be emitted in those frame + positions where FACCH was received instead of speech, + or where the DSP otherwise indicated BFI=1. The gsmx + output from this utility needs to be fed to gsmfr-decode + from the present package, so that our FR1 Rx DTX + preprocessor will take care of SIDs and BFIs, completing + the required GSM MS processing chain for TCH/FS DL. - Unfortunately I couldn't find TAF anywhere - but oh well. +gsmefr-dlcap-parse This program reads a TCH/EFS DL capture file and parses + it for human analysis. All input fields are passed + through to the output, but the program also computes + the ternary SID flag of GSM 06.81 section 6.1.1 from + the payload bits (for comparison against what the DSP + wrote in its status word 0) and prints all broken-down + parameter fields of each EFR codec frame. Finally, each + triplicated bit group of GSM 05.03 section 3.1.1.2 is + printed as an octal digit, to aid human analysis of how + the DSP writes these bits in its a_dd_0 buffer. -* DSP status word 1 appears to be garbage, DSP status word 2 appears to be some - kind of error count - no new knowledge here. +gsmefr-dlcap-gsmx This program reads a TCH/EFS DL capture file and + converts it into a gsmx binary file, containing a mix + of EFR codec frames and Themyscira BFI markers. The + latter BFI markers will be emitted in those frame + positions where FACCH was received instead of speech, + or where the DSP otherwise indicated BFI=1. The gsmx + output from this utility needs to be fed to + gsmefr-decode (or gsmefr-decode-r) from the present + package. -* By heeding the newly-understood BFI flag in DSP status word 0, we can do a - much cleaner decoding of TCH downlink capture than we did back in 2016. +gsmefr-dlcap-dec This program reads a TCH/EFS DL capture file and feeds + it directly to the EFR reference decoder implemented in + libgsmefr, without going through a gsmx intermediary. + +Additional notes: + +* The new gsmfr-dlcap-gsmx utility described above replaces the old fc-tch2fr + utility from FC host tools - the latter should now be considered a bogon. + The required GSM MS processing chain for TCH/FS DL includes the step of Rx + DTX handler between the output of GSM 05.03 channel decoder and the input of + GSM 06.10 speech decoder; the old chain of fc-tch2fr followed by libgsm + decoding omitted this critical step and thus produced very unkind-on-ears + sounds. -The new gsmfr-cvt-dlcap utility in the present package replaces the old -fc-tch2fr utility from FC host tools - the latter should now be considered a -bogon. The output from gsmfr-cvt-dlcap is in our gsmx format, containing -Themyscira BFI markers in those frame positions where Calypso DSP indicated BFI -on TCH downlink, and it needs to be fed to gsmfr-decode from the present -package; our FR1 Rx DTX preprocessor will take care of SIDs and BFIs, and the -complete decoding chain should then closely replicate what must be happening -inside the original DSP. +* gsmefr-dlcap-dec has been written as a bold attempt to replicate the complete + Rx DTX handler and speech decoder (the part of TCH DL processing chain that + sits past the a_dd_0 buffer) as they are implemented inside TI's DSP. Such a + feat won't be possible for FR1 codec (other than by a Herculean effort of full + static reversing of the DSP ROM) because there is no bit-exact definition of + FR1 Rx DTX functions in GSM specs, but for EFR there is a bit-exact reference + implementation from ETSI. *If* TI's DSP matches this bit-exact reference + (there are some aspects of Rx DTX handling where this bit-exact reference is + considered to be an example rather than normative, see GSM 06.61), then there + is a chance we could replicate TI's DSP chain externally - but only if we can + figure out exactly how the bits of a_dd_0[0] drive the logic of their Rx DTX + handler. The Mother's plan is to capture the DSP's decoded speech output from + MCSI on an FCDEV3B using a small FPGA board with a PCM-to-UART logic function, + while simultaneously capturing TCH DL bits in the a_dd_0 buffer, then run + gsmefr-dlcap-dec on the captured TCH DL booty and see if we can replicate the + DSP's end output - but until then, this gsmefr-dlcap-dec program should be + treated as an unfinished experiment in progress. + +* In the case of FR1 codec, there is no prescribed bit-exact definition for the + Rx DTX handler (GSM 06.11, 06.12 and 06.31 specs define general requirements, + but aren't bit-exact in most aspects), and the way in which we (Themyscira + Wireless) have implemented our FR1 Rx DTX handler (libgsmfrp in the present + package) perfectly matches our gsmx binary file format for good vs bad frames. + Therefore, in the case of FR1 codec there is nothing to be gained by skipping + gsmx and calling library functions directly, and thus there is no FR1 + counterpart to gsmefr-dlcap-dec - just use gsmfr-dlcap-gsmx followed by + gsmfr-decode or gsmfr-decode-r.