changeset 297:6b479cfb06a4

beginning of libgsmfr2 documentation
author Mychaela Falconia <falcon@freecalypso.org>
date Mon, 15 Apr 2024 07:12:31 +0000
parents e0d42e87da96
children a45f806cada9
files doc/FR1-library-API doc/FR1-library-desc
diffstat 2 files changed, 292 insertions(+), 0 deletions(-) [+]
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/FR1-library-API	Mon Apr 15 07:12:31 2024 +0000
@@ -0,0 +1,211 @@
+Libgsmfr2 general usage
+=======================
+
+The external public interface to Themyscira libgsmfr2 consists of a single
+header file <tw_gsmfr.h>; it should be installed in some system include
+directory.
+
+The dialect of C used by all Themyscira GSM codec libraries is ANSI C (function
+prototypes), const qualifier is used where appropriate, and the interface is
+defined in terms of <stdint.h> types; <tw_gsmfr.h> includes <stdint.h>.  The
+use of old libgsm defined types (gsm_byte, gsm_frame and gsm_signal) has been
+abolished in the migration from libgsm+libgsmfrp to libgsmfr2.
+
+GSM 06.10 encoder and decoder
+=============================
+
+Both the encoder and the decoder are stateful; each running instance of either
+element needs its own state structure.  However, this GSM 06.10 component of
+libgsmfr2 shares a peculiar property with old libgsm from which it was derived:
+the same state structure (struct gsmfr_0610_state) is used by both entities.
+Needless to say, each given instance of struct gsmfr_0610_state must be used
+for only one purpose, either for the encoder or for the decoder; mixing calls
+to encoder and decoder functions with the same state structure is an invalid
+operation with undefined results.
+
+State structures for the basic encoder or decoder are allocated with this
+function:
+
+struct gsmfr_0610_state *gsmfr_0610_create(void);
+
+This function allocates dynamic memory for the state structure with malloc()
+(the size of the struct is internal to the library and not exposed) and returns
+a pointer to the allocated and initialized struct if successful, or NULL if
+malloc() fails.  The state structure is malloc'ed as a single chunk, hence when
+you are done with it, simply free() it.
+
+The initialization or reset portion of gsmfr_0610_create() operation can always
+be repeated with this function:
+
+void gsmfr_0610_reset(struct gsmfr_0610_state *state);
+
+Immediately after gsmfr_0610_create() or gsmfr_0610_reset(), the "virgin" state
+structure can be used either for the encoder or for the decoder; however, once
+that state struct has been passed to functions of either group, it can only be
+used for that functional group.
+
+Encoder specifics
+-----------------
+
+The most elementary single-frame processing function of libgsmfr2 GSM 06.10
+encoder is:
+
+void gsmfr_0610_encode_params(struct gsmfr_0610_state *st, const int16_t *pcm,
+			      struct gsmfr_param_frame *param);
+
+The input is an array of 160 linear PCM samples (left-justified in int16_t),
+and the output is this structure:
+
+struct gsmfr_param_frame {
+	int16_t	LARc[8];
+	int16_t	Nc[4];
+	int16_t	bc[4];
+	int16_t	Mc[4];
+	int16_t	xmaxc[4];
+	int16_t	xMc[4][13];
+};
+
+Most of the time the following wrapper function is more useful:
+
+void gsmfr_0610_encode_frame(struct gsmfr_0610_state *st, const int16_t *pcm,
+			     uint8_t *frame);
+
+The output is a 33-byte buffer, filled with the encoded GSM-FR speech frame in
+the RTP format specified in ETSI TS 101 318 and IETF RFC 3551.
+
+If the optional encoder homing feature is desired, call this function right
+after the call to gsmfr_0610_encode_frame() or gsmfr_0610_encode_params():
+
+void gsmfr_0610_encoder_homing(struct gsmfr_0610_state *st, const int16_t *pcm);
+
+This function checks to see if the PCM frame (160 linear PCM samples) is an EHF;
+if the input frame is indeed EHF, the function calls gsmfr_0610_reset().
+
+Decoder specifics
+-----------------
+
+The internal native form of the 06.10 decoder once again uses
+struct gsmfr_param_frame:
+
+void gsmfr_0610_decode_params(struct gsmfr_0610_state *st,
+			      const struct gsmfr_param_frame *param,
+			      int16_t *pcm);
+
+The more commonly used RTP-format version is:
+
+void gsmfr_0610_decode_frame(struct gsmfr_0610_state *st, const uint8_t *frame,
+			     int16_t *pcm);
+
+Please note:
+
+1) The basic GSM 06.10 decoder is just that: there is no SID recognition or DTX
+   handling, every possible input bit pattern will be interpreted and decoded
+   as a GSM 06.10 speech frame.
+
+2) There is no decoder homing function at this layer, and no check for DHF.
+
+3) The RTP signature nibble 0xD is ignored (not checked) by
+   gsmfr_0610_decode_frame().
+
+Rx DTX preprocessor block
+=========================
+
+The Rx DTX preprocessor is its own stateful element, independent from the 06.10
+decoder to which it is usually coupled.  Libgsmfr2 provides a "fulldec" wrapper
+that incorporates both elements, but the ability to use the Rx DTX preprocessor
+by itself still remains, unchanged from our previous libgsmfrp offering.  One
+potential application for this preprocessor by itself, without immediately
+following it with the GSM 06.10 decode step, is the possibility of implementing
+the TFO/TrFO transform of 3GPP TS 28.062 section C.3.2.1.1 for GSM-FR: our Rx
+DTX preprocessor does exactly what that section calls for, specifically in
+"case 1" where the input UL frame stream may contain SIDs and BFI frame gaps,
+but the output must be 100% valid frames and SID-free.
+
+The state structure for this block is struct gsmfr_preproc_state, and it is
+allocated with this function:
+
+struct gsmfr_preproc_state *gsmfr_preproc_create(void);
+
+Like other state structures in Themyscira GSM codec libraries, this opaque
+state is malloc'ed as a single chunk and can be simply freed afterward.  A
+reset function is also provided:
+
+void gsmfr_preproc_reset(struct gsmfr_preproc_state *state);
+
+Preprocessing good frames
+-------------------------
+
+For every good traffic frame (BFI=0) you receive from the radio subsystem, you
+need to call this preprocessor function:
+
+void gsmfr_preproc_good_frame(struct gsmfr_preproc_state *state,
+				uint8_t *frame);
+
+The second argument is both input and output, i.e., the frame is modified in
+place.  If the received frame is not SID (specifically, if the SID field
+deviates from the SID codeword by 16 or more bits, per GSM 06.31 section 6.1.1),
+then the frame (considered a good speech frame) will be left unmodified (i.e.,
+it is to be passed unchanged to the GSM 06.10 decoder), but preprocessor state
+will be updated.  OTOH, if the received frame is classified as either valid or
+invalid SID per GSM 06.31, then the output frame will contain comfort noise
+generated by the preprocessor using a PRNG, or a silence frame in one particular
+corner case.
+
+GSM-FR RTP (originally libgsm) 0xD magic: the upper nibble of the first byte
+can be anything on input to gsmfr_preproc_good_frame(), but the output frame
+will always have the correct magic in it.
+
+Handling BFI conditions
+-----------------------
+
+If you received a lost/missing frame indication instead of a good traffic frame,
+call this preprocessor function:
+
+void gsmfr_preproc_bfi(struct gsmfr_preproc_state *state, int taf,
+			uint8_t *frame_out);
+
+TAF is a flag defined in GSM 06.31 section 6.1.1; if you don't have this flag,
+pass 0 - you will lose the function of comfort noise muting in the event of
+prolonged SID loss, but all other Rx DTX functions will still work the same.
+
+With this function the 33-byte frame buffer is only an output, i.e., prior
+buffer content is a don't-care and there is no provision for making any use of
+erroneous frames like in EFR.  The frame generated by the preprocessor may be
+substitution/muting, comfort noise or silence depending on the state.
+
+GSM-FR full decoder
+===================
+
+The full decoder is a high-level feature of libgsmfr2, incorporating both the
+Rx DTX preprocessor block and the GSM 06.10 decoder block.  The state structure
+for the full decoder (struct gsmfr_fulldec_state) internally incorporates both
+struct gsmfr_0610_state and gsmfr_preproc_state, but because it is implemented
+inside libgsmfr2, it is still malloc'ed as a single chunk and can thus be
+released with a single free() call.  The functions for allocating and
+initializing this state follow the established pattern:
+
+struct gsmfr_fulldec_state *gsmfr_fulldec_create(void);
+
+void gsmfr_fulldec_reset(struct gsmfr_fulldec_state *state);
+
+The reset function internally calls gsmfr_0610_reset() and
+gsmfr_preproc_reset(), initializing both processing blocks.
+
+Frame processing functions are also straightforward:
+
+void gsmfr_fulldec_good_frame(struct gsmfr_fulldec_state *state,
+				const uint8_t *frame, int16_t *pcm);
+
+void gsmfr_fulldec_bfi(struct gsmfr_fulldec_state *state, int taf,
+			int16_t *pcm);
+
+These functions follow the same pattern as gsmfr_preproc_good_frame() and
+gsmfr_preproc_bfi(), but the output is a 160-sample linear PCM buffer.  Also
+note that the frame input to gsmfr_fulldec_good_frame() is const, unlike the
+situation with gsmfr_preproc_good_frame() - the copying into a scratchpad
+buffer (on the stack) happens inside this "fulldec" wrapper.
+
+The "fulldec" layer also adds the decoder homing feature:
+gsmfr_fulldec_good_frame() detects decoder homing frames and invokes
+gsmfr_fulldec_reset() when required, and also implements EHF output per the
+spec.
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/FR1-library-desc	Mon Apr 15 07:12:31 2024 +0000
@@ -0,0 +1,81 @@
+Themyscira libgsmfr2 is our new (2024) library offering for GSM FRv1 codec,
+replacing the previous combination of libgsm (classic 1990s free sw offering)
+and libgsmfrp (our add-on).  That combination appeared satisfactory at first
+because of how the decoder processing chain is defined for FRv1 (the Rx DTX
+handler forms a modular piece, passing a frame of 260 bits to unmodified "pure"
+GSM 06.10 decoder), but use of legacy libgsm presents some difficulties:
+
+* Inconvenience and inconsistency: for all other supported GSM speech codecs,
+  Themyscira libraries provide the complete solution, not depending on anyone
+  else's software, but for FRv1 we depended on a library that was created in a
+  different era for non-GSM applications but just happens, almost by accident,
+  to be a bit-exact implementation of GSM 06.10 spec.
+
+* Poor design in frame packing and unpacking functions: the operations of
+  bit-shuffling a GSM-FR codec frame between the array of parameters form
+  (76 words) and the packed RTP format used in IP-based GSM RAN (33 bytes) are
+  stateless pure functions, but their implementations in libgsm (gsm_explode()
+  and gsm_implode()) require a state structure.  (Libgsm supports WAV49 format
+  in addition to the RTP-adopted one, and WAV49 packing is stateful - but this
+  WAV49 feature is of no use and no relevance in real GSM applications.)
+
+* No ability to implement homing: the internal state structure used by libgsm
+  is set to the home state when it is allocated with gsm_create(), but there is
+  no reset function, and such function cannot be implemented externally when
+  the state structure is private to the library and not exposed.  Therefore,
+  the optional codec homing feature defined in later versions of GSM 06.10 spec
+  cannot be implemented in a wrapper around legacy libgsm, causing the resulting
+  FOSS implementation to be inferior to existing commercial implementations
+  (deployed in practice by incumbent nation-scale networks) which do implement
+  this feature.
+
+In response to the above issues, we now have a new library named libgsmfr2 that
+provides all needed functions for GSM-FR codec "under one roof", harmonized
+with our support for other GSM speech codecs.  However, the modularity that is
+inherent in the way this codec is defined in the specs (contrast with EFR) is
+still retained in the design of our library, which exhibits the following 4
+functional divisions:
+
+1) Libgsmfr2 includes a "pure" GSM 06.10 encoder and decoder, directly
+   corresponding to old libgsm.  This code is in fact lifted from libgsm,
+   ported into Themyscira gsm-codec-lib style in terms of C dialect, defined
+   types and naming conventions.  The reset function that is missing in libgsm
+   is now provided, however, fixing that defect.
+
+2) The Rx DTX handler component is unchanged from our previous libgsmfrp.  Per
+   the relevant specs (GSM 06.11, 06.12 and 06.31) this component is a modular
+   piece: it emits a standard 260-bit frame of GSM 06.10 parameters that can be
+   fed to anyone else's implementation of the latter standard.
+
+3) Full decoder: this component is a wrapper around an 06.10 decoder instance
+   and an Rx DTX instance, providing functionality equivalent to the standard
+   decoder function in other GSM speech codecs.
+
+4) Stateless utility functions for frame format conversions (packing and
+   unpacking) and for incoming SID classification.  An application can freely
+   use just these functions, without pulling in any encoder or decoder or
+   stateful preprocessor functionality, making the present library very
+   convenient for debug utilities.
+
+The homing feature is available in both encoder and decoder directions, but it
+is implemented differently between the two:
+
+* In the encoder direction, if the application wishes to enable the possibility
+  of in-band homing, it needs to call gsmfr_0610_encoder_homing() after the
+  regular call to gsmfr_0610_encode_frame() or gsmfr_0610_encode_params().
+
+* In the decoder direction, the homing feature is always included if one uses
+  the "fulldec" (full decoder) wrapper (it is implemented in that layer), and
+  never included if one uses the "basic" GSM 06.10 decoder by itself, or the
+  Rx DTX handler block by itself.
+
+The only major feature of GSM-FR codec that is currently absent in libgsmfr2 is
+application of DTX in the encoder direction: GSM 06.32 VAD followed by DTX
+hangover logic and SID output.  This omission is currently acceptable for
+Themyscira Wireless: DTXd (DTX in the radio downlink direction) is not allowed
+when each BTS operates at only one carrier frequency, which makes it pointless
+to enable DTX for speech encoding at the network edge transcoder, and we are not
+trying to replace TI Calypso DSP on the MS side of GSM.  However, if this
+situation changes and some need arises for a FOSS implementation of DTX-enabled
+GSM-FR encoder, the architecture of Themyscira libgsmfr2 should make it possible
+to integrate such addition.