FreeCalypso > hg > gsm-codec-lib

annotate doc/TW-TS-005 @ 608:d4e42ec4a688 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
hrutil: new program gsmhr-decode-r
author Mychaela Falconia <falcon@freecalypso.org>
date Fri, 05 Dec 2025 09:01:14 +0000
parents 24aba0e7aa35
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 The original set of Themyscira Wireless utilities for FR and EFR codecs uses an
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 ad hoc binary file format to represent streams of FR or EFR codec frames - see
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3 Binary-file-format article. However, a newer hexadecimal format has now been
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 standardized as Themyscira Wireless Technical Specification TW-TS-005:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
6 https://www.freecalypso.org/specs/tw-ts-005-v010100.txt
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
8 The original version of this standard had two annexes intended for practical
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
9 use:
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 * TW-TS-005 Annex A defines a representation format for FR and EFR codecs;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 * TW-TS-005 Annex B defines a representation format for HR codec.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
14 The current version of TW-TS-005 linked above has two additional annexes:
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
15
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
16 * TW-TS-005 Annex C specifies representation of AMR RTP payloads in hex files;
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
17 * TW-TS-005 Annex D specifies representation of CSD RTP payloads in hex files.
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
18
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 The present version of ThemWi GSM codec libraries & utilities suite includes
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
20 some utilities that operate on TW-TS-005 Annex A hex files; support for Annex B
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
21 will appear in a future version when our work on GSM-HR codec integration
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
22 progresses further. Support for Annex C is described in AMR-hex-utils article;
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
23 no utilities for Annex D exist as of this writing.
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25 TW-TS-005 Annex A vs gsmx binary format
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 =======================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 For working with FR and EFR codecs, our original binary file format has one
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29 major defect: it cannot represent bad traffic frames (in GSM 06.31 & 06.81
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30 definition, i.e., BFI=1) that have payload data bits included, as happens in
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 well-designed GSM networks that use GSM 08.60 TRAU-UL frames or TW-TS-001
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32 enhanced RTP transport. This file format deficiency leads to the following
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 downstream defects:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35 * The combination of "bad traffic frame" and "accepted SID frame" (again,
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36 GSM 06.31 & 06.81 terminology) gets incorrectly treated as "unusable frame"
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37 rather than "invalid SID frame" as the specs decree.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39 * In the case of EFR, the reference decoder C code that forms the basis for
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 Themyscira libgsmefr makes use of "fixed codebook excitation pulses" portion
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41 of bad frames during speech (as opposed to comfort noise) state - but these
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42 bits were lost to file format shortcoming.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 The new hexadecimal format of TW-TS-005 Annex A solves this shortcoming: each
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45 frame is stored as a hex line that directly corresponds to a single RTP payload,
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46 hence the full capabilities of TW-TS-001 extended RTP format are made available
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47 in a file at rest.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 Because we have so many existing utilities that read and write gsmx binary
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50 files, and this binary format is so entrenched in Themyscira development
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51 environment, we are not doing a "forklift" migration of all of our tools to the
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 new format. Instead we are taking a more tempered approach:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54 * For the decoding operation (taking a frame stream from an Rx Radio Subsystem
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55 and producing linear PCM output) that is most affected by the shortcomings of
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56 gsmx format, we have new utilities that read TW-TS-005 Annex A input, while
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57 the old gsmx-reading utilities are still preserved and maintained;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59 * For most other workflows (for example, encoding of new speech) conversion
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60 utilities between the two formats (described below) are deemed sufficient;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62 * New developments such as TFO transform use TW-TS-005 Annex A format natively.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64 Human-readable dump decoding of TW-TS-005 hex files
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 ===================================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 A line-based hexadecimal file format with one line per stored codec frame is
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
68 inherently more human-readable than a binary file, but we also desire a more
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
69 complete decoding such as that produced by gsmrec-dump, showing all codec
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
70 parameters and frame metadata flags. tw5a-dump produces such decoding for
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
71 TW-TS-005 Annex A hex files; there will also be a corresponding tw5b-dump
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
72 utility for TW-TS-005 Annex B when we finish integrating GSM-HR codec support.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
73
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
74 Conversion utilities (FR and EFR codecs)
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
75 ========================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
76
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
77 gsmx-to-tw5a and tw5a-to-gsmx utilities do what their names suggest: convert
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
78 FR/EFR speech recordings or test sequences between gsmx (binary) and TW-TS-005
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
79 Annex A (hex) formats. Important semantic notes:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
80
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
81 * gsmx-to-tw5a emits basic RTP format (no TEH) for all good frames, while each
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
82 BFI marker record is converted to a TEH-only No_Data frame.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84 * tw5a-to-gsmx is the lossy conversion: distinction between basic and extended
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85 RTP formats is lost, ditto for TAF without BFI, all BFIs become BFI-no-data.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87 A conversion from gsmx to tw5a back to gsmx is lossless, but not the other way
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 around.