FreeCalypso > hg > gsm-codec-lib

annotate doc/TW-TS-005 @ 640:e0e5905261e2 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
document tw5b-dump and tw5c-dump
author Mychaela Falconia <falcon@freecalypso.org>
date Fri, 20 Mar 2026 06:43:50 +0000
parents 24aba0e7aa35
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 The original set of Themyscira Wireless utilities for FR and EFR codecs uses an
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 ad hoc binary file format to represent streams of FR or EFR codec frames - see
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3 Binary-file-format article. However, a newer hexadecimal format has now been
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 standardized as Themyscira Wireless Technical Specification TW-TS-005:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
6 https://www.freecalypso.org/specs/tw-ts-005-v010100.txt
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
8 The original version of this standard had two annexes intended for practical
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
9 use:
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 * TW-TS-005 Annex A defines a representation format for FR and EFR codecs;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 * TW-TS-005 Annex B defines a representation format for HR codec.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
14 The current version of TW-TS-005 linked above has two additional annexes:
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
15
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
16 * TW-TS-005 Annex C specifies representation of AMR RTP payloads in hex files;
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
17 * TW-TS-005 Annex D specifies representation of CSD RTP payloads in hex files.
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
18
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 The present version of ThemWi GSM codec libraries & utilities suite includes
640
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
20 some utilities that operate on TW-TS-005 Annex A, Annex B and Annex C hex files;
592
24aba0e7aa35 document AMR hex (TW-TS-005 Annex C) utilities addition
Mychaela Falconia <falcon@freecalypso.org>
parents: 549
diff changeset
21 no utilities for Annex D exist as of this writing.
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
22
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
23 TW-TS-005 Annex A vs gsmx binary format
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24 =======================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 For working with FR and EFR codecs, our original binary file format has one
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27 major defect: it cannot represent bad traffic frames (in GSM 06.31 & 06.81
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 definition, i.e., BFI=1) that have payload data bits included, as happens in
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29 well-designed GSM networks that use GSM 08.60 TRAU-UL frames or TW-TS-001
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30 enhanced RTP transport. This file format deficiency leads to the following
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 downstream defects:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 * The combination of "bad traffic frame" and "accepted SID frame" (again,
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34 GSM 06.31 & 06.81 terminology) gets incorrectly treated as "unusable frame"
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35 rather than "invalid SID frame" as the specs decree.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37 * In the case of EFR, the reference decoder C code that forms the basis for
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38 Themyscira libgsmefr makes use of "fixed codebook excitation pulses" portion
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39 of bad frames during speech (as opposed to comfort noise) state - but these
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 bits were lost to file format shortcoming.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42 The new hexadecimal format of TW-TS-005 Annex A solves this shortcoming: each
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43 frame is stored as a hex line that directly corresponds to a single RTP payload,
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 hence the full capabilities of TW-TS-001 extended RTP format are made available
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45 in a file at rest.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47 Because we have so many existing utilities that read and write gsmx binary
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48 files, and this binary format is so entrenched in Themyscira development
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 environment, we are not doing a "forklift" migration of all of our tools to the
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50 new format. Instead we are taking a more tempered approach:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 * For the decoding operation (taking a frame stream from an Rx Radio Subsystem
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53 and producing linear PCM output) that is most affected by the shortcomings of
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54 gsmx format, we have new utilities that read TW-TS-005 Annex A input, while
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55 the old gsmx-reading utilities are still preserved and maintained;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57 * For most other workflows (for example, encoding of new speech) conversion
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58 utilities between the two formats (described below) are deemed sufficient;
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60 * New developments such as TFO transform use TW-TS-005 Annex A format natively.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62 Human-readable dump decoding of TW-TS-005 hex files
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63 ===================================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 A line-based hexadecimal file format with one line per stored codec frame is
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66 inherently more human-readable than a binary file, but we also desire a more
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 complete decoding such as that produced by gsmrec-dump, showing all codec
640
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
68 parameters and frame metadata flags. The following TW-TS-005 dump utilities
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
69 are provided:
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
70
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
71 tw5a-dump Decodes TW-TS-005 Annex A hex files and displays FR and EFR
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
72 codec frames.
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
73
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
74 tw5b-dump Decodes TW-TS-005 Annex B hex files and displays HR codec
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
75 frames.
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
76
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
77 tw5c-dump Decodes TW-TS-005 Annex C hex files and displays AMR codec
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
78 frames.
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
79
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
80 For tw5a-dump and tw5b-dump, the only required command line argument is the
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
81 name of the hex file to be read and dumped. For tw5c-dump, please refer to
e0e5905261e2 document tw5b-dump and tw5c-dump
Mychaela Falconia <falcon@freecalypso.org>
parents: 592
diff changeset
82 AMR-hex-utils article for further explanation.
549
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84 Conversion utilities (FR and EFR codecs)
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85 ========================================
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87 gsmx-to-tw5a and tw5a-to-gsmx utilities do what their names suggest: convert
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 FR/EFR speech recordings or test sequences between gsmx (binary) and TW-TS-005
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
89 Annex A (hex) formats. Important semantic notes:
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
90
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
91 * gsmx-to-tw5a emits basic RTP format (no TEH) for all good frames, while each
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
92 BFI marker record is converted to a TEH-only No_Data frame.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
93
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
94 * tw5a-to-gsmx is the lossy conversion: distinction between basic and extended
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
95 RTP formats is lost, ditto for TAF without BFI, all BFIs become BFI-no-data.
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
96
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
97 A conversion from gsmx to tw5a back to gsmx is lossless, but not the other way
d9f6b3125259 document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
98 around.