FreeCalypso > hg > freecalypso-tools

comparison doc/Binary-file-formats @ 676:b6b8307d195b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc: new articles Binary-file-formats and Flash-programming
author Mychaela Falconia <falcon@freecalypso.org>
date Sun, 08 Mar 2020 22:15:57 +0000
parents
children
comparison
equal deleted inserted replaced
675:8b1e86dcc3ac 676:b6b8307d195b
1 In FreeCalypso we use 3 different file formats for Calypso binary images, i.e.,
2 code images to be loaded into either flash or RAM or retrieved flash dumps.
3 These 3 different file formats are straight binary (*.bin), moko-style m0 (*.m0)
4 and little-endian S-records (*.srec).
5
6 Straight binary (*.bin)
7 =======================
8
9 Straight binary is our preferred format for flash dumps. It is written in the
10 native little-endian byte order of the Calypso ARM7 processor, i.e., the order
11 of bytes in the raw binary file directly corresponds to incrementing byte
12 addresses as visible to the ARM7 - any ASCII strings in the image thus appear
13 naturally. We also use the same straight binary format in native LE byte order
14 for flashable code images generated with the gcc+binutils toolchain (as opposed
15 to TI's TMS470), generated with arm-elf-objcopy -O binary - although we don't
16 have too many such code images currently given that neither FC Citrine nor FC
17 Selenite ever achieved production quality.
18
19 Another unrelated use of this straight binary format is for RAM-loadable code
20 images that are fed to Compal's bootloader (Motorola C1xx and Sony Ericsson
21 J100) as opposed to Calypso boot ROM. Our generally preferred image format for
22 RAM-loadable code pieces is little-endian S-records (*.srec, see below), but
23 for Compal's bootloader we use straight binary instead because of the way this
24 bootloader protocol works.
25
26 moko-style m0
27 =============
28
29 This format is a variant of Motorola hex (S-records), a variant invented by TI
30 rather than by us. This format is produced by TI's hex470 tool when run with
31 -m -memwidth 16 -romwidth 16 options, which is the configuration used by TI in
32 the Calypso program, and is read by TI's flash programming tool called FLUID.
33 TI used this format not only for flashable firmware images, but also for various
34 RAM-loadable code pieces, particularly those that comprise the target-side
35 component of FLUID.
36
37 The special quirk of this S-record variant format is its peculiar byte order.
38 TI viewed it as "16-bit hex", meaning that the image is logically viewed as
39 consisting of 16-bit words rather than 8-bit bytes, each S3 record carries an
40 even number of bytes to be loaded at an even address, and each 16-bit word
41 (4 hex digits) appears in these S3 records with its most-significant hex nibble
42 toward the left, just like the address field of each S-record. But if this
43 image gets interpreted by some more naive tool (for example, objcopy from GNU
44 binutils) as bytes rather than 16-bit words, the result will be a reversed byte
45 order, with all strings etc messed up.
46
47 In FreeCalypso we use this moko-style m0 format (our new name for what TI called
48 16-bit hex) only for flashable firmware images built with TI's TMS470 toolchain
49 (can be our own FC Magnetite or historical ones built by Openmoko or other
50 similar historical vendors), but never for any RAM-loadable code pieces - we use
51 little-endian SREC for the latter as explained below.
52
53 And what about the name? Why do we call it moko-style m0 rather than just m0?
54 The reason is because Compal muddied our waters by introducing their own *.m0
55 files that were generated with -memwidth 8 -romwidth 8 instead of -memwidth 16
56 -romwidth 16, producing 8-bit hex instead of 16-bit hex. We do not support
57 Compal's different *.m0 files at all, and we needed some name to specifically
58 identify TI-style m0 files rather than Compal-style. We ended up with the name
59 moko-style rather than TI-style because we already had our mokosrec2bin program
60 going back to 2013-04-15, one of the very first programs written in the
61 FreeCalypso family of projects: our very first encounter with this file format
62 were mokoN firmware images put out in this *.m0 format by That Company.
63
64 Little-endian S-records (*.srec)
65 ================================
66
67 Back at the beginning of FreeCalypso in the spring/summer of 2013 I (Mother
68 Mychaela) decided to use S-records instead of straight binary for our
69 RAM-loadable code pieces, i.e., code that is loaded into RAM either through the
70 Calypso boot ROM (fc-iram) or by chain-loading via loadagent (fc-xram). I made
71 this decision based on two factors:
72
73 1) ARM code generated by common toolchains (both TI's TMS470 and gcc+binutils)
74 without special contortions is not position-independent: a code image that
75 was linked for a given address needs to be loaded at that specific address,
76 not some other.
77
78 2) An S-record image has its load address embedded in the image itself, whereas
79 a raw binary naturally does not carry any such extra metadata.
80
81 With SREC as the standardized hand-off format from code generation tools to
82 loadtools, the choice of load address is made entirely on the code generation
83 side; loadtools do not impose a fixed load address, nor do they require it to
84 be communicated via extra command line arguments or options.
85
86 However, the variant of SREC we use for RAM-loadable code pieces is not the same
87 as moko-style m0 - the byte order is the opposite, with our RAM-loadable code
88 pieces using the native little-endian byte order of the ARM7 processor as the
89 byte order within S3 records. Prior to the introduction of RAM-loadable FC
90 Magnetite fw images for Pirelli DP-L10 in late 2016 (and then likewise for our
91 own FCDEV3B), the only RAM-loadable code pieces we have had were built with
92 gcc+binutils, not with TMS470, and GNU binutils got a different take on the
93 S-record format than TI did: they generate byte-oriented SREC files, with the
94 byte order being the same as it would be in a straight binary file, matching
95 the target processor's memory byte addressing order. Thus GNU-style SREC has
96 been adopted as the format for our RAM-loadable code images for both fc-iram
97 and fc-xram, as opposed to TI-style SREC aka moko-style m0. The convention we
98 have adopted is that *.m0 filename suffix means TI-style aka moko-style,
99 whereas *.srec means GNU-style.
100
101 Besides the S3 record byte order, there is one other difference between TI-built
102 *.m0 code images and GNU-built *.srec ones: the final S7 record carries the
103 entry point address in GNU-built *.srec images, whereas TI's *.m0 images always
104 have a zero dummy address in there. Our fc-iram and fc-xram tools require the
105 real entry point address in the S7 record.
106
107 How do we generate ramimage.srec RAM-loadable images for fc-xram in FC
108 Magnetite? Answer: FC Magnetite build system includes a special ad hoc
109 converter program that reads ramimage.m0 produced by TI's hex470 tool and
110 produces ramimage.srec: it reverses the order of bytes, adds another S3 record
111 that writes the boot-ROM-redirected interrupt and exception vectors and
112 generates an S7 record with the right entry point address.
113
114 This little-endian *.srec format is actively used only for RAM-loadable code
115 pieces in FreeCalypso, not for anything that goes into or gets read from flash.
116 We do have flash dump2srec and flash program-srec commands in fc-loadtool, they
117 were implemented back in the founding stage of FreeCalypso in 2013 for the sake
118 of completeness and symmetry (it seemed right to support both binary and
119 S-record formats), but they never got any practical use: if you are making a
120 flash dump, you would normally want to examine it afterward, and any such
121 examination almost always needs a straight binary image, not S-records.