FreeCalypso > hg > freecalypso-tools

comparison doc/User-phone-tools @ 388:3d45660f78f0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc/User-phone-tools article written
author Mychaela Falconia <falcon@freecalypso.org>
date Sun, 11 Mar 2018 06:32:00 +0000
parents
children b1864e3f8fb4
comparison
equal deleted inserted replaced
387:b61b81d3cece 388:3d45660f78f0
1 FreeCalypso User Phone Tools are a new software addition to the FreeCalypso
2 family. These tools are programs that run on a Unix host computer such as a
3 GNU/Linux PC or laptop and communicate with a FreeCalypso phone or modem via
4 the standard AT command interface, rather than any of the formerly proprietary
5 interfaces specific to TI's internal architecture. The following tools are
6 currently available:
7
8 fcup-at Issues an arbitrary AT command given on the command line.
9
10 fcup-settime Issues AT+CCLK command to the target to set its clock to the
11 host computer's notion of local time.
12
13 fcup-smdump Retrieves a dump of SMS records (received, sent or stored
14 messages) from the FC device's SMS storage (currently SIM
15 storage; ME storage may be implemented in the future),
16 optionally deleting them from the severely space-limited
17 SIM/ME storage afterward.
18
19 fcup-smsend* Tools for sending outgoing SMS from a host computer through a
20 FreeCalypso phone or modem and/or writing such outgoing SMS
21 into the FC device's SMS storage.
22
23 fcup-smwrite Debug and development tool: writes arbitrary message records
24 into the FC device's SMS storage (currently SIM storage) in any
25 of the possible 4 states, with arbitrary incoming or outgoing
26 SMS PDU content.
27
28 Because these tools communicate with the target via standards-defined AT
29 commands, in theory they ought to work with any AT-command-speaking 3GPP phone
30 or modem and not just our own FreeCalypso. However, experience has shown that
31 in the case of the common proprietary implementations, practice does not match
32 theory: when I (Mychaela) tried these same AT commands against a random
33 off-the-shelf proprietary modem (Huawei E303 USB stick modem for 3G), the
34 following problems were seen:
35
36 * The essential AT+CMGL=4 command for retrieving the full set of SMS records
37 from SIM storage in PDU mode appears to be broken: all I got was a hang.
38 Its text mode counterpart AT+CMGL="ALL" produces incomplete output.
39
40 * Qualcomm/Huawei's implementation of the AT command interface does not allow
41 AT+CSCS to be set to "HEX"; our fcup-smdump implementation uses this setting
42 so that the phonebook names returned along with SMS PDUs in the +CMGL
43 responses can be parsed reliably no matter what weird characters they might
44 contain.
45
46 * Setting AT+CSCS to "8859-1" is not supported either; this setting is used by
47 our fcup-smsend and fcup-smsendmult tools when sending in text mode.
48
49 * Sending outgoing SMS with fcup-smsend in PDU mode (which does not touch
50 AT+CSCS) works in that the message goes out, but the tool complains afterward
51 because the echo after the ^Z is different from what our tools expect.
52
53 Because of these quirks, our FC User Phone Tools officially work only with our
54 own FreeCalypso phones and modems, and are not expected to work against various
55 proprietary implementations. Let us not forget that the broken and buggy nature
56 of the common proprietary implementations is the very reason why we need
57 FreeCalypso in the first place.
58
59 Target interface options
60 ========================
61
62 Our fcup-* tools can communicate with the AT-command-speaking target in one of
63 two ways:
64
65 * The default is the standard AT command interface over a dedicated UART. As
66 of this writing, the only FreeCalypso device that provides a full-featured AT
67 command interface of this kind is our FCDEV3B modem, but the ultimate goal of
68 the project is to build our own end user phone handset (a Libre Dumbphone)
69 that will also provide a full-featured AT command interface on its USB port
70 via a built-in CP2102 or FT232R chip.
71
72 * As a dirty hack, one can run FreeCalypso GSM fw on some alien hw targets,
73 currently Motorola C139 and Pirelli DP-L10. In this hacked-up configuration
74 there is no dedicated UART available for a standard AT command interface, but
75 there is a hack that allows a limited subset of AT commands to be passed over
76 the RVTMUX binary packet interface provided by the running FreeCalypso GSM fw.
77 Our fcup-* tools can work with such targets to a limited extent.
78
79 The AT-over-RVTMUX mechanism was originally invented back in 2015 as a
80 development aid, and was never intended for production use or to support any
81 kind of end user functionality. One of its limitations is that the strings
82 that are sent to ATI via this interface are limited to 254 characters, whereas
83 sending or writing SMS in hex format requires longer strings. As a result, SMS
84 sending and writing functionality via fcup-smsend* is limited when a crippled
85 Motorola or Pirelli hw target is used instead of proper FreeCalypso hardware.
86
87 All fcup-* tools take the following common command line options for selecting
88 the AT command target interface:
89
90 -B baud Valid only when -p is also given; selects a different baud rate
91 than the default 115200 bps.
92
93 -n Dry run debug mode with no target interface at all: the AT
94 commands which would otherwise be sent to the target are simply
95 printed on stdout.
96
97 -p ttyport Names the serial port to be used to talk to the target.
98
99 -R Use the AT-over-RVTMUX interface instead of the standard AT
100 command interface over a dedicated UART.
101
102 -X program Use the specified external program as the AT target
103 communication back-end; read the source code for the details.
104
105 -R and -p options interact as follows:
106
107 Neither -R The standard dedicated AT command interface is used;
108 nor -p FC_GSM_DEVICE= environment variable needs to be set
109 to point to the serial port.
110
111 -p only The standard dedicated AT command interface is used;
112 the serial port is named with the -p option.
113
114 -R only AT-over-RVTMUX interface is used; the fcup-* tool connects
115 to an already running rvinterf process.
116
117 -R and -p AT-over-RVTMUX interface is used; a new rvinterf process
118 is launched to talk RVTMUX on the specified serial port.
119
120 Retrieving and decoding stored SMS
121 ==================================
122
123 As of this writing, our current FreeCalypso GSM firmware supports only SIM
124 storage for SMS, i.e., there is no working mechanism currently for storing SMS
125 records (received and sent messages) in the phone's or modem's own flash file
126 system. The capacity of this SIM SMS storage is determined by the SIM issuer,
127 but it is typically quite limited, on the order of 20 to 30 messages.
128
129 The model adopted for FreeCalypso is that incoming (and possibly saved outgoing)
130 messages initially accumulate in the SIM storage as they come in, and then the
131 user periodically transfers them to her larger host computer, simultaneously
132 deleting them from the SIM storage to reclaim the limited space. The retrieval
133 of stored SMS from FreeCalypso GSM devices is accomplished with our fcup-smdump
134 utility; unlike SMS sending/writing, this operation works exactly the same
135 whether the FC GSM device offers a full-featured AT command interface or only
136 AT over RVTMUX. SMS retrieval is always done in PDU mode, and the output from
137 fcup-smdump contains raw SMS PDUs in the form of long hex strings. A separate
138 utility called sms-pdu-decode then does what its name says.
139
140 The intended mode of usage is something like this:
141
142 fcup-smdump -d >> long-term-sms-log
143
144 The -d option to fcup-smdump tells it to delete the retrieved messages from the
145 SIM or future ME storage; this option should only be used when the output is
146 redirected into some kind of longer-term storage. In the above model the file
147 named long-term-sms-log becomes what its name says as new messages retrieved
148 from the FC GSM device get added to it; the format will look like this:
149
150 Received message:
151 XXXXXX...
152
153 Received message:
154 XXXXXX...
155
156 Sent message:
157 XXXXXX...
158
159 Stored unsent message:
160 XXXXXX...
161
162 Received message:
163 XXXXXX...
164
165 Each of the "XXXXXX..." lines will be a long hex string giving an SMS PDU. The
166 idea is that the complete record of all received and sent messages should be
167 stored on the user's big computer in raw PDU form, rather than decoded, and the
168 decoding utility sms-pdu-decode should be invoked by the user (with the message
169 log file as input) as needed for reading these messages.
170
171 The message decoding utility sms-pdu-decode does its best to decode and show
172 everything without dropping any bits: in addition to the actual decoded message
173 characters and the From/To address (the "end user" content of the message), it
174 decodes and shows the SC address, the first octet, the MR octet for outgoing
175 messages, PID and DCS octets, the SC timestamp or the validity period fields,
176 and the UDH bytes if present. However, some bits can still be lost in the
177 decoding, which is why it is important to archive messages in the raw PDU form:
178
179 * Padding bits used to round the From/To address and septet-based user data to
180 an octet boundary and to round any UDH to a septet boundary are not decoded.
181
182 * If the user data portion of the message is 8-bit or compressed data (per the
183 DCS octet), it is shown as a raw hex dump, which is lossless, but if it is
184 GSM7 or UCS-2 text (GSM 03.38 character encodings), the characters are
185 converted to the user's character set (plain ASCII only by default) for
186 display, and some characters may not be displayable.
187
188 Character sets and encodings
189 ----------------------------
190
191 By default, sms-pdu-decode only emits 7-bit ASCII characters in its output; any
192 GSM7 or UCS-2 characters which fall outside of this plain ASCII repertoire are
193 displayed as the '?' error character and the presence of such decoding errors
194 is indicated in the Length: header. This conservative default behaviour can be
195 modified as follows:
196
197 -e option extends the potential output character repertoire from 7-bit ASCII to
198 8-bit ISO 8859-1. Any 8859-1 high characters are emitted as single bytes,
199 i.e., are NOT encoded in UTF-8 - this option is intended for non-UTF-8
200 environments.
201
202 -u option extends the potential output character repertoire to the entire Basic
203 Multilingual Plane of Unicode, and changes the output encoding to UTF-8.
204
205 -h option causes the user data portion of every message to be displayed as a
206 raw hex dump; in the case of GSM7-encoded messages, this hex dump shows the
207 unpacked septets.
208
209 Composing and sending outgoing SMS
210 ==================================
211
212 When used with a FreeCalypso GSM device that offers the full AT command
213 interface (currently only the FCDEV3B modem), the primary SMS sending/writing
214 tool fcup-smsend offers the following capabilities:
215
216 * Sending outgoing messages in either GSM7 or UCS-2 encoding;
217 * Sending either single or long (concatenated) SMS;
218 * Message body input in ASCII, ISO 8859-1 or UTF-8;
219 * Message body input either on the command line or on stdin;
220 * Any messages sent through this tool (single or concatenated) may be
221 multiline, i.e., may contain embedded newlines;
222 * Messages sent in GSM7 encoding can contain ASCII characters [\]^ and {|}~
223 - the tool is smart enough to do the necessary escape encoding.
224
225 The default and preferred AT command interface mode for sending/writing SMS is
226 PDU mode, which works great when the GSM device provides a proper AT command
227 interface. However, when a message of maximum or near-maximum length is being
228 submitted to the modem in PDU mode, the hex string that needs to be sent is
229 longer than what the crippled AT-over-RVTMUX mechanism can handle, thus if you
230 are using crippled Motorola or Pirelli hardware, you need to give the -t option
231 to fcup-smsend or fcup-sendmult, telling these tools to use text mode instead
232 of PDU mode on the AT command interface. In this text (-t) mode the following
233 restrictions apply:
234
235 * Only single SMS can be sent, not concatenated;
236 * Only GSM7-encoded messages can be sent, not UCS-2;
237 * No multiline messages can be sent, i.e., no newlines in the message body;
238 * ASCII characters [\]^ and {|}~ won't be sent correctly - GSM 07.05 text mode
239 drops them.
240
241 But if you have to use FreeCalypso on crippled hardware, the -t option does
242 allow you to send GSM7-encoded SMS of the full maximum length of 160 characters.
243 If you attempt to use PDU mode (no -t option) with an AT-over-RVTMUX back-end
244 (-R option), the send or write operation will fail if the generated PDU is
245 longer than 127 octets; the length of the generated PDU depends not only on the
246 message body length, but also on the length of the destination address.
247
248 The invokation syntax is as follows:
249
250 fcup-smsend [options] dest-addr [message]
251
252 The destination address must be given on the command line; the address digits
253 may be optionally followed by a comma and an address type byte, either decimal
254 or hexadecimal with 0x prefix. The default address type is 0x91 if the number
255 begins with a '+' or 0x81 otherwise. If the message body is given on the
256 command line, it must be given as a single argument; if no message body argument
257 is given, the message body will be read from stdin. Any trailing newlines are
258 stripped before SMS encoding.
259
260 The following options are supported, in addition to the common target interface
261 options listed earlier:
262
263 -c Enables concatenated SMS. Concatenated SMS will be sent only
264 if the message body exceeds 160 GSM7 or 70 UCS-2 characters,
265 otherwise plain SMS will be sent whether -c is given or not -
266 but the -c option enables the possibility of sending
267 concatenated SMS.
268
269 -C refno Enables concatenated SMS like -c, but also explicitly sets the
270 concatenated SMS reference number to be used. The number can
271 be either decimal or hexadecimal with 0x prefix.
272
273 -q Concatenated SMS quiet mode. If -c is given without -q, the
274 tool prints a message on stdout indicating whether the message
275 was sent as single or concatenated, and in how many parts.
276 -q suppresses this additional output.
277
278 -t Use text mode instead of PDU mode on the AT command interface.
279 This option is incompatible with -c and with -U, and introduces
280 other restrictions listed above.
281
282 -u By default, if the message body input contains any 8-bit
283 characters, they are interpreted as ISO 8859-1. With -u they
284 are interpreted as UTF-8 instead. This option is only relevant
285 for GSM7 output encoding, and it is implemented by converting
286 the input first from UTF-8 to 8859-1, and then from 8859-1 to
287 GSM7 - thus all UTF-8 input characters must fall into the
288 8859-1 repertoire, and it is not currently possible to send
289 GSM7-encoded messages containing the few Greek letters or the
290 Euro currency symbol allowed by GSM 03.38 encoding.
291
292 -U Send message in UCS-2 encoding instead of GSM7. Any 8-bit
293 characters in the message body input are interpreted as UTF-8,
294 and the entire Basic Multilingual Plane of Unicode is allowed.
295
296 -w By default the outgoing message is sent out on the GSM network
297 with the AT+CMGS command. With this -w option, the message is
298 first written into SIM or future ME SMS storage with AT+CMGW,
299 then sent out on the GSM network with AT+CMSS.
300
301 -W Write only, not send: the message is written into storage with
302 AT+CMGW and no further action is taken. The modem's +CMGW:
303 responses with message storage indices are forwarded to stdout.
304 With this option the destination address argument can be a null
305 string or omitted altogether.
306
307 Concatenated SMS reference numbers
308 ----------------------------------
309
310 Every concatenated SMS transmission needs a reference number, and this number
311 needs to increment from one concatenated SMS to the next, to help message
312 recipients sort out which is which. If the reference number is not given
313 explicitly with -C, fcup-smsend creates (opens with O_RDWR|O_CREAT) a file
314 named .concat_sms_refno in the invoking user's $HOME directory; automatically
315 incrementing reference numbers are maintained in this file. The initial seed
316 is an XOR of all bytes of the current time returned by gettimeofday(2),
317 followed by simple linear incrementing; these reference numbers do not need to
318 be random in any kind of cryptographically secure sense.
319
320 fcup-smsendmult
321 ===============
322
323 As an alternative to sending concatenated SMS, one can use the fcup-smsendmult
324 utility to send several single (no UDH) messages in one batch. This utlity
325 supports both text and PDU modes (PDU mode is still the preferred default when
326 it can be used), and when PDU mode is used, it supports both GSM7 and UCS-2
327 output encodings just like fcup-smsend. The messages to be sent are read from
328 stdin, and each input line produces a new message.
329
330 The entire batch of messages can be sent to a single recipient, or each message
331 in the batch can have its own individual destination address. If the
332 destination address is given on the command line, each input line read from
333 stdin is just a message body; if no destination address is given on the command
334 line, each input line must have the following format:
335
336 <dest addr><white space><message body>
337
338 -t, -u, -U, -w and -W command line options are unchanged from fcup-smsend.
339
340 If you have to use FreeCalypso on crippled hardware, fcup-smsendmult -t can be
341 a viable alternative to sending concatenated SMS, as each message in the batch
342 can be up to the maximum limit of 160 characters.
343
344 fcup-smsendpdu
345 ==============
346
347 This utility sends out SMS PDUs that have been prepared externally; it only
348 works in PDU mode, which limits its usefulness to high-end FreeCalypso hardware
349 with a full AT command interface. The PDUs to be sent out are read from stdin,
350 one long hex string PDU per line; one can send either a single message or a
351 batch. Because the destination address and all content details are encoded in
352 the PDU, the tool does not care if the messages are going to the same recipient
353 or to different recipients, nor does it care if they constitute a concatenated
354 SMS transmission or not. -w and -W options work the same way as in fcup-smsend
355 and fcup-smsendmult.
356
357 fcup-smwrite
358 ============
359
360 This utility is a debug and development tool; it differs from fcup-smsendpdu in
361 the following ways:
362
363 * fcup-smsendpdu can send messages out with AT+CMGS, write them into memory
364 with AT+CMGW, or do a write-then-send sequence (-w option) with AT+CMGW
365 followed by AT+CMSS. fcup-smwrite only issues AT+CMGW commands.
366
367 * fcup-smwrite passes a second argument to AT+CMGW that sets the message state
368 to any of the possible 4 values; fcup-smsend* -W put them in the "stored
369 unsent" state.
370
371 * The input to fcup-smsendpdu is just PDU hex strings; the input to
372 fcup-smwrite needs to have the same format as fcup-smdump output in order to
373 indicate what state each message should be written in.