FreeCalypso > hg > freecalypso-tools

comparison doc/Nonfree-phones-user-data @ 808:ac527de89a0d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
pcm-sms-decode documentation
author Mychaela Falconia <falcon@freecalypso.org>
date Sat, 27 Mar 2021 00:28:49 +0000
parents
children
comparison
equal deleted inserted replaced
807:3a80bfa87496 808:ac527de89a0d
1 Using FreeCalypso tools to access user data on non-free phones
2 ==============================================================
3
4 If you are using a Motorola C1xx or Pirelli DP-L10 phone as your daily driver,
5 running its original proprietary fw for the lack of anything better, you can
6 still use FreeCalypso tools to get to some of your user data, namely, your
7 phonebook and your SMS store. You can even do the same with a non-Calypso
8 phone if you can move the SIM in and out of it. This article tells you how to
9 do such tricks.
10
11 Phonebook howto
12 ===============
13
14 Whenever you have to use a phone whose firmware is not fully liberated, you
15 should always store your phonebook contacts on the SIM, rather than "on the
16 phone". SIM phonebook storage format is strictly defined by standard specs,
17 leaving no room for proprietary firmwares to apply any wacky proprietary ideas
18 of their own - OTOH, "on the phone" storage really means storing your data in
19 proprietary fw data structures in only-Cthulhu-knows-what format.
20
21 If you have your contacts stored in your SIM phonebook, you can manipulate the
22 latter with FreeCalypso tools: you can dump your SIM phonebook and save it on
23 your larger host computer, you can restore these backups back to the SIM, and
24 you can freely edit your SIM phonebook. If your phone is a supported Calypso
25 model, you can perform all of these manipulations without removing your SIM -
26 you only need to shut down the phone's regular fw for the duration of the
27 operation, just like you would do for flashing with fc-loadtool. The other way
28 is to remove your SIM from the phone and stick it into a standalone smart card
29 "reader" device - this path is totally independent of all phones.
30
31 You will need to have both the present FC host tools package and the separate
32 FC SIM tools package installed on your host machine. See the instructions in
33 the SIM-manipulation article for how to run fc-simint for the purpose of
34 operating on a SIM card inside your Calypso phone. Once you have landed at the
35 simtool> prompt, see the doc/User-oriented-commands article in the fc-sim-tools
36 repository for instructions on how to enter your PIN (if your SIM requires one)
37 and how to manipulate your phonebooks.
38
39 SMS storage howto
40 =================
41
42 Motorola's firmware always stores received SMS on the SIM, whereas sent SMS are
43 stored in their FFS (flash file system, see Compal-FFS article) in their own
44 proprietary format. You can transfer your received SMS to your larger host
45 computer for longer-term archival as follows:
46
47 1) Shut down the phone's regular fw and run fc-simint to operate on the SIM;
48
49 2) Once you are at the simtool> prompt, enter your PIN if your SIM requires one,
50 and then save the SIM SMS store to a Unix host file with the save-sms-bin
51 command, as explained in the doc/User-oriented-commands article in the
52 fc-sim-tools repository. You can then issue an sms-erase-all command in the
53 same seesion to clean out the SIM SMS store.
54
55 3) The files saved by the fc-simint/fc-simtool save-sms-bin command are binary,
56 but the message content can then be fully decoded to human-readable ASCII
57 (or ISO 8859-1 or UTF-8 if desired) with our pcm-sms-decode utility,
58 described later in this article.
59
60 To retrieve your sent SMS from Compal's FFS, you will need to similarly shut
61 down the phone's regular fw and read out the FFS sectors with fc-loadtool. You
62 can then use our tiffs utility to parse the FFS structure, but the format of the
63 actual files used by Compal's proprietary fw implementation has not been studied
64 in detail by the Mother, thus you will need to use your own brain cells for
65 further decoding.
66
67 If you use a Pirelli DP-L10 phone rather than Motorola C1xx, Pirelli's fw gives
68 you the option of storing received SMS either on the SIM or "on the phone", the
69 latter meaning Pirelli's FFS. Sent SMS are always stored in Pirelli's FFS.
70
71 If you use SIM storage for received SMS on your Pirelli phone, then the
72 procedure for retrieving those SMS is the same as for Mot C1xx, as detailed
73 above. However, with this phone model, configuring the fw to store received SMS
74 in its FFS ("on the phone") is generally preferable: Pirelli's fw maintains a
75 file in its FFS (/pcm/SMS) whose binary format is exactly the same as the
76 standards-defined format for storing SMS on the SIM, thus reading out /pcm/SMS
77 with fc-fsio is just as good as reading out EF_SMS with fc-simint or fc-simtool,
78 but is much more convenient: fc-fsio operates on the phone while its regular fw
79 is running, whereas direct SIM manipulation with fc-simint requires shutting
80 down the phone's regular fw. Pirelli's fw also accesses its SMS store faster
81 when FFS storage is used, rather than SIM.
82
83 Decoding received SMS binary files with pcm-sms-decode
84 ======================================================
85
86 Whether you have retrieved your SMS store from a SIM with fc-simint/fc-simtool
87 save-sms-bin command or read out Pirelli's /pcm/SMS file with fc-fsio, the
88 binary format is exactly the same: a bundle of 176-byte records directly abutted
89 together, with each individual record having the standard format defined in GSM
90 TS 11.11 and its successor 3GPP TS 51.011. Looking at raw hex dumps won't be
91 very helpful, as the reversed nibble format used for phone numbers and SC
92 timestamps is not directly human-readable, and the message bodies will usually
93 be in the form of packed septets - hence a special decoding program is needed.
94
95 Our pcm-sms-decode utility does the needed job of reading these binary files and
96 fully decoding them into human-readable ASCII, or ISO 8859-1 or UTF-8 if your
97 host system supports such extended character sets and you have received SMS
98 content that uses beyond-ASCII characters. pcm-sms-decode was added to FC host
99 tools suite in early 2020; it is based on and shares most of its code with our
100 earlier sms-pdu-decode utility from 2018. (The latter is meant for decoding SMS
101 PDUs retrieved from FreeCalypso AT command modems with fcup-smdump.)
102
103 pcm-sms-decode uses the same backslash escapes in its output and accepts the
104 same -e, -u and -h options as sms-pdu-decode - please refer to the description
105 of sms-pdu-decode in the User-phone-tools article for the details.
106
107 By default, pcm-sms-decode numbers the records being decoded as 0 to N-1, where
108 N is the total number of records in the binary file given to it (file size equal
109 to N*176 bytes). As of fc-host-tools-r15, newly added -s option causes it to
110 number these same records as 1 to N instead. The original 0-based record
111 numbering was implemented when the tool was used solely to decode /pcm/SMS from
112 Pirelli's fw (before the introduction of fc-simtool), but now that we have
113 fc-simtool and fc-simint, and given that the SIM interface protocol and SIM file
114 system definition use 1-based record numbers, we need to support the same
115 convention in pcm-sms-decode for consistency.