FreeCalypso progress update

[Mychaela Falconia]

Hello FC community,

I've been asked for a status update, so here it is...

My primary FreeCalypso activity right now is exchanging emails back
and forth with Rohde&Schwarz people at their service center in
Maryland - I sent my CMU200 to them to have it externally calibrated,
as I had no way of knowing where its measured and generated signal
power levels stood relative to the real truth.  The good news on that
front is that they already sent me an incoming test report showing the
state of the unit as it came to them, and among other things that
report shows the power level offsets which I was after, i.e., exactly
how the power level measurements reported by this CMU200 and the
signal levels it generates differ from the true value.  Thus even if I
take the option of having them send the unit back to me without doing
anything to it, I can use the numbers from their report to adjust the
power levels in our software (fc-cmu200d).

There is also a complication, however, in that the non-technical front
end customer service people told me that the unit needs repairs before
it can be calibrated, accompanied by an exorbitant price quote for
said repairs.  Thankfully they also offered the option of sending the
unit back to me without doing anything to it for half of the original
calibration price quote (which I would be OK with as the price for the
report giving the very important power level offset data which we
didn't have before and had no other way of obtaining), and I may very
well end up going with that option - but first I am trying to extract
as much technical info out of them as I can regarding what they think
is wrong with this CMU200 beyond it being out of calibration after the
internal module swap which I did to repair it earlier.  The process of
extracting this additional technical info out of them is moving
slowly, and it will probably be another few days before we reach a
decision as to whether they are going to send the instrument back to
me as-is or if we are going to agree to some reasonably-priced repair
or calibration services.

On the FCDEV3B front, we now know that we can fix the sleep mode
problem by populating the smaller flash+RAM chip that was used by
Openmoko; the next step in the investigation is to figure out if we
can make the sleep mode problem go away *without* giving up our larger
RAM and flash capacity copied from the Pirelli DP-L10.  Between the
back-and-forth with R&S Maryland and other life business (outside of
FreeCalypso) which I need to attend to, it will probably be another
couple of weeks before I'll be ready to schedule the next hardware
rework experiment session with Technotronix, but the actual experiment
plan is already set: we are going to try upping the bypass capacitors
on the V-FLASH and V-SRAM power nets.  (Note that these are the
internal power nets from Iota regulators to the memory IC, *not* the
input power to the board or the SIM power net which someone on this
list kept harping about some months back.)

In between all of the above, I am using my idle cycles in between
interrupts to progress slowly toward the end user phone goal, i.e.,
working on various software pieces that are needed toward that goal,
but which are currently missing or broken.  There is little in the way
of sequential ordering: there is a large set of software pieces which
need to be implemented or fixed, all of them are needed before we can
have any kind of usable phone, but they can be worked on in any order
for the most part.  Thus I pick the piece to be worked on based on
feelings.

Right now I am tackling SMS storage and the tools needed for working
with this storage.  The code we got from TI has non-broken support
only for storing messages on the SIM, both received and sent ones, in
both AT command modem and self-contained handset configurations.
There is also a provision for storing messages in the Calypso device's
own flash file system ("on the phone" or in the modem, depending on
one's point of view), but it is broken; TI's UI code never attempts to
enter this mode, and commanding it with AT+CPMS would put the SMS
subsystem into a non-working state.

The current broken implementation of in-modem or "on the phone" SMS
storage ("ME" storage in AT+CPMS terms) is based on the old and highly
inefficient PCM API from Condat, and for this reason I am not
particularly interested in trying to fix it in its present form -
instead I would like to reimplement it in my own different way.  Aside
from changing the way in which the storage is implemented in terms of
FFS facilities (flash usage and access efficiency issues), I would
like to add a local timestamp to each stored message.  When SMS
storage is implemented according to the public and vendor-neutral GSM
standards, i.e., when the messages are stored on the SIM, received
messages only have a network timestamp set by the SMS Service Centre
(the network component that does the store and forward), whereas sent
messages have no timestamps at all.

Right now I use a Pirelli DP-L10 running Pirelli's proprietary fw as
my personal phone, and I use Pirelli's "on the phone" message storage.
The message bits are stored in FFS in a format controlled by Pirelli's
proprietary fw which I have only partially reversed, but the essential
data model is the same as it would be with the 100% standards-based
on-the-SIM storage: received messages only have a timestamp controlled
by the network, not one set by the receiving phone itself, and sent
messages have no timestamp at all.  Thus when I look through my sent
messages, I can see whom I sent a given message to, but absolutely no
information as to when - this info is simply not preserved anywhere.
The received messages do have network-set timestamps, but they are in
the network's timezone, which differs from my own personal timezone
for about 8 months out of each year (I commit civil disobedience by
refusing to obey DST and instead keep my body's biological clock and
all of my personal electronic clocks on natural mean solar time all
year round), so even then the timestamp information is poor.

I plan on solving this problem in FreeCalypso by adding a local
timestamp based on the phone's or modem's own clock to each stored
message, both sent and received; this addition will only be possible
when messages are stored on the FreeCalypso device itself and not on
the SIM (the latter storage format is rigidly fixed by standards), and
when I implement the new "ME" SMS storage mechanism, I would like to
have the room for the local timestamps allocated in the data structures
from the beginning.  But it will be a fairly significant amount of
work, so I am currently putting it off for later; the first FreeCalypso
end user phone will only offer on-the-SIM storage of messages, meaning
limited capacity and no local timestamps.

Even when we do implement our own FFS-based message storage facility,
we will always need to retain the ability to work with standards-based
SIM storage, i.e., the ability to read messages stored on the SIM,
clear them out, and if the user needs to for some reason, store new
messages (both sent and received) in there as well.  Thus I've been
working on software tools to do the latter.

So far I've implemented the fcup-smdump utility that retrieves saved
messages from a FreeCalypso AT-command-speaking device via the AT+CMGL
command in PDU mode (meaning that the messages are retrieved in the
same raw bit form as how they are stored on the SIM) and optionally
deletes them from the SIM storage afterward.  Because SIM message
storage is quite limited (the exact capacity depends on the SIM
issuer, but it is typically 20 to 30 messages), my envisioned usage
model is that long-term message storage (potentially spanning years)
is to be kept on the user's personal computer, with the user
periodically (once a week or so) retrieving accumulated messages from
her phone, saving them on her computer long-term, and clearing them
from the capacity-limited SIM storage.  The fcup-smdump utility I just
wrote accomplishes exactly what I just described.

The output from fcup-smdump is raw SMS PDUs in the form of long hex
strings, and it is my vision that this is how the messages should be
stored for long-term record-keeping on the user's full-size computer.
For display of the stored message logs, there is the sms-pdu-decode
utility: it fully decodes each SMS PDU, showing not only the user
text, but also all of the header fields (which are typically hidden
from users) in fully decoded form.

Next to be implemented are tools for sending outgoing SMS from the
command line on a full-size computer connected serially to the phone
or modem: while we certainly need the ability to send SMS
conventionally by painfully entering message text through the phone's
numeric keypad, as smart, geeky and empowered users we also need the
ability to compose outgoing SMS in vi on a real computer and then send
them out through the FreeCalypso phone or modem with a command line.
I'm already working on those tools - stay tuned.

Hasta la Victoria, Siempre,
Mychaela aka The Mother