New SIM card testing and peek/poke tool

[Mychaela Falconia]

Hello FreeCalypso community,

Just a quick note to let everyone know what I am working on these
days: I am continuing to further develop my fc-simtool utility for
manipulating (reading and writing) SIM cards.  This utility began its
life about a month ago in the freecalypso-hwlab repository, but has
now been moved to a new Hg repository of its own:

https://www.freecalypso.org/hg/fc-pcsc-tools/

There is a doc subdirectory in the new repository, and I got two
documentation articles written so far: Low-level-commands and
User-oriented-commands.  I invite all potentially interested parties
to read these two documentation articles and see if this new tool is
something you might like to play with.  If you do wish to play with
fc-simtool, you will need one of these SIM card "reader" (CCID)
devices:

http://shop.sysmocom.de/products/cm3121
http://shop.sysmocom.de/products/cm6121

If the card you wish to play with is still in its original credit-card-
sized plastic carrier piece, you will need the 3121 CCID; if you are
going to play with SIMs that have already been broken out (just the
"plug-in form factor" piece that goes into phones), you will need the
6121.  I have both, as well as this adapter for examining other
people's 3FF/4FF SIMs pulled out of abomination phones:

http://shop.sysmocom.de/products/sim-adapter-pcb

One particularly useful feature of fc-simtool is SIM phonebook
manipulation: it allows you to examine, backup, restore and manually
edit SIM phonebooks.  Harald Welte gave a presentation on SIM cards at
the last CCC before the world went to hell (36C3), and in that talk he
made a statement along the lines of "no one stores contacts on SIMs
any more".  I am here to refute that statement: there is one person
still alive on Earth who does store contacts on SIMs, and that person
is me!  And if anyone wishes to join me in storing contacts on SIMs,
fc-simtool is the perfect tool to manage them.

Why store contacts on SIMs?  Consider this analysis: when you use any
classic GSM/2G dumbphone, it will typically offer you the option of
storing your contacts either on the SIM or "on the phone".  What does
"on the phone" actually mean?  Answer: it means contacts being stored
in some proprietary format invented by the authors of that phone's
particular proprietary fw, and known only to that proprietary fw.
Sure, it may be possible to reverse-engineer that proprietary "on the
phone" phonebook storage format - but why take the extra pain?  OTOH,
if you store your contacts on the SIM, they are stored in a format
that is defined by real published standards, not proprietary - the
standards are GSM TS 11.11 and its successor 3GPP TS 51.011.  It is a
standard format for which anyone in the world can freely write his or
her own tools, as I have just proven with fc-simtool.

If we ever get to a point of usable FreeCalypso fw running on either
Motorola C139 or Pirelli DP-L10 (or any other pre-existing phone hw
model), our FC fw will most definitely NOT share any FFS with that
phone's original proprietary fw - our FreeCalypso FFS in such
aftermarket hack configurations is always our own.  Therefore, any
"on the phone" contacts *cannot* be shared between the phone's
original proprietary fw and future FC fw - neither fw will grok the
other's ad hoc formats for storing user data.  But if you have
contacts stored on your SIM, those contacts are accessible to every fw
on every phone, by the very design of GSM and the whole concept of SIM
cards.

I do take the opposite stance for SMS storage though.  Contacts are
long-term data: I have some contacts in my SIM phonebook which were
created many years ago and are still active.  In contrast, all message
stores for SMS (whether SIM-based or phone-based) are transient by
nature: the message store fills up with received and sent SMS, its
capacity is limited, and the user has to clean it out.  In my personal
workflow, I also strongly prefer to transfer the content of my message
store from the phone to my full-size computer for longer-term storage
(years) before cleaning it out.  All of these flows can work the same
whether the message store is SIM-based or phone-based, but my plan for
FreeCalypso fw is to implement a local FFS-based SMS store that follows
the same data model as the standards-defined EF_SMS on SIMs, with only
minor extensions.  Having this local FFS-based SMS store to replace
the SIM-based one for everyday usage will give us the following
benefits:

* SIM SMS store capacity is up to the SIM card issuer - my legacy
T-Mobile SIMs have room for 30 records in their EF_SMS.  Our own local
FFS-based SMS store will be able to hold all the way up to the
architectural limit of 254 records.

* One really unpleasant limitation of the standard SMS storage data
model as implemented in EF_SMS is the lack of timestamps on sent
messages.  Received messages have their network (SC) timestamps, but
not sent ones.  Pirelli's fw does not use SIM storage for sent SMS
(received ones can be stored on the phone or on the SIM, but sent ones
are only stored on the phone), but its proprietary "on the phone"
storage format for sent SMS has the same data model: you can see the
message content and whom you sent it to, but not when you sent it.  My
idea is to implement our own message store for SMS whose data model
would be almost exactly the same as the standard SIM-based EF_SMS, but
with one extra data field added for a local timestamp, based on the
phone's own user-controlled notion of date and time.

* Not constantly writing to the SIM for SMS storage and deletion will
help prolong the SIM's internal flash write endurance.

Right now I use a Pirelli DP-L10 running its original proprietary fw
as my "daily driver" phone, and I have it configured to store all SMS
on the phone, not on the SIM.  It stores up to 50 messages, and when
this capacity fills up, I use fc-fsio to read out the relevant parts
of Pirelli's proprietary FFS before cleaning out the message store.
Far from perfect, but it is my current interim solution in the absence
of a usable FreeCalypso phone.

But if someone does have SMS stored on their SIM, then you can read
them with fc-simtool: use the tool's save-sms-bin command to save the
content of EF_SMS in a binary file, then run pcm-sms-decode from FC
host tools (a utility I wrote earlier for decoding Pirelli's /pcm/SMS
which has exactly the same format) to fully decode and display the
message content.

Finally, one more project I am working on is trying to get our own
SIMs.  Some day soon we will need to start running our own GSM networks
to replace the big carrier-based ones which are being shut down left
and right, and we will need to issue our own SIMs.  Back when Sysmocom
sold their peach-colored sysmoUSIM-SJS1 cards, I was thinking of using
those peach SIMs for our own issue - they weren't perfect, I would
much rather have classic SIM only (no USIM) and a 2FF-only cut rather
than 2FF+3FF - but those peach SIMs seemed acceptable to me at that
time.  But now those peach SIMs have been discontinued, and their
successor sysmoISIM-SJA2 are even worse:

* The very name "sysmoISIM" (or anything-ISIM for that matter) is
rather offensive to me, glorifying that abominable 4G/LTE technology
that is the root cause of 2G shutdowns - ISIM is a thing for VoLTE,
and VoLTE implies LTE underneath.

* The new cards are triple-cut in 2FF+3FF+4FF, which is even worse
than the previous 2FF+3FF deal.

* The brand printing on Sysmocom's previous peach and mint-colored
SIMs was reasonable and non-offensive in my mind: just a company
promoting its products and services, nothing wrong with that.  But
the printing on the new black cards gives advertising and promotion
to newer-than-2G technologies, as well as unfair advertising and
promotion to the Osmocom organization to the exclusion of other
community projects in the cellular telephony arena that aren't Osmocom
and live under different domain names - not OK for me.

As a result of these developments, I feel that it is time for us to
bite the bullet and make our own SIMs.  I did inquire with Sysmocom
about making a batch of SIMs customized for us, as in 2FF-only cut,
our own FreeCalypso brand printing, and application customization to
remove USIM and ISIM, leaving only classic SIM.  I was told that what
I seek would be quite doable, but the price is rather steep: we would
have to order a minimum of 1000 cards (certainly enough for our needs
for many lifetimes) at the cost of around 4 kUSD.  Needless to say,
4 kUSD is a rather high price for a venture driven purely by personal
passion, without any expectation ever of any profit or return.

If I am unable to find any more affordable options, I *can* spend the
4 kUSD, and I will do that if I exhaust all other avenues.  However,
since the end of January I have been in a discussion with Grcard in
China (a rather well-known major vendor of SIM cards) about getting
the kind of programmable SIMs I want (2FF-only cut, SIM-only
functionality without USIM or ISIM), and they did offer me very
affordable prices.  Their asking price for 1000 SIMs (with custom
printing included) is less than half of Sysmocom's price, and they
also allow smaller quantity orders - right now I am considering buying
300 SIMs initially.  The problem, however, is extreme uncertainty:
they were going to send me a few sample cards for technical testing,
but the shipment hit some snafu with Chinese customs, and now we have
to wait till the end of their Lunar New Year holiday before they can
even begin another attempt at resending the package - and then it will
be gawd knows however much longer before I actually receive the cards.
And until I receive some sample cards for testing, we have no way of
knowing whether or not these SIMs really are what we want in technical
terms, i.e., whether or not they will actually work for our needs -
thus uncertainties all around.

But I am confident that even if it takes a long time, we *will*
eventually have our own SIMs, whether cheap Grcard-based ones or
expensive Sysmocom-based, and they will be fully programmable with
fc-simtool without needing any Python tools.

Hasta la Victoria, Siempre,
Mychaela aka The Mother