view CHANGES @ 798:ccaa1319740c

doc/SIM-manipulation article written
author Mychaela Falconia <falcon@freecalypso.org>
date Mon, 22 Mar 2021 00:24:34 +0000
parents 1a084d7512dc
children 843850c526b7
line wrap: on
line source

Changes in the current code since fc-host-tools-r14:

* New fc-simint host program and its associated simagent target back end work
  together with fc-simtool (separate FC SIM tools package), allowing the latter
  to operate on SIM cards inside Calypso phones and development boards.  See
  the new doc/SIM-manipulation article for the details.

Changes from fc-host-tools-r13 to fc-host-tools-r14:

* fc-loadtool got new commands for reading the OTP protection register of
  Intel-style flash chips; this OTP protection register holds the factory IMEI
  on Compal phones.

* fc-iram second program invokation mode fix: options for the second program
  are no longer misinterpreted as fc-iram options.

* rvinterf -b writes the PID of the backgrounded process into /tmp/rvinterf.pid;
  see doc/Rvinterf-tools article for the explanation.

* New support for target boot control via DTR and RTS modem control outputs as
  implemented on our new DUART28 adapter - see doc/DUART28-boot-control.

* New make-imeisv utility is an aid for shell scripts that need to construct a
  16-digit IMEISV from a 15-digit traditional IMEI, including Luhn check digit
  verification.

* ADC calibration table /sys/adccal is now supported much like various RF
  parameter tables: it can be decoded from binary to our own ASCII format with
  fc-bin2rftab, it can be compiled back into binary with fc-cal2bin, and it can
  be uploaded with fc-fsio upload-rf-table.

* Tiffs IVA (In Vitro Analyzer) new features, see doc/TIFFS-IVA-usage for more
  details:

  + New tiffs decode facility allows single-command extraction and decoding of
    RF parameter tables and some other FFS files for which tiffs cat -h is
    insufficient.

  + New tiffs-8m convenience wrapper for examining FFS images read out of Tango
    modules.

* New tiffs-mkfs utility can generate TIFFS images completely from scratch
  "in vitro", containing initial directory and file content given by a Unix
  directory and file tree on the host system.  The resulting image can then be
  flashed into a Calypso GSM device running FreeCalypso firmware.  There is
  also a companion utility named tiffs-mkfile that generates a few specific
  binary file types that go into TIFFS.

* A new target-utils program named lunadrv has been added: it is a driver for
  FreeCalypso Luna LCD, or rather a utility that allows this LCD hardware to be
  exercised and tested in a standalone manner, independent of complex firmware
  builds with phone UI functionality.  FC Luna is an LCD add-on to iWOW DSK or
  FC Caramel2 boards.

* fc-shell got a new batt command for sending MMI_BATTERY_IND primitives to
  target firmwares, to be used as part of phone handset UI firmware development.

* fc-fsio write-battery-table command has been replaced with new
  write-battery-table-v1 and write-battery-table-v2 commands.  Both commands
  accept the new battery table ASCII source format with a third field added for
  battery bars icon thresholds; write-battery-table-v1 writes the original
  /etc/batterytab format needed for Magnetite and Selenite firmwares, whereas
  write-battery-table-v2 writes the new /etc/batterytab2 format needed for our
  new Tourmaline firmware.  The corresponding back end programs are
  compile-fc-batt and compile-fc-batt2.

* New arfcn2ti and ti2arfcn utilities convert back and forth between standard
  ARFCNs and TCS211 L1 radio_freq numbers - see the new doc/ARFCN-mapping
  article.

* Documentation updates that don't involve any changes in actual code:
  doc/Loadtools-usage instructions for FC Tango and for new Compal targets
  (Mot V17x), doc/TIFFS-Overview updates for new tools including the new FFS
  editor (maintained externally), doc/IMEI updates for Compal IMEI discovery.

Changes from fc-host-tools-r12 to fc-host-tools-r13:

* fc-loadtool changes:

  + Memory dump commands (dump2bin and dump2srec) have been reimplemented using
    loadagent's new BINDUMP command, sending the dump stream from the target to
    the host in binary format instead of hex.  This change speeds up the dump
    transfer time by a little over 2x.

  + Flash programming commands flash program-bin, program-m0 and program-srec
    have likewise been changed to use a new binary protocol, producing some
    improvement in flash programming times - see the updated
    doc/Loadtools-performance article.

  + New combined erase+program commands have been implemented: flash
    e-program-bin, e-program-m0 and e-program-srec.  See the new
    doc/Flash-programming article for explanation of the intended usage.

  + Batch mode extended to run either a command script or a single flash
    command - see doc/Loadtools-usage for the details.

  + Scripting fix for Compal phones: flash erase-program-boot command can be
    put into a script followed by other commands, and the following commands
    will execute if the erase-program-boot operation was successful.
    Previously this command always stopped script execution because of an
    implementation quirk.

* fc-xram uses the new binary protocol to transfer the user's XRAM image to the
  target, producing a speed increase of 3.5x to 5x for the most common use
  cases.  Also added CRC-32 verification of the downloaded RAM regions before
  jumping to the downloaded image.

* New version of loadagent has added support for the new binary protocols for
  memory dumps, flash programming and XRAM loading.

* All loadtools programs:

  + The step of feeding the IRAM code image to the Calypso boot ROM has been
    sped up by about 700 ms by grouping contiguous S-record payloads into larger
    blocks (up to the boot ROM limit of 1014 bytes per block) instead of sending
    each S-record payload (30 bytes typically) as its own write command.

  + New -t option allows the wait for the boot ROM response to be time-limited,
    needed for automated (unattended) environments with target boot control.

Changes from fc-host-tools-r11 to fc-host-tools-r12:

* To facilitate objective comparison of loadtools performance between what the
  Mother sees on her Slackware reference system vs. what other users see on
  their different host systems, fc-loadtool now reports measured operation times
  for all lengthy operations: dump2bin, dump2srec and all flash programming
  commands.  fc-xram likewise reports the measured operation time for its
  singular operation of transferring an XRAM image.  The new
  doc/Loadtools-performance article explains some of the underlying theory and
  presents the performance numbers which you SHOULD be getting if your host
  system works as well as the Mother's Slackware reference.

* The mystery of poor fc-loadtool and fc-xram performance on newer Linux host
  systems with FTDI adapters has been solved and documented in the same
  doc/Loadtools-performance article.

* fc-loadtool changes:

  + Added support for Spansion S29/S71PL032J and S29/S71PL064J flash chips:
    these flash chips are found in some historical modem modules and may also
    be used in future mass-produced FreeCalypso modem products.  Historical
    flash chip Am29DL640G is also supported as it is identical to S29PL064J for
    our purposes.

  + The implementation of flash program-m0 and program-srec commands (needed
    for flashing classic TI fw images with large gaps between discontiguous
    program regions) has been revamped, making these commands work as well as
    our generally preferred flash program-bin command: same performance, same
    progress indication, same CRC-32 verification at the end.

* All loadtools programs: when feeding IRAM code images to the Calypso boot ROM,
  we now collect and check for the 00 04 bytes after the >p response, and we
  also collect the checksum byte which the boot ROM sends with its >c response.
  The old code which failed to collect the checksum byte after >c worked only
  by luck.

* loadtools config: the repertoire of -h targets has been significantly cleaned
  up and generalized, moving away from special cases and toward more generic
  configurations.  Please refer to the new doc/Loadtool-targets article for the
  details.

* For operating on GTM900 modem modules, the new -h gen8 loadtools target needs
  to be used instead of the old -h gtm900 config which has been removed; while
  most GTM900 variants have 4 MiB flash chips, some have been found to contain
  S71PL064J (8 MiB) flash instead.

* The old -h gta02 target for fc-loadtool and friends is deprecated and will be
  removed in the next release; please use the new -h fic target configuration
  instead.

* The ancient provision for building the loadtools subset of FC host tools in a
  special configuration for running on the application processor of OM GTA02
  has been removed.  It has been replaced with a generalized mechanism for
  target boot control - see the new doc/Target-boot-control article.

* mokosrec2bin default fill byte has been changed from 0x00 to 0xFF, producing
  binary images by default that match what would result in flash if the m0 file
  were to be programmed as-is.

* A pair of little ad hoc programs has been been added to the FC host tools
  suite for the purpose of dumping the Calypso chip's internal DSP mask ROM -
  see doc/DSP-ROM-dump.

* fc-fsio got some new commands for cleaning stale junk from FFS, typically
  needed when converting modems from alien manufacturers to FreeCalypso fw:

  + rm deletes an elementary object (file, empty directory or symlink); rm -f
    does the same but the case where the object to be deleted does not exist is
    treated as not-an-error.

  + rm-subtree deletes an entire subtree of directories and files starting with
    a given directory (equivalent of UNIX rm -r); -f option is also available
    like with plain rm.

  + cleandir removes all content of a directory (all files and subdirectories
    and their content), but keeps the remaining empty directory.

* Added some fc-fsio command scripts (installed in /opt/freecalypso/scripts)
  for preening GTM900-B and GTM900-P FFS content when converting these modem
  modules to FreeCalypso.

* New pcm-sms-decode utility can decode /pcm/SMS binary files read out of FFS
  maintained by certain firmwares (Pirelli DP-L10 is the prime use case),
  displaying the stored SMS content in human-readable form.

Changes from fc-host-tools-r10 to fc-host-tools-r11:

* Bugfix: the ability to override CFLAGS= from the top level (see INSTALL)
  has been fixed and now actually works as intended.

* New developer utility fc-bin2rftab can read or extract RF parameter tables
  from binary files and convert them into our FreeCalypso ASCII format.

* The way in which fc-loadtool handles different flash configurations has been
  completely redesigned; there should be no user impact on any previously
  supported target, but the new architecture makes it much easier to add
  support for new target devices and allows the possibility of different flash
  chip types (autodetected at run time) on the same -h target.  See the new
  doc/Loadtool-flash-support article for the details.

* fc-loadtool supports two new targets: Huawei GTM900 (-h gtm900) and
  Motorola W220 (-h w220).

* Iota poweroff command in the target-utils suite (primarily affecting the
  loadagent target program used by loadtools) now programs the VRPCAUX register
  like TI's TCS211 firmware does before sending the DEVOFF command.  This
  change fixes the erratic behaviour that was occurring on TI/FC development
  boards (D-Sample and FCDEV3B) when fc-loadtool (or fc-iram with some
  specialized target-utils program) was entered via the RESET button rather
  than PWON, followed by the soft poweroff operation and another switch-on via
  PWON.  For more details, please refer to doc/Target-utils and the
  Calypso-test-reset article in the freecalypso-docs repository.

* Aside from the just-described poweroff change, a lot of clean-up has been
  done throughout the target-utils suite: removed all use of bogo-millisecond
  delays which were previously copied from OsmocomBB; all programs with ABB
  access functionality can access the VRPCAUX register for manual debugging;
  helloapp clean-up; new watchdog experimentation commands in flash-boot-test;
  simtest no longer requires an explicit abbinit command.  A new
  doc/Target-utils document has been written describing this target-utils
  suite.

* Loadtools configuration for Mot C155/156 target (-h c155) fixed to use WS=4
  memory timing setting as used by the official fw on this hardware, as opposed
  to the previously-used WS=3 setting that was blindly copied from OsmocomBB.

* fc-xram got a new -j option for JTAG mode; see doc/Loadtools-usage.

* Documentation updates for the new bits of knowledge learned from the
  examination of a Sony Ericsson J100 phone: no code changes were needed to
  support flash manipulation and RF calibration data extraction on this model,
  but this is the first time the Mother got her hands on one of these phones,
  hence there was no documentation previously.

Changes from fc-host-tools-r9a to fc-host-tools-r10:

* For those who need to run our tools on a 64-bit host system (or more
  precisely, any system on which pointers are longer than the basic int type),
  the present release includes fixes for null pointer passing constructs
  throughout the code base.  The original code was only correct for classic
  systems on which ints and pointers are equivalent for K&R C function argument
  passing purposes.

* The back end program supporting fcup-* -R mode has been extended to support
  sending strings longer than 254 characters using the extended version of the
  AT-over-RVTMUX mechanism supported by newer firmwares - see the updates in
  the doc/User-phone-tools article.  A new unterm command has been added to
  fc-shell for the same functionality.

* New developer utility fc-rftab2c can convert per-band RF parameter tables
  into C code snippets for inclusion into FreeCalypso firmware sources.

* New target utility calversion can sometimes identify unknown Calypso chip
  versions - see doc/Calypso-chip-versions.

* New target utility simtest has been developed for the purpose of facilitating
  hardware troubleshooting - see doc/SIM-hardware-debugging.

* The tiny (120 bytes) flash-boot-wa.srec code piece which has been removed in
  fc-host-tools-r9 has been reinstated; it is not currently needed for anything
  (it was originally developed as a workaround for a problem that only affected
  one single defective board), but removing previously-released functionality
  for no good reason is not the way of FOSS.  See doc/Flash-boot-wa for more
  details.

* Compilation and installation: the Makefile hierarchy has been reworked to
  allow CC= and CFLAGS= to be overridden from the top for the entire package,
  and also to allow installation in a staging location - see INSTALL.

Changes from fc-host-tools-r9 to fc-host-tools-r9a:

* fc-loadtool can now correctly program the flash on Mot C155/156 phones:
  it turns out that their flash is like the one on TI's D-Sample board
  and needs the same partition quirks.

Changes from fc-host-tools-r8 to fc-host-tools-r9:

* mokosrec2bin utility added: it was originally written as a reverse
  engineering aid and was previously maintained in the freecalypso-reveng
  repository, but it has since been repurposed into a legitimate development
  tool.

* Added workarounds for FTDI USB-serial adapters that feed previously
  accumulated serial Rx bytes or outright garbage to newly started serial I/O
  programs:

  + fcup-atinterf back-end program for fcup-* does a sleep of 20 ms followed by
    tcflush(target_fd, TCIFLUSH) on startup to flush any previously accumulated
    serial Rx buffer garbage that would otherwise cause breakage.  The two
    delays in the deep sleep wakeup logic (see doc/Deep-sleep-support) have been
    reduced from 50 to 30 ms each, thus the total length of initial delays has
    been reduced from 100 to 80 ms.

  + In the rvinterf suite (see doc/Rvinterf-tools) a similar startup delay has
    been added in the special corner case of a non-interactive one-shot
    operation being commanded with rvinterf also being launched from the client
    program with -p; see the "Startup synchronization hack" section in the new
    Rvinterf-tools document.

* fcup-smdump now restores the character set selection (AT+CSCS) setting on the
  target to the default "IRA" (ASCII) upon successful completion, after
  initially setting it to "HEX" for the operation; the intent of this change is
  to avoid leaving a strange state for anyone using the AT command interface
  manually after fcup-smdump.

* fcup-smsend bugfix: the corner case of omitting the destination address
  argument with -W now works as documented.

* fcup-settime: minor code rearrangement should slightly reduce the unavoidable
  delay between the time retrieval on the host and that time being set on the
  target.

* A hack that was added in fc-host-tools-r5 solely to support one (1) defective
  FCDEV3B board with a particular flash boot defect has been removed.

* A number of documentation updates have been made: Deep-sleep-support and
  Rvinterf-tools articles are new, RVTMUX and TIFFS-Overview write-ups got
  long-overdue major updates, plus a number of minor updates as usual.

* For those who build the rarely needed ARM7 gcc toolchain from source, an old
  bug in the toolchain/build+install.sh script which was masked on the Mother's
  Slackware system for some reason has been fixed.

Changes from fc-host-tools-r7 to fc-host-tools-r8:

* fc-loadtool can now correctly program the flash on TI's D-Sample board
  after implementing the needed quirk for the Intel 28F640W30B flash chip.

* target-utils (loadagent and friends) serial output code changed to wait for
  the Tx FIFO not full condition instead of FIFO empty; this change makes these
  target utilities actually utilize the Tx FIFO feature of Calypso UARTs.

* New c1xx-calextr utility can extract the factory RF calibration values for
  Mot C1xx phones from a dump of their flash; see doc/Compal-calibration.

* Support for RF tables in FreeCalypso ASCII format (see doc/RF_tables) is
  more complete: there is an ASCII format for the full 512 byte Tx ramps table,
  any ASCII RF table can be converted to native binary with fc-cal2bin, most
  ASCII RF tables can be uploaded directly into FFS with fc-fsio's new
  upload-rf-table command.

* fc-cal2text change: Tx ramps tables are now emitted in the new ASCII format
  with one file for the whole table, not split into 16 single ramp template
  files any more.

* fc-fsio changes:

  + new write-battery-table and write-charging-config commands for uploading
    FreeCalypso-invented battery discharge threshold tables and charging config
    files into the FFS of our FC devices with built-in compilation to binary;

  + added upload-rf-table command for uploading ASCII RF tables directly into
    FFS with built-in compilation to binary;

  + pirelli-magnetite-init command now copies the previously missed
    /gsm/rf/afcdac record.

* fc-tmsh can now upload audio FIR coefficient sets from FIR coefficient table
  files with the new auw-fir command.

* The audio mode configurations that have been added in fc-host-tools-r7 have
  been moved to a separate fc-audio-config repository and are no longer a part
  of the core FC host tools package.

* The UI development aid utility fc-lcdemu which has been excluded from the
  default build set since fc-host-tools-r5 has been moved to a separate
  freecalypso-ui-dev repository and is no longer a part of the core FC host
  tools package.

* rvinterf -X mechanism which pipes LCD output to fc-lcdemu now always inverts
  the RGB565 values as needed, and the -v option has been removed; see usage
  documentation in the freecalypso-ui-dev repository.

* fc-shell: added key command for sending sp MMI CONFIG KEY_SEQUENCE=<string>
  with a lot less typing, and keydown and keyup commands that do the same for
  KEY_PRESS and KEY_RELEASE "config" sysprim commands.

* There is a new suite of tools for talking to the AT command interface
  provided by FreeCalypso phones and modems, primarily for sending and
  retrieving SMS - see doc/User-phone-tools for more info.

Changes from fc-host-tools-r6 to fc-host-tools-r7:

* The way in which loadtools, rvinterf and miscellaneous tools operate on
  serial ports for the purpose of talking to Calypso GSM devices has been
  majorly reworked: the serial port handling code has been factored out into
  libserial, and the new libserial uses Linux-specific direct ioctl calls
  instead of generic termios.  The effect is that you can now use the GSM-
  specific high serial baud rates (up to 812500 bps) with FTDI adapters
  (like the FT2232D adapter commonly used with FCDEV3B) with the standard
  unpatched ftdi_sio kernel driver.  See doc/High-speed-serial for more info.

* Added new utilities tiaud-compile and tiaud-mkvol for generating *.cfg and
  *.vol binary files for the audio mode switching facility of TI's RiViera
  Audio Service, to be uploaded under /aud into FreeCalypso GSM devices.

* A set of audio mode tables is built and installed in
  /opt/freecalypso/aud-fcdev3b for uploading into /aud on FCDEV3B boards.

Changes from fc-host-tools-r5 to fc-host-tools-r6:

* loadtools and rvinterf now set TIOCEXCL on the tty to prevent the common
  error scenario of multiple processes trying to read from the same serial
  port.

* fc-loadtool: added a batch operation mode whereby the tool executes a given
  command script, cleans up (usually powers off) the target and exits.

* fc-loadtool exec command: scripts without slashes in the given name are now
  sought in the current directory.

* fc-fsio format command: if the FFS is already formatted, the tool now prints
  the error returned by the FFS code in the target fw, but does not stop the
  execution of scripts; this change makes production scripts rerun-able.

* exitcodes.h file installed in /opt/freecalypso/include/rvinterf: added
  ERROR_RFFAIL definition for use by fc-rfcal-tools.

Changes from fc-host-tools-r4 to fc-host-tools-r5:

* New install location: everything is now under /opt/freecalypso instead of
  /usr/local/bin and /usr/local/share/freecalypso.

* loadtools: added support for dsample (TI D-Sample) and fcfam (FreeCalypso
  hardware family starting with FCDEV3B) targets.

* loadagent: UART autodetection now works not only with Calypso boot ROM
  version 0300 found in the common Calypso devices, but also with the older
  version 0200 found in the older Calypso silicon on early D-Sample boards.

* loadtools support for C139/140 target: use -h compal -c 1004 command line
  options instead of the previous -h compal -c 1003 to support the newer
  versions of C139/140 boot code which expect "1004" instead of the more
  familiar "1003" in the serially downloaded code image.  The new -c 1004
  option ("1004" signature bytes) should work with all currently known
  Mot/Compal boot code versions, thus -c 1003 is retained only for backward
  compatibility with existing usage.

* fc-fsio: one can now type ls -l and not just the ll shorthand,
  pirelli-get-imei and pirelli-magnetite-init commands added (former
  fc-getpirimei and fc-pirhackinit separate utilities absorbed into fc-fsio),
  help facility added.

* fc-tmsh changes:

  + fc-tmsh now takes the -p option and supports one-shot command invokation
    like fc-fsio and fc-shell;

  + added support for L1/RF test mode commands and other TM3 protocol commands
    (omr, omw, oabbr, oabbw, tm3ver);

  + added ETM_AUDIO commands aul, aus, aur and auw.

* fc-olddump renamed to fc-memdump, can now use either TM3 or ETM memory read
  commands.

* tiffs: added -O option to parse FFS images from the ancient 20020917 firmware
  on the D-Sample.

* New in vitro analysis utilities: fc-cal2text and tiaud-decomp.

* Added tools for working with ringtone melodies: fc-e1decode and fc-e1gen for
  loudspeaker melodies (see doc/Melody_E1) and fc-buzplay (plus target-side
  buzplayer.srec) for devices that use a piezoelectric buzzer.

* rvinterf: added support for the keepalive mechanism, see the description in
  doc/RVTMUX.

* rvinterf & fc-shell: added support for the experimental TCH rerouting feature
  of FreeCalypso Citrine firmware.

* fc-fr2tch, fc-gsm2vm, fc-tch2fr, fc-vm2hex: new utilities for converting
  GSM 06.10 FR codec bits between different formats.

* c139explore & pirexplore: added host shell script wrappers invoking
  fc-compalram and fc-iram with the respective target binaries.

* Internal cleanup and refactoring in various places.

Changes from fc-host-tools-r3 to fc-host-tools-r4:

* Shellcode-based Compal phone break-in utility tfc139 reworked in a new way
  that should work with all Mot C1xx firmwares beyond the original TFC139
  target: see doc/Compal-unlock and doc/TFC139-breakin for the details.

* Documentation and cosmetic code changes to reflect the new understanding of
  TI's TM (Test Mode) predating ETM (Enhanced Test Mode): see doc/RVTMUX for
  a detailed explanation.

* Added support for the old non-enhanced Test Mode memory read command in the
  form of fc-tmsh omr command, fc-fsio omemdump command and fc-olddump utility.

* rvtdump and rvinterf recognize the old all-ASCII GPF trace format emitted by
  the D-Sample firmware from 20020917 and print these traces as ASCII instead
  of hex.

* fc-dspapidump fixed to be independent of host byte order.