# HG changeset patch # User Mychaela Falconia # Date 1568359680 0 # Node ID f68ca40fa5c1d6abef1e7453c573544aa3f95efc # Parent 7ba5c951803c5fae40d27a1b4aed091c98284a51 Firmware-deblobbing document written diff -r 7ba5c951803c -r f68ca40fa5c1 Firmware-deblobbing --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Firmware-deblobbing Fri Sep 13 07:28:00 2019 +0000 @@ -0,0 +1,125 @@ +The state of blobs vs. blob-free firmware in FreeCalypso +======================================================== + +As of 2019, we have 3 different firmware versions for Standard Modem +functionality: + +* Magnetite hybrid is the current production firmware version. The only blobs + are Nucleus, OSL and OSX glue components of GPF, and TI's proprietary TMS470 + compiler. Everything other than Nucleus and OSL/OSX is compiled from source, + but the compiler is TI's proprietary TMS470. The same Magnetite source tree + also supports other configurations (maintained only for regression testing) + which have more blobs, as well as handset configurations which are currently + frozen for lack of suitable development hardware. + +* Selenite-470 is FC Selenite built with TI's TMS470 compiler: all code is + compiled from source, no blobs other than the compiler and its RTS library + (libc/libgcc equivalent). The blob version of Nucleus is replaced with a + different (slightly newer) version in full source form, while the blob + versions of OSL and OSX glue components have been replaced with reconstructed + sources, i.e., a reconstruction based on disassembly. This fw version is + currently considered experimental, not cleared for production, and the reason + is OSL/OSX: the reconstruction is of uncertain correctness and known to be + incomplete. + +* Selenite-gcc is FC Selenite built with gcc. This version has no blobs + whatsoever (there is no support in the gcc+binutils toolchain for TI's TMS470 + ABI, thus it is physically impossible to include any of TI's COFF blobs in + the link), and it is built with a FLOSS compiler. However, it is even more + experimental and not-for-production than Selenite-470: not only is the OSL/OSX + issue still there, but there is also widespread breakage from the use of a + different compiler which was never anticipated by the original developers. + The old FC Citrine firmware (unmaintained since 2016) also suffers from all + of the same problems, plus additional ones, and therefore should not be + considered at all. + +When it comes to the firmware, right now those OSL and OSX glue components of +GPF form the biggest bone in our collective throat. Just 10 C source files are +missing: + +os_com.c +os_drv.c +os_evt.c +os_isr.c +os_mem.c +os_mis.c +os_pro.c +os_sem.c +os_tim.c +osx.c + +If we can ever find these 10 missing files (does not even need to be exactly +the same version as in TCS211 GPF), then Selenite-470 would immediately become +the new production firmware replacing Magnetite (the Nucleus change isn't the +problem, it's OSL and OSX), and the way would be cleared to begin work on +bringing Selenite-gcc up to par. But in the absence of these 10 files, the +following two interlocking problems get in the way of FC Selenite: + +1) I (Mother Mychaela) am not willing to skip Selenite-470 and jump directly to + Selenite-gcc, as doing so would violate the fundamental principle of + incrementality: we need to be making one small change at a time, requiring + full stability after each incremental change before going to the next. + +2) I am not able to produce a reconstructed C source for certain parts of OSL + which would result in correct object code when compiled with TMS470: the + issue is potential race conditions in the OSL timer code. The existing COFF + object code avoids the race, I can produce C code for compilation with gcc + which also avoids the race, but I don't know the requisite magic for C code + to be compiled with TMS470. + +At this point you are going to ask - OK, so what do we do if we can't find +those 10 missing files? The Mother's current plan is as follows: if these 10 +files are still missing when we get our handset UI development board built, I +will create a third firmware source tree (not Magnetite, not Selenite, but to +be named after some other mineral) with the following key properties: + +* Just like Selenite, it will be hybrid only, no legacy blob-laden + configurations; + +* Both modem and handset configurations will be included; + +* The compiler will be TMS470 - sorry, no gcc; + +* The version of Nucleus will be the new source-enabled one, same as Selenite; + +* I will do some careful surgery on the OSL/OSX blobs to make them work with + the new version of Nucleus. + +The result of these listed key properties is that the new firmware tree will +have blobs for OSL and OSX and will use the TMS470 compiler as required by +these blobs, but absolutely everything else will be source-enabled. This +situation will then persist until someone can wave a few million dollars in +front of TI to convince their execs to direct their archivists to dig up the 10 +missing files, or until the world civilization collapses into a Mad Max world, +allowing us to seize those archives with a Spetsnaz unit without police +interference. + +Special modem applications +========================== + +The above plan states that the third firmware source tree will be created as +described if the original OSL and OSX source files are still missing when we +get our handset UI development board built. The reason for this coupling is +that when we get this UIDB built, the floodgates will open for intensive +handset UI development. The latter work will need to be done without the +clutter of Magnetite, yet Selenite is blocked by the lack of the 10 missing +files - hence the case for the third firmware source tree as described above. + +Alternatively, a third fw source tree similar to the one described (but perhaps +without the handset configuration) can be created if someone commissions +significant work on modem firmware, work that is significant enough to call for +a source tree that is as stable as Magnetite, but free of the clutter. + +cdginc header files +=================== + +Another area of deblobbing that hasn't been done yet, but can be done when and +if a serious need arises, is the cdginc header file set. The cdginc files which +are currently used for our hybrid config aren't blobs in the strict sense: they +are C header files included by the sources being recompiled, but they have been +auto-generated (from true human-editable sources which we do have) by a tool +(ccdgen) which currently exists only as a Windows binary sans source. + +If anyone needs to make changes to cdginc, the proper course of action should +be to hire a Windows reverser to reverse ccdgen.exe and to produce a perfect +form, fit and function replacement.