FCDEV3B hardware bug: sleep mode self-reboot

[Mychaela Falconia]

Hi Kent,

> I heard you and I usually have access a RIGOL DS1052E here in London.
> However, I can not help with the problem before next weekend and I would
> love to.

Waiting till next weekend is not a problem - I don't know whether or
not I will be able to get any o'scope help on my end before then.

It is too bad that we don't have traditional graphical schematics for
our board, to aid community volunteers like you in poking at the
hardware.  The closest thing we have are TI Leonardo schematics in
several different versions; here is one of those versions:

ftp://ftp.freecalypso.org/pub/GSM/Calypso/Leonardo_rev05.pdf

Our own board has always been meant to be a re-creation of TI's
historical Leonardo, hence TI's historical schematics apply to most of
our board.  However, we could not find a surviving copy of PCB layout
for any Leonardo variant, so we had to use Openmoko's GTA02 PCB layout
instead - but Om's modem is itself a very direct derivative of the
Leonardo.

The only part of our (and Openmoko's) modem that is different from
Leonardo at the schematic level is the RF front end, the part that
makes our modems triband.  TI had a fully quadband design and a
hobbled EMEA-only version (900 and 1800 MHz only), in both cases using
an integrated FEM with the antenna switch and Rx SAW filters in one
package, whereas the triband design with a separate antenna switch and
Rx SAW filters we are using is an FIC/Openmoko invention.  The RF PA
is also a little newer and smaller than what TI used, and the matching
networks between RF components (the black magic which I don't
understand) are different between TI's Leonardo and us/Openmoko.

You can find Openmoko's modem schematics here:

ftp://ftp.freecalypso.org/pub/GSM/GTA02/GTA02_Schematic_MB_A5_1220.pdf

If you compare Om's modem schematics against Leonardo, you will see
that it is exactly the same circuit for the most part, but the
reference designators are different.  On our own FCDEV3B the reference
designators we use are the ones from Leonardo, not Om's, except in the
400 block which is the triband RFFE and comes solely from Openmoko,
with no Leonardo counterpart.

You should also at least try to understand the non-graphical
"schematics" which we have for our own board:

ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/fcdev3b/fcdev3b-netlist-20160127.zip

All files inside the ZIP are plain ASCII text, written in vi and read
as input by my ueda ad hoc tools.  The file named MCL is the Master
Component List, every refdes used on the board is defined there along
with the full details about that component.  All components that have
been taken from Leonardo and Openmoko as well as the new ones of my
own addition are gathered there.  The *.v files are structural Verilog
sources, they define the netlist as built from the primitives listed
in the primitives file, these primitives are then turned into physical
component packages and their pins in the MCL binding step.

Finally, here are the PCB layout files:

ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/fcdev3b/ggg/FCDEV3B-20161223-2_altium_proj.zip

The above is the Altium project which I am not able to open myself as
I don't use Windows and don't have any Windows machines - the use of
Altium was imposed on us by the Iranians who did our initial layout on
a donated labor basis, and I subsequently had to pay an arm and a leg
to a hired Altium contractor to clean up the mess they left us with.

If you are like me in not running a recent version of Windows and thus
not being able to install Altium from The Pirate Bay to open those
files, here are the gerber files that can be viewed in any gerber
viewer:

ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/fcdev3b/fcdev3b-gerbers-20161223-official.zip

Finally, a PDF with the component placement drawing (where you can see
every component refdes on our board, including those that aren't on
the silk screen) lives inside this ZIP:

ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/fcdev3b/ggg/FCDEV3B-20161214_for_assembly.zip

Once you are able to sort out all of the above and can find each
component both on the physical board as well as on the Leonardo or
Openmoko schematics it came from, please let me know and I'll tell you
where I would like to get some o'scope probing done.

> Remark (I'm sure you already know it): going from 100nF to 1uF capacity
> with a unit from the same series of capabilities could make the problem
> bigger due to the, under normal conditions, increased Equivalent Series
> Resistance (ESR).

Thanks, I will keep this point in mind.  So far it has only been
experimentation to see if a cap change improves the sleep mode
behaviour; if there is no improvement (like there hasn't been any so
far), I won't change the caps for the next board assembly batch, i.e.,
keep Leonardo/Openmoko original caps.

> On the PCB my eyes catch the C306, among several hot spot, it is not a
> physical small PCB.

You got my attention here: what is it about C306 that drew your
attention?  You said "hot spot" - do you mean that you detected it as
being hot with an IR thermometer?

C306 is a bypass cap on the V-SIM net, i.e., the voltage to the SIM,
near the SIM socket itself, following TI's Leonardo schematics and
MCL.  (We found an OrCAD DSN file for one of the versions of Leonardo
schematics, and a helpful person on the net extracted the MCL info
from that DSN for me.)

> When you say the same design/layout as other products,
> it is very important that the decoupling capacitors actually is the same,
> not only in farad, but also in type.

Yes, as I mentioned above, the MCL information (full specs for each
component) came from TI's Leonardo_rev05.dsn, and when I later got the
design files for the GTA02, I saw that Openmoko had not made any
changes either in this area.

> If we do not know, we go for X7R.

The decoupling caps in my current MCL (i.e., what got populated on our
current batch of boards) are all X5Rs, and to the best of my
recollection, they all came from Leonardo MCL data like this.  But I
am going to double-check against Openmoko's BOM data, especially since
I just found this chilling article:

https://www.maximintegrated.com/en/app-notes/index.mvp/id/5527

> Never save on the decoupling capacitor (the 100nF over Vcc), it is a matter
> of production methods, to lower the ESR which is frequency dependent. In
> addition, modern small capabilities are very temperature sensitive (large
> variation in F), go for X7R.

It was never a matter of trying to save on the caps on our boards,
instead I am making my best attempt, given limited information, at
recreating what worked for our predecessors at TI and Openmoko.

> Minimum 1.0 uF ceramic capacitor is recommended to be connected across
> SIM_VCC and GND:
> https://www.onsemi.com/pub/Collateral/NCN4555-D.PDF

Please note that on our board we have not one but two bypass caps on
the V-SIM net: C218 near the Iota chip where this supply is produced
and C306 near the SIM socket.  C218 is 1 uF per Leonardo schematics;
Openmoko increased its physical size from 0402 to 0603 (a change we've
kept as we are using Om's PCB layout), but did not change the value.
Also note that Om's GTA02 board has no equivalent of our C306 from
Leonardo, i.e., they don't have another cap near the SIM socket and
their equivalent of C218 inside the modem core is the only cap they
have on V-SIM.

> For later versions, this is good:
> http://www.telit.com/fileadmin/user_upload/products/Downloads/3G/Telit_SIM_Integration_Design_Guide_Application_Note_r10.pdf

Unless we run into some issues with the SIM interface, I would rather
stick with following TI/Openmoko reference designs for *our* chipset
than switch to some other vendor's way of doing things.

Right now I don't think we have any problems at all around our SIM
interface, I feel that the self-reboot problem on AT+CFUN=1 is
probably not related to the SIM interface in any way, it's just that
the structure of Nucleus tasks and interrupt handlers in the firmware
is such that when small sleep is enabled, the execution of AT+CFUN=1
causes a lot of rapid back-to-back ARM7 CPU sleep-wake sequences, and
it is the latter that must be killing the board, not anything around
the SIM interface.

If you or someone else is able to do some o'scope probing on our board
as the AT+CFUN=1 self-reboot occurs, the first place I would like to
look at is the V-DBB net, see caps C214, C211 and C212 on Leonardo
schematics.  It should be a steady 1.5 V rail, and I would really like
to see if any voltage drops can be spotted at any of the 3 caps just
named.

M~