Preparing for new release of FC host tools

Mychaela Falconia mychaela.falconia at gmail.com
Sat Feb 29 23:15:41 UTC 2020


Hello FC community,

After some more investigation I have uncovered what looks like the root
cause of the problem which Serg L reported back in 2019-07, namely the
problem where the performance of our loadtools (particularly fc-xram)
suddenly became very poor on newer Linux host systems.  The root cause
appears to be this Linux kernel driver change:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c6dce2626606ef16434802989466636bc28c1419

This commit is included in all mainline Linux kernel versions since
4.11, i.e., for quite some time now.  Looking at the explanation in
the commit comments, it is difficult to fault Linux kernel developers
for making this change or to classify it as a kernel bug or defect -
but it does unfortunately break those applications which have
traditionally depended on fast turnaround times between commands sent
to serially-interfaced targets and responses from those targets, and
our loadtools just happen to be among such affected applications.

The commit comments do tell what people like us need to do: we need to
set the ASYNC_LOW_LATENCY flag for our turnaround-time-sensitive
applications, the same flag which just happened to be set by default
previously.

I agree with needing to set this ASYNC_LOW_LATENCY flag when working
with FTDI adapters (just FTDI) on the affected newer Linux host
systems, but given the new knowledge I do not agree with Serg's
approach of hiding this setting in an unconditional piece of code
snuck into FC host tools libserial, hence I will NOT be reapplying his
patch which I already reverted earlier.  Instead I argue that the
proper solution to this problem is to instruct users of affected
systems to issue this simple command after connecting their USB cable
but before running fc-loadtool or fc-xram:

setserial /dev/ttyUSBx low_latency

The above user instruction is now included in the Loadtools-performance
article in the current freecalypso-tools, soon to become the new
fc-host-tools-r12 release.

Why did Das Signal not notice this problem on his Debian system with a
4.19-based kernel, i.e., why was he getting the same performance with
the ASYNC_LOW_LATENCY libserial patch included or reverted?  The trick
is that this ASYNC_LOW_LATENCY flag persists through closing and
subsequent reopening of a given serial port (that is why one can make
the setting with a command line tool and have it persist), and I
suspect that DS had tried the version with the libserial hack included
first, then tried the version with the patch reverted, but didn't
realize that the ASYNC_LOW_LATENCY flag was still set.  To see the
poor performance without this flag set, one would need to either
remove and reinsert the USB-serial device or manually clear the flag
with setserial, then run a version of loadtools without the libserial
hack.

Going forward, I am going to make a few more small fixes and then put
out fc-host-tools-r12 as a checkpoint release.  I already have plans
for a bigger change of implementing a binary protocol between loadtools
and loadagent, a change which I expect will dramatically speed up all
of our lengthy operations, but the current freecalypso-tools tree
already has too many new developments which need to be put out in a
release, hence my plan to make fc-host-tools-r12 as a checkpoint
release and then almost immediately proceed to new developments with
the binary protocol.

Hasta la Victoria, Siempre,
Mychaela aka The Mother


More information about the Community mailing list