The independent security researcher Adam Gowdiak has published an extensive report on flaws he found in some eUICCs (the chips used to store eSIM profiles within the GSMA eSIM architecture). While the specific demonstrable exploit was in a product of one specific CardOS Vendor (Kigen, formerly part of ARM), the fundamental underlying issue is actually an architectural one.
The Oracle Javacard [memory] safety architecture relies on a so-called bytecode verifier which is a
program that you run after compiling an application, but before executing the code on the Javacard. The specifications allow for both on-card and off-card verification. However, the computational complexity of this verifier is generally assumed to exceed the resources available inside many microcontrollers used to implement java cards. Such microcontrollers often are ARM SC000 (Cortex-M0 based) or SC300 (Cortex-M3 based) based, with only tens of kilobytes of RAM and hundreds of kilobytes of flash.
Javacard was originally developed for use cases within the banking/payment industry. In that industry, the card-issuing bank is the sole entity that has the keys to load java applets onto a card. That entity is of course interested in the security of the card, and will hence always run an off-card bytecode verifier. In a world of physical SIM/USIM cards issued by a single mobile operator, the situation is the same: The card-issuing MNO/MVNO is the only entity with key materials to install additional java applets on the card.
However, when the industry represented in GSMA standardized the eSIM architecture, this changed. Suddenly we have various eSIM profiles of various different operators, each holding key material to install Java applets on the shared card. In such an environment, it is no longer safe to assume that every MNO/MVNO can be trusted to be non-adverserial and hence trusted to run that off-card bytecode verifier before loading applets onto the card.
If the Javacard runtime on the existing card/chip itself cannot autonomously perform those verification tasks, I don't see how the problem can ever be solved short of completely removing/disabling Javacard support in such eUICCs. Luckily it is an optional feature and not a mandatory requirement for an eUICC to be approved/accredited. Sadly many MNOs/MVNOS however will mandate Javacard support in their eSIM profiles and hence refuse to install into an eUICC without it :(
In my opinion, the solution to the problem can only be to either make the GSMA require full on-card bytecode
verification on all eUICCs, or to remove Javacard support from the eUICC.
We have to keep in mind that there are hundreds if not thousands of MVNOs around the planet, and all of them
are subject to whatever local jurisdiction they operate in, and also subject to whatever government pressure
(e.g from intelligence agencies).
In hindsight, anyone familiar with the 2019 work by Gowdiak and an understanding of the fundamental change to multiple stakeholders in an eUICC (compared to classic SIM/USIM) should have arrived at the conclusion that there is a serious problem that needs addressing. I think the 2019 work had not been publicized and covered significantly enough to make sure that everyone in the industry was made aware of the problems. And that in turn is mostly a result of Oracle + Gemalto downplaying the 2019 findings back in the day, rather than raising awareness within all relevant groups and bodies of the industry.
Mitigation via TS.48 key diversification
The specific attack presented was using a GSMA TS.48 test profile to install the malicious java bytecode; those TS.48 profiles are standardized profiles used by the industry for cellular testing; until the attack they contained well-known static OTA key material. The mitigation to randomize/diversity those keys in TS.48v7 closes that particular vector, but the attack itself is not dependent on test profiles. Any MNO/NVNO (or rather, anyone with access to a commercial service of a SM-DP+ accredited by GSMA) obviously has the ability to load java applets into the eSIM profile that they create, using keys that they themselves specify.
What IMHO ought to be done
Oracle should get off their we only provide a reference implementation and vendors should invent their own prporietary verification mechanisms horse. This is just covering their own ass and not helping any of their downstream users/implementers. The reference implementation should show how proper verification can be done in the most resource-constrained environment of cards (it's JavaCard, after all!), and any advances of the verifier should happen once at Oracle, and then used by all the implementers (CardOS vendors). Anyone who really cares about security of a standardized platform (like Javacard) should never leave key aspects of it up to each and every implementer, but rather should solve the problem once, publicly, with validation and testing tools, independent 3rd party penetration testing and then ensure that every implementer uses that proven implementation.
GSMA should have security requirements (and mandatory penetration tests) specifically regarding the JVM/JRE of each card that gets SAS-UP accredited.
GSMA should require that Javacard support should be disabled on all existing eUICCs that cannot legitimately claim/demonstrate that they are performing full bytecode verification entirely on-card.
GSMA should refuse any future SAS-UP accreditation to any product that requires off-card bytecode verification
The entire industry should find a way to think beyond Javacard, or in fact any technology whose security requires verification of the executable program that is too complex to perform on-card on the targeted microcontrollers.
I sincerely regret to see Linux kernel patches like this one removing Russian developers from the MAINTAINERS
file. To me, it is a sign or maybe even
a symbol of how far the Linux kernel developer community I remember from ~ 20 years ago has changed, and how
much it has alienated itself from what I remember back in the day.
In my opinion this commit is wrong at so many different levels:
it is intransparent. Initially it gave no explanation whatsoever (other than some compliance
hand-waving). There was some follow-up paraphrasing one paragraph of presumed legal advice that was given
presumably by Linux Foundation to Linus. That's not a thorough legal analysis at all. It doesn't even say
to whom it was given, and who (the individual developers? Linux Foundation? Distributors?) is presumed to be
subject to the unspecified regulations in which specific jurisdiction
it discriminates developers based on their presumed [Russian] nationality based on their name, e-mail
address domain name or employer.
A later post in the thread has clarified
that it's about an U.S. embargo list against certain Russian individuals / companies. It is news to me that
the MAINTAINERS file was usually containing Companies or that the Linux kernel development is Companies
engaging with each other. I was under the naive assumption that it's individual developers who work together,
and their employers do not really matter. Contributions are judged by their merit, and not by the author or
their employer / affiliation. In the super unlikely case that indeed those individual developers removed from
the MAINTAINERS file would be personally listed in the embargo list: Then yes, of course, I agree, they'd have
to be removed. But then the commit log should of course point to [the version] of that list and explicitly
mention that they were personally listed there.
And no, I am of course not a friend of the Russian government at all. They are committing war crimes,
no doubt about it. But since when has the collaboration of individual developers in an open source project
been something related to actions completely unrelated to those individuals? Should I as a German developer
be excluded due to the track record of Germany having started two world wars killing millions? Should
Americans be excluded due to a very extensive track record of violating international law? Should we exclude
Palestinians? Israelis? Syrians? Iranians? [In case it's not obvious: Those are rhetorical questions, my
position is of course no to all of them].
I just think there's nothing more wrong than discriminating against people just because of their passport,
their employer or their place of residence. Maybe it's my German upbringing/socialization, but we've had
multiple times in our history where the concept of **Sippenhaft** (kin liability) existed. In those dark ages of history you
could be prosecuted for crimes committed by other family members.
Now of course removal from the MAINTAINERS file or any other exclusion from the Linux kernel development
process is of course not in any way comparable to prosecution like imprisonment or execution. However, the
principle seems the same: An individual is punished for mere association with some others who happen to be
committing crimes.
Now if there really was a compelling legal argument for this (I doubt it, but let's assume for a second
there is): In that case I'd expect a broad discussion against it; a reluctance to comply with it; a search
for a way to circumvent said legal requirement; a petition or political movement against that requirement.
Even if there was absolutely no way around performing such a "removal of names": At the very least I'd expect
some civil disobedience by at least then introducing a statement into the file that one would have hoped to
still be listing those individuals as co-maintainers but one was forced by [regulation, court order, ...] to
remove them.
But the least I would expect is for senior Kernel developers to simply do apply the patch with a one-sentence
commit log message and thereby disrespect the work of said [presumed] Russian developers. All that does is to
alienate individuals of the developer community. Not just those who are subject to said treatment today, but
any others who see this sad example how Linux developers treat each other and feel discouraged from becoming
or remaining active in a community with such behaviour.
It literally hurts me personally to see this happening. It's like a kick in the gut. I used to be proud
about having had an involvement with the Linux kernel community in a previous life. This doesn't feel like
the community I remember being part of.
As some of you know, I've been spending a lot of time in recent years researching (and practically exploring +
re-implementing) historical telecommunications with my retronetworking project.
Retrocomputing itself is not my main focus. I usually feel there's more than enough people operating,
repairing, documenting at least many older computers, as well as keeping archives of related software and
continuing to spread knowledge on how they operated. Nevertheless, it is a very interesting topic - I just
decided that with my limited spare time I want to focus on retro-communications which is under-explored and
under-represented.
What's equally important than keeping the old technology alive, is keeping the knowledge around its creation
alive. How did it happen that certain technologies were created and became successful or not? How where they
key people behind it? etc.
Given my personal history with Taiwan during the last 18 years, it's actually surprising I haven't yet given
thought on how or where the history of the Taiwanese IT industry is documented or kept alive. So far I didn't
know of any computer museums that would focus especially on the Taiwanese developments. It didn't even occur
to me to even check if there are any.
During my work in Taiwan I've had the chance to briefly meet a few senior people at FIC (large mainboard maker
that made many PC mainboards I personally used) and both at VIA (chipset + CPU maker). But I didn't ever have
a chance to talk about the history.
In any case, I now found those transcripts of interviews. And what a trove of interesting first-hand
information they are! If you have an interest in computer history, and want to understand how it came about
that Taiwan became such a major player in either the PC industry or in the semiconductor design +
manufacturing, then I believe those transcripts are a "must read".
Now they've made me interested to learn more. I have little hope of many books being published on that
subject, particularly in a Language I can read (i.e. English, not mandarin Chinese). But I shall research
that subject. I'd also be interested to hear about any other information, like collections of historical
artifacts, archives, libraries, etc. So in the unlikely case anybody reading this has some pointers on
information about the history of the Taiwanese Chip and Computer history, please by all means do reach out and
share!.
Once I have sufficiently prepared myself in reading whatever I can find in terms of written materials, I might
be tempted to try to reach out and see if I can find some first-hand witnesses who'd want to share their
stories on a future trip to Taiwan...
Some of the readers of this blog know that I have a very special relationship with Taiwan. As a teenager, it
was the magical far-away country that built most of the PC components in all my PCs since my first 286-16 I
got in 1989. Around 2006-2008 I had the very unexpected opportunity to work in Taiwan for some time (mainly
for Openmoko, later some consulting for VIA). During that time I have always felt most welcome in and fascinated
by the small island nation who managed to turn themselves into a high-tech development and manufacturing site
for ever more complex electronics. And who managed to evolve from decades of military dictatorship and turn
into a true democracy - all the while being discriminated by pretty much all of the countries around the
world, as everybody wanted to benefit from cheap manufacturing in mainland China and hence expel democratic
Taiwan from the united nations in favour of communist mainland Chine.
I have the deepest admiration for Taiwan to manage all of their economic success and progress in terms of
democracy and freedom despite the political situation across the Taiwan strait, and despite everything that
comes along with it. May they continue to have the chance of continuing their path.
Setting economy, society and politics behind: On a more personal level I've enjoyed their culinary marvels
from excellent dumplings around every street corner to niu rou mien (beef noodle soup) to ma la huo guo
(spicy hot pot). Plus then the natural beauty, particularly of the rural mountainous regions once you leave
the densely populated areas around the coast line and the plains of the north west.
While working in Taiwan in 2006/2007 I decided to buy a motorbike. Using that bike I've first made humble
day trips and later (once I was no longer busy with stressful work at Openmoko) multiple week-long road trips
around the island, riding on virtually any passable road you can find. My typical routing algorithm is "take
the smallest possible road from A to B".
So even after concluding my work in Taiwan, I returned again and again for holidays, each one with more road
trips. For some time, Taiwan had literally become my second home. I had my favorite restaurants, shops, as
well as some places around the rural parts of the Island I cam back to several times. I even managed to take
up some mandarin classes, something I never had the time for while doing [more than] full time work. To my
big regret, it's still very humble beginner level; I guess had I not co-started a company (sysmocom) in Berlin
in 2011, I'd have spent more time for a more serious story.
In any case, I have nothing but the fondest memory of Taiwan. My frequent visits cam to a forcible halt with
the COVID-19 pandemic, Taiwan was in full isolation in 2020/21, and even irrespective of government
regulations, I've been very cautious about travel and contact. Plus of course, there's always the bad
conscience of frequent intercontinental air travel.
Originally I was planning to finally go on an extended Taiwan holiday in Summer 2024, but then the island was
hit by a relatively serious earthquake in April, affecting particularly many of the remote mountain regions
that are of main interest to me. There are some roads that I'd have wanted to ride ever since 2008, but which
had been closed every successive year when I went there, due to years of reconstructions after [mostly
landslides following] earthquakes and typhoons. So I decided to postpone it for another year to 2025.
However, in an unexpected change of faith, the opportunity arose to give the opening Keyonte at the 2024
Open Compliance Summit in Japan, and along with that the opportunity to do a stop-over in Taiwan. It will
just be a few days of Taipei this time (no motorbike trips), but I'm very much looking forward to being
back in the city I probably know second or third-best on the planet (after Berlin, my home for 23 years, as
well as Nuernberg, my place of birth). Let's see what is still the same and what has changed during the past
5 years!
I've presented a talk
Using bpftrace to analyze osmocom performance
as part of the
OsmoDevCon 2024 conference on Open Source Mobile Communications.
bpftrace is a utility that uses the Linux kernel tracing infrastructure (and eBPF) in order to provide tracing capabilities within the kernel, like uprobe, kprobe, tracepoints, etc.
bpftrace can help us to analyze the performance of [unmodified] Osmocom programs and quickly provide information like, for example:
Histogram of time spent in a specific system call
Histogram of any argument or return value of any system call
I've presented a talk
Introduction to XDP, eBPF and AF_XDP
as part of the
OsmoDevCon 2024 conference on Open Source Mobile Communications.
This talk provides a generic introduction to a set of modern Linux kernel technologies:
eBPF (extended Berkeley Packet Filter) is a kind of virtual machine that runs sandboxed programs inside the Linux kernel.
XDP (eXpress Data Path) is a framework for eBPF that enables high-performance programmable packet processing in the Linux kernel
AF_XDP is an address family that is optimized for high-performance packet processing. It allows in-kernel XDP eBPF programs to efficiently pass packets to userspace via memory-mapped ring buffers.
The talk provides a high-level overview. It should provide some basics before the other/later talks on bpftrace and eUPF.
I've co-presented a talk (together with Andreas EversbergHigh-performance I/O using io_uring via osmo_io
as part of the
OsmoDevCon 2024 conference on Open Source Mobile Communications.
Traditional socket I/O via read/write/recvfrom/sendto/recvmsg/sendmsg and friends creates a very high system call load. A highly-loaded osmo-bsc spends most of its time in syscall entry and syscall exit.
io_uring is a modern Linux kernel mechanism to avoid this syscall overhead. We have introduced the osmo_io`API to libosmocore as a generic back-end for non-blocking/asynchronous I/O and a back-end for our classic `osmo_fd / poll approach as well as a new backend for io_uring.
The talk will cover
a very basic io_uring introduction
a description of the osmo_io API
the difficulties porting from osmo_fd to osmo_io
status of porting various sub-systems over to osmo_io
I'm a strong proponent of self-hosting all your services, if not on your own hardware than at least on
dedicated rented hardware. For IT nerds of my generation, this has been the norm sicne the early 1990s: If
you wante to run your own webserver/mailserver/... back then, the only way was to self-host.
So over the last 30 years, I've always been running a fleet of machines, some my own hardware colocated,
and during the past ~18 years also some rented dedicated "root servers". They run a plethora of services for
either my personal stuff (like this blog, or my personal email server), or any of the IT services of the open
source projects I'm involved in (like osmocom) or the company I co-founded and run (sysmocom).
Every few years there's the need to migrate to new hardware. Either due to power consumption/efficiency, or
to increase performance, or to simply avoid aging hardware that may be dying soon.
When upgrading from one [hosted] server to another [hosted] server, there's always the question of how to
manage the migration with minimal interruption to services. For very simple services like http/https,
it can usually be done entirely within DNS: You reduce the TTL of the records, bring up the service
on the new server (with a new IP), make the change in the DNS and once the TTL of the DNS record is expired in
all caches, everybody will access the new server/IP.
However, there are services where the IP address must be retained. SMTP is a prime example of that. Given how
spam filtering works, you certainly will not want to give up your years if not decadeds of good reputation for
your IP address. As a result, you will want to keep the IP address while doing the migration.
If it's a physical machine in colocation or your home, you can of course do that all rather easily under your
control. You can synchronize the various steps from stopping the services on the old machine, rsync'ing over
the spool files to the new, then migrate the IP over to the new machine.
However, if it's a rented "root" server at a company like Hetzner or KVH, then you do not have full control
over when exactly the IP address will be migrated over to the new server.
Also, if there are many different services on that same physical machine, running on a variety of different
IPv4/IPv6 addresess and ports, it may be difficult to migrate all of them at once. It would be much more
practical, if individual services could be migrated step by step.
The poor man's approach would be to use port-forwarding / reverse-proxying. In this case, the client
establishes a TCP connection to the old IP address on the old server, and a small port-forward proxy accepts
that TCP connectin, creates a second TCP connection to the new server, and bridges those two together.
This approach only works for the most simplistic of services (like web servers), where
there are only inbound connections from remote clients (as outbound connections from the new server would
originate from the new IP, not the old one), and
where the source IP of the client doesn't matter. To the new server all connections' source IP addresses
suddenly are masked and there's only one source IP (the old server) for all connections.
For more sophisticated serviecs (like e-mail/SMTP, again), this is not an option. The SMTP client IP address
matters for whitelists/blacklists/relay rules, etc. And of course there are also plenty of outbound SMTP
connections which need to originate from the old IP, not the new IP.
So in bed last night [don't ask], I was brainstorming if the goal of fully transparent migration of individual
TCP+UDP/IP (IPv4+IPv6) services could be made between and old and new server. In theory it's rather simple,
but in practice the IP stack is not really designed for this, and we're breaking a lot of the assumptions
and principles of IP networking.
After some playing around earlier today, I was actually able to create a working setup!
It fulfills the followign goals / exhibits the following properties:
old and new server run concurrently for any amount of time
individual IP:port tuples can be migrated from old to new server, as services are migrated step by step
fully transparent to any remote peer: Old IP:port of server visible to client
fully transparent to the local service: Real client IP:port of client visible to server
no constraints on whether or not the old and new IPs are in the same subnet, link-layer, data centre, ...
use only stock features of the Linux kernel, no custom software, kernel patches, ...
no requirement for controlling a router in front of either old or new server
General Idea
The general idea is to receive and classify incoming packets on the old server, and then selectively tunnel
some of them via a GRE tunnel from the old machine to the new machine, where they are decapsulated and passed
to local processes on the new server. Any packets generated by the service on the new server (responses to
clients or outbound connections to remote serveers) will take the opposite route: They will be encapsulated
on the new server, passed through that GRE tunnel back to the old server, from where they will be sent off to
the internet.
That sounds simple in theory, but it poses a number of challenges:
packets destined for a local IP address of the old server need to be re-routed/forwarded, not delivered to
local sockets. This is easily done with fwmark, multiple routing tables and a rule, similar ro many other
policy routing setups.
I've presented about
Datex-L, the German CSPDN (Circuit Switched Public Data Network)
as part of the
RetroNetCall
talk series on retro-networking technology.
I've always been fascinated by "ancient" data communications - both
the kind that I personally consciously witnessed from the late 1980s as
well as the kind that I never experienced myself (like Datex-L).
I am not an expert in the subject by all means, as I was never involved
in its design, implementation or even used it. However, given that
there's very few public information online about Datex-L and/or other
CSPDNs, I thought I could improve the situation by presenting about it.
Many people with some kind of telecom background are familiar with D-channel (signalling) protocols of ISDN, and you can find many publications on that topic. Surprisingly, much less publications are talking about the B-channel protocols used for data transmission, like VX.75, V.110, V.120, T.70, ...
I've mentioned some of my various retronetworking projects in some
past blog posts. One of those projects is Osmocom Community TDM over
IP (OCTOI). During the
past 5 or so months, we have been using a number of GPS-synchronized
open source icE1usb
interconnected by a new, efficient but strill transparent TDMoIP protocol in order to run a distributed
TDM/PDH network. This network is currently only used to provide ISDN
services to retronetworking enthusiasts, but other uses like frame relay
have also been validated.
So far, the central hub of this OCTOI network has been operating in
the basement of my home, behind a consumer-grade DOCSIS cable modem
connection. Given that TDMoIP is relatively sensitive to packet loss,
this has been sub-optimal.
Luckily some of my old friends at noris.net have
agreed to host a new OCTOI hub free of charge in one of their
ultra-reliable co-location data centres. I'm already hosting some other
machines there for 20+ years, and noris.net is a good fit given that
they were - in their early days as an ISP - the driving force in the
early 90s behind one of the Linux kernel ISDN stracks called u-isdn. So after many decades, ISDN
returns to them in a very different way.
Side note: In case you're curious, a reconstructed partial release
history of the u-isdn code can be found on gitea.osmocom.org
But I digress. So today, there was the installation of this new OCTOI
hub setup. It has been prepared for several weeks in advance, and the
hub contains two circuit boards designed entirely only for this use
case. The most difficult challenge was the fact that this data centre
has no existing GPS RF distribution, and the roof is ~ 100m of CAT5
cable (no fiber!) away from the roof. So we faced the challenge of
passing the 1PPS (1 pulse per second) signal reliably through several
steps of lightning/over-voltage protection into the icE1usb whose
internal GPS-DO serves as a grandmaster clock for the TDM network.
The equipment deployed in this installation currently contains:
a rather beefy Supermicro 2U server with EPYC 7113P CPU and 4x PCIe, two of which are populated with Digium TE820 cards resulting in a total of 16 E1 ports
an icE1usb with RS422 interface board connected via 100m RS422 to an
Ericsson GPS03 receiver. There's two layers of of over-voltage
protection on the RS422 (each with gas discharge tubes and TVS) and
two stages of over-voltage protection in the coaxial cable between
antenna and GPS receiver.
Now that the physical deployment has been made, the next steps will be
to migrate all the TDMoIP links from the existing user base over to the
new hub. We hope the reliability and performance will be much better
than behind DOCSIS.
In any case, this new setup for sure has a lot of capacity to connect
many more more users to this network. At this point we can still only
offer E1 PRI interfaces. I expect that at some point during the coming
winter the project for remote TDMoIP BRI (S/T, S0-Bus) connectivity will
become available.
Acknowledgements
I'd like to thank anyone helping this effort, specifically
* Sylvain "tnt" Munaut for his work on the RS422 interface board (+ gateware/firmware)
* noris.net for sponsoring the co-location
* sysmocom for sponsoring the EPYC server hardware
The exhibit will be similar to the exhibit at the retrocomputing village
of the last CCC congress (36C3): A digital telephony network with ISDN
BRI and POTS lines providing services to a number of laptops with Modems
and ISDN terminal adapters.
We plan to demo the following things:
* analog modem and ISDN dial-up into BBSs
** text / ANSI interfaces via Telix, Telemate, Terminate
** RIPterm graphical interfaces
* analog modem and ISDN dial-up IP/internet
* ISDN video telephony
The client computers will be contemporary 486/Pentium machines wit DOS,
Windows 3.11 and OS/2.
Almost one year after my post regarding first steps towards a V5
implementation, some friends
and I were finally able to visit Wobcom, a
small German city carrier and pick up a lot of decommissioned
POTS/ISDN/PDH/SDH equipment, primarily V5 access networks.
This means that a number of retronetworking enthusiasts now have a
chance to play with Siemens Fastlink, Nokia EKSOS and DeTeWe ALIAN
access networks/multiplexers.
My primary interest is in Nokia EKSOS, which looks like an rather easy,
low-complexity target. As one of the first steps, I took PCB
photographs of the various modules/cards in the shelf, take note of the
main chip designations and started to search for the related
data sheets.
In short: Unsurprisingly, a lot of Infineon analog and digital ICs for
the POTS and ISDN ports, as well as a number of Motorola M68k based
QUICC32 microprocessors and several unknown ASICs.
So with V5 hardware at my disposal, I've slowly re-started my efforts to
implement the LE (local exchange) side of the V5 protocol stack, with
the goal of eventually being able to interface those V5 AN with the
Osmocom Community TDM over IP network. Once that is in place, we
should also be able to offer real ISDN Uk0 (BRI) and POTS lines at
retrocomputing events or hacker camps in the coming years.
If you have ever worked with Digium (now part of Sangoma) digital
telephony interface cards such as the TE110/410/420/820 (single to octal
E1/T1/J1 PRI cards), you will probably have seen that they always have a
timing connector, where the timing information can be passed from one
card to another.
In PDH/ISDN (or even SDH) networks, it is very important to have a
synchronized clock across the network. If the clocks are drifting,
there will be underruns or overruns, with associated phase jumps that
are particularly dangerous when analog modem calls are transported.
In traditional ISDN use cases, the clock is always provided by the
network operator, and any customer/user side equipment is expected to
synchronize to that clock.
So this Digium timing cable is needed in applications where you have
more PRI lines than possible with one card, but only a subset of your
lines (spans) are connected to the public operator. The timing cable
should make sure that the clock received on one port from the public
operator should be used as transmit bit-clock on all of the other ports,
no matter on which card.
Unfortunately this decades-old Digium timing cable approach seems to
suffer from some problems.
bursty bit clock changes until link is up
The first problem is that downstream port transmit bit clock was jumping
around in bursts every two or so seconds. You can see an oscillogram of
the E1 master signal (yellow) received by one TE820 card and the
transmit of the slave ports on the other card at
https://people.osmocom.org/laforge/photos/te820_timingcable_problem.mp4
As you can see, for some seconds the two clocks seem to be in perfect
lock/sync, but in between there are periods of immense clock drift.
As I found out much later, this problem only occurs until any of the
downstream/slave ports is fully OK/GREEN.
This is surprising, as any other E1 equipment I've seen always transmits
at a constant bit clock irrespective whether there's any signal in the
opposite direction, and irrespective of whether any other ports are
up/aligned or not.
But ok, once you adjust your expectations to this Digium peculiarity,
you can actually proceed.
clock drift between master and slave cards
Once any of the spans of a slave card on the timing bus are fully
aligned, the transmit bit clocks of all of its ports appear to be in
sync/lock - yay - but unfortunately only at the very first glance.
When looking at it for more than a few seconds, one can see a slow,
continuous drift of the slave bit clocks compared to the master :(
Some initial measurements show that the clock of the slave card of the
timing cable is drifting at about 12.5 ppb (parts per billion) when
compared against the master clock reference.
This is rather disappointing, given that the whole point of a timing cable
is to ensure you have one reference clock with all signals locked to
it.
The work-around
If you are willing to sacrifice one port (span) of each card, you can
work around that slow-clock-drift issue by connecting an external
loopback cable. So the master card is configured to use the clock
provided by the upstream provider. Its other ports (spans) will transmit
at the exact recovered clock rate with no drift. You can use any of
those ports to provide the clock reference to a port on the slave
card using an external loopback cable.
In this setup, your slave card[s] will have perfect bit clock
sync/lock.
Its just rather sad that you need to sacrifice ports just for
achieving proper clock sync - something that the timing connectors and
cables claim to do, but in reality don't achieve, at least not in my
setup with the most modern and high-end octal-port PCIe cards (TE820).
I've presented on
Advanced SIM card topics: GlobalPlatform SCP02, OTA, ARA-M, ISIM
as part of the
OsmoDevCall
talk series on Osmocom and related technology.
As some of you may know, I've been starting to collect "vintage"
telecommunications equipment starting from analog modems to ISDN
adapters, but also PBXs and even SDH equipment. The goal is to keep
this equipment (and related software) alive for demonstration and
practical exploration.
Working with PBXs to simulate the PSTN (ISDN/POTS) network is fine to
some extent, but it's of course not the real deal. You only get
S0-buses and no actual Uk0 like actual ISDN lines of the late 80ies and
90ies. You have problems with modems not liking the PBX dialtone, etc.
Hence, I've always wanted to get my hand on some more real-world
central-office telephone network equipment, and I finally have a source
for so-called V5.1/V5.2
access multiplexers. Those are like remote extension boxes for the
central office switch (like EWSD
or System 12). They
aggregate/multiplex a number of analog or ISDN BRI subscriber lines into
E1 lines, while not implementing any of the actual call control or ISDN
signalling logic. All of that is provided by the actual telephone
switch/exchange.
So in order to integrate such access multiplexers in my retronetworking
setup, I will have to implement the LE (local exchange) side of the V5.1
and/or V5.2 protocols, as specified in ITU-T G.964 and
G.965.
In the limited spare time I have next to my dayjob and various FOSS
projects, progress will likely be slow. Nonetheless I started with an
implementation now, and I already had a lot of fun learning about more
details of those interfaces and their related protocols.
One of the unresolved questions is to what kind of software I would want
to integrate once the V5.x part is resolved.
lcr would probably be the most
ISDN-native approach, but it is mostly unused and quite EOL.
Asterisk or FreeSWITCH would of course be obvious candidates, but they
are all relatively alien to ISDN, and hence not very transparent once
you start to do anything but voice calls (e.g. dialup ISDN data calls
in various forms).
yate is another potential candidate. It
already supports classic SS7 including ISUP, so it would be a good
candidate to build an actual ISDN exchange with V5.2 access
multiplexers on the customer-facing side (Q.921+Q.931 on it) and
SS7/ISUP towards other exchanges.
For now I think yate would be the most promising approach. Time will
tell.
The final goal would then be to have a setup [e.g. at a future CCC
congress] where we would have SDH add/drop multiplexers in several
halls, and V5.x access multiplexers attached to that, connecting analog
and ISDN BRI lines from individual participants to a software-defined
central exchange. Ideally actually multiple exchanges, so we can show
the signaling on the V5.x side, the Q.921/Q.931 side and the SS7/ISUP
between the exchanges.
Given that the next CCC congress is not before December 2022, there is a
chance to actually implement this before then ;)
[excuse this German-language post, this is targeted at the current German public discourse]
Ein paar Ergänzungen zu meinem blog-post gestern.
Ich benutzt den generischen Begriff PWS statt SMSCB, weil SMSCB strikt genommen nur im 2G-System
existiert, und nur ein Layer ist, der dort für Notfallalarmierung verwendet wird.
Zu Notfallwarn-Apps
Natürlich sind spezielle, nationale Deutsche Katastrophenschutz-Apps auch nützlich! Aber diese sollten
allenfalls zusätzlich angeboten werden, nachdem man erstmal die grundlegende Alarmierung konform der
relevanten internationalen (und auch EU-)Standards via Cell Broadcast / PWS realisiert. Man sagt ja auch
nicht: Nachrichtensendungen braucht man im Radio nicht mehr, weil man die bereits im Fernsehen hat. Man will
auf allen verfügbaren Kanälen senden, und zunächst jene mit möglichst universeller Reichweite und klaren
technischen Vorteilen benutzen, bevor man dann zusätzlich auch auf anderen Kanälen alarmiert.
Wie sieht PWS für mich als Anwender aus
Hier scheint es größere Missverständnisse zu geben, wie das auf dem Telefon letztlich aussieht. Ist ja auch
verständlich, hierzulande sieht man das nie, ausser man ist zufällig in einem Labor/Basttel-Netz z.B. auf
einer CCC-Veranstaltung unterwegs, in der das Osmocom-Projekt mal solche Nachrichten versendet hat.
Die PWS (ETWS, CMAS, WEA, KPAS, EU-ALERT, ...) nachrichten werden vom Telefon empfangen, und dann je nach
Konfiguration und Priorität behandelt. Für die USA ist im WEA vorgeschrieben, dass Alarme einer bestimmten
Prioritatsklasse (z.B. der Presidential Level Alert) immer zwangsweise zur Anzeige gebracht werden und
immer mit einem lauten sirenenartigen Alarmton einhergehen. Es ist sogar explizit verboten, dass der
Anwender diese Alarme irgendwo ausstellen, stumm schalten o.ä. kann. Insofern spielt es keine Rolle,
ob das Telefon gerade Lautlos gestellt ist, oder es nicht gerade unmittelbar bei mir ist.
Bei manchen Geräten werden die Warnungen sogar mittels einer text2speech-Engine laut über den Lautsprecher
vorgelesen, nachdem der Alarmton erscheint. Ob das eine regulatorische Anforderung eines der nationalen
System ist, weiss ich nicht - ich habe es jedenfalls bereits in manchen Fällen gesehen, als ich mittels
Osmocom-Software solche Alarme in privaten Labornetzen versandt habe.
Noch ein paar technische Details
PWS-Nachrichten werden auch dann noch ausgestrahlt, wenn die Zelle ihre Netzanbindung verloren hat. Wenn
also z.B. das Glasfaserkabel zum Kernnetz bereits weg ist, aber noch Strom da ist, werden bereits vorher vom
CBC (Cell Broadcast Centre) an die Mobilfunkzelle übermittelte Warnungen entsprechend ihrer
Gültigkeitsdauer weiter autonom von der Zelle ausgesendet Das ist wieder ein inhärenter technischer
Vorteil, der niemals mit einer App erreichbar ist, weil diese erfordert dass das komplette Mobilfunknetz mit
allen internen Verbindungen und dem Kernnetz sowie die Internetverbindung vom Netzbetreiber zum Server des
App-Anbieters durchgehend funktioniert.
PWS-Nachrichten können zumindest technisch auch von Telefonen empfangen werden, die garnicht im Netz
eingebucht sind, oder die keine SIM eingelegt haben. Ob dies in den Standards gefordert wird, und/oder
ob dies die jeweilige Telefonsoftware das so umsetzt, weiss ich nicht und müsste man prüfen. Technisch
liegt es nahe, ähnlich wie das Absetzen von Notrufen, das ja auch technisch in diesen Fällen möglich ist.
Zu den Kosten
Wenn - wie in der idealen Welt - das Vorhalten von Notfallalarmierung eine Vorgabe bereits zum Zeitpunkt der
Lizenzvergabe für Funkfrequenzen gewesen wäre, wäre das alles einfach ganz lautlos von Anfang an immer
unterstützt gewesen. Keiner hätte extra Geld investieren müssen, weil diese minimale technische Vorgabe dann
ja bereits Teil der Ausschreibungen der Betreiber für den Einkauf ihres Equipments gewesen wäre. Zudem hatten
wir ja bereits in der Vergangenheit Cell Brodacast in allen drei Deutschen Netzen, d.h. die Technik war mal
[aus ganz andern Gründen] vorhanden aber wurde irgendwann weggespart.
Das jetzt nachträglich einzuführen heisst natürlich, dass es niemand eingeplant hat, und dass jeder beteiligte
am Markt sich das vergolden lassen will. Die Hersteller freuen sich in etwa wie "Oh, Ihr wollt jetzt mehr als
ihr damals beim Einkauf spezifiziert habt? Schön, dann schreiben wir mal ein Angebot".
Technisch ist das alles ein Klacks. Die komplette Entwicklung aller Bestandteile für PWS in 2G/3G/4G/5G würde
ich auf einen niedrigen einmaligen sechsstelligen Betrag schätzen. Und das ist die einmalige Investition in
der Entwicklung, welche dann über alle Geräte/Länder/Netze umgebrochen wird. Bei den Milliarden, die in
Entwicklung und Anschaffung von Mobilfunktechnik investiert wird, ist das ein Witz.
Die Geräte wie Basisstationen aller relevanten Hersteller unterstützen natürlich von Haus aus PWS. Die bauen
für Deutschland ja nicht andere Geräte, als jene, die in UK, NL, RO, US, ... verbaut werden. Der Markt ist
international, die gleiche Technik steht überall.
Weil man jetzt zu spät ist, wird das natürlich von allen Seiten ausgenutzt. Jeder Basisstationshersteller
wird die Hand aufhalten und sagen, das kostet jetzt pro Zelle X EUR im Jahr zusätzliche Lizenzgebühren. Und
die Anbieter der zentralen Komponente CBC werden auch branchenüblich die Hand aufhalten, mit satten jährlichen
Lizenzgebühren. Und die Consultants werden auch alle die Hand aufhalten, weil es gibt wieder etwas zu
Integrieren, zu testen, ... Das CBC ist keine komplexe Technik. Wenn man das einmalig als Open Source
entwickeln lässt, und in allen Netzen einsetzt, bekommt man es quasi zum Nulltarif. Aber das würde ja
Voraussetzen, dass man sich wirklich mit der Technik befasst, versteht um welch simple Software es hier geht,
und dass man mal andere Wege in der Beschaffung geht, als nur mal eben bei seinen existierenden 3
Lieferanten anzurufen, die sich dann eine goldene Nase verdienen wollen.
In der öffentlichen Diskussion wird von 20-40 Millionen EUR gesprochen. Das sind überzogene Forderungen der
Marktteilnehmer, nichts sonst. Aber selbst wenn man der Meinung ist, dass man lieber das Geld zum Fenster
hinauswerfen will, statt Open Source Alternativen zu [ver]suchen, dann ist auch diese Größenordnung etwas,
dass im Vergleich zu den sonstigen Anschaffungs- und Betriebskosten eines Mobilfunknetzes verschwindend gering
ist. Ganz zu schweigen von den Folgekosten im Bereich Bergung/Rettung, Personenschäden, etc. die sich dadurch
mittelfristig bei Katastrophen einsparen lassen.
Oder anders betrachtet: Wenn sogar das wirtschaftlich viel schwächere Rumänien sich sowas leisten kann, dann
wird es wohl auch die Bundesrepublik Deutschland stemmen können.
[excuse this German-language post, this is targeted at the current German public discourse]
In mehrerern Gegenden Deutschlands gab es verheerende Hochwasser, und die Öffentlichkeit diskutiert deshalb
mal wieder die gute alte Frage nach dem adäquaten Mittel der Alarmierung der Bevölkerung.
Es ist einfach nur ein gigantisches Trauerspiel, wie sehr die Deutsche Politik und Verwaltung in diesem Punkt
inzwischen seit Jahrzehnten sämtliche relevanten Standards verpennt, und dann immer wieder öffentlich durch
fachlich falsche und völlig uninformierte Aussagen auffällt.
Das Thema wurde vor dem aktuellen Hochwasser bereits letztes Jahr im
Rahmen des sog. WarnTag öffentlich diskutiert. Auch hier von Seiten der
öffentlichen Hand ausschliesslich mit falschen Aussagen, wie z.B. dass
es bei Cell Broadcast Datenschutzprobleme gibt. Dabei ist Cell
Broadcast die einzige Technologie, wo keine Rückmeldung des einzelnen
Netzteilnehmers erfolgt, und das Netz nichtmal weiss, wer die Nachricht
empfangen hat, und wo dieser Empfang stattgefunden hat. Ganz wie beim
UKW-Radio.
Fakt ist, dass alle digitalen Mobilfunkstandards seit GSM/2G, d.h. seit
1991 die Möglichkeit mitbringen, effizient, schnell und datensparsam
alle Nutzer (einer bestimmten geographischen Region) mit sogenannten
broadcast Nachrichten zu informieren. Diese Technik, in GSM/2G
genannt Cell Broacast (oder auch _SMSCB_), unterscheidet sich
Grundlegend von allen anderen Kommunikationsformen im Mobilfunknetz, wie
Anrufe und herkömmliche SMS (offiziell SMS-PP). Anrufe, SMS und auch mobile Paketdaten (Internet) werden
immer für jeden Teilnehmer individuell auf ihm zugewiesenen Funkressourcen übermittelt. Diese Ressourcen
sind beschränkt. Es können in keinem Mobilfunknetz der Welt alle Teilnehmer gleichzeitig telefonieren, oder
gleichzeitig SMS empfangen.
Stattdessen benutzt Cell Broadcast - wie der Name bereits
unmissverständlich klar macht - Einen broadcast, d.h.
Rundsendemechanismus. Eine Nachricht wird einmal gesendet, benötigt
also nur eine geteilte Ressource auf der Luftschnittstelle, und wird
dann von allen Geräten im Empfangsbereich zeitgleich empfangen und
dekodiert. Das ist wie UKW-Radio oder klassisches terrestrisches
Fernsehen.
Cell Broadcast wurde bereits in den 1990er Jahren von Deutschen
Netzbetreibern benutzt. Und zwar nicht für etwas lebensnotwendiges wie
die Notfallsignalisierung, sondern für so banale Dinge wie die Liste
jener Vorwahlen, zu denen gerade ein vergünstigter "wandernder
Ortstarif" Besteht. Ja, sowas gab es mal bei Vodafone. Oder bei O2
wurden über lange Zeit (aus unbekannten Gründen) die GPS-Koordinaten der
jeweiligen Basisstation als Cell Broadcast versendet.
In der folgenden (nun fast abgeschalteten) Mobilfunkgeneration 3G
wurde Cell Broadcast leicht umbenannt als Service Area Broadcast
beibehalten. Schliesslich gibt es ja Länder mit - anders als in
Deutschland - funktionierender und kompetenter Regulierung des
Telekommunikationsmarktes, und die langjährig bestehenden gesetzlichen
Anforderungen solcher Länder zwingen die Netzbetreiber und auch die
Ausrüster der Neztbetreiber, neue Mobilfunkstandards so zu entwickeln,
dass die gesetzlichen Vorgaben bzgl. der Alarmierung der Bevölkerung im
Notfall funktioniert.
Im Rahmen dieser Standardisierung haben eine Reihe von Ländern innerhalb
der 3GPP-Standardisierung (zuständig für 2G, 3G, 4G, 5G) sogenannte
Public Warning Systems (PWS) standardisiert. Zu diesen gehören z.B.
das Japanische ETWAS (Earthquake and Tsunami Warning System), das
Koreanische KPAS (Korean Public Alerting System), das US-Amerikanische
WEA (Wireless Emergency Alerts, früher bekannt als CMAS) und auch das EU-ALERT mit den nationalen
Implementationen NL-ALERT (Niederlande) und UK-ALERT (Großbritannien)
sowie RO-ALERT (Rumänien).
Die zahlreichen Studien und Untersuchungen, die zur Gestaltung obiger
Systeme und der internationalen Standards im Mobilfunk geführt haben,
weisen auch nochmal nach, was sowieso vorher jedem Techniker
offensichtlich erscheint: Eine schelle Alarmierung aller Teilnehmer
(einer Region) kann nur über einen Broadcast-Mechanismus erfolgen. In
Japan war die Zielvorgabe, die Alarmierung in Erdbebenfällen innerhalb
von weniger als 4 Sekunden an die gesamte betroffene Bevölkerung zu
übertragen. Und das ist mit PWS möglich!
Die relevanten PWS-Standards in 2G/3G/4G/5G bieten jede Menge nützliche Funktionen:
Benachrichtigung in bestimmten geographischen Regionen
Interoperable Schnittstellen, so dass Netzwerkelemente unterschiedlicher Hersteller miteinander
kommunizieren
Konfigurierbare Benachrichtigungstexte, nicht nur in der primären Landessprache, sondern auch in mehreren
anderen Sprachen, die dann automatisch je nach Spracheinstellung des Telefons wiedergegeben werden
Unterschiedliche Schweregrade von Alarmierungen
Übermittlung nicht nur im Broadcast, sondern auch im Unicast an jeden Teilnehmer, der gerade in einem
Telefongespräch ist, und dessen Telefon gerade währenddessen aus technischen Gründen den Broadcast nicht
empfangen würde
Unterschied zwischen Wiederholung einer Übertragung ohne Änderung des Inhalts und einer übertragung mit
geändertem Inhalt
Es gibt also seit vielen Jahren internationale Standards, wie sämtliche heute eingesetzten Mobilfunktechniken
zur schnellen, effizienten und datensparsamen Alarmierung der Bevölkerung eingesetzt werden können.
Es gibt zahlreiche Länder, die diese Systeme seit langem einsetzen. Das US-Amerikanische WEA wurde nach
eigenen Angaben seit 2012 bereits mehr als 61.000 Mal benutzt, um Menschen vor Unwetter oder anderen
Katastrophen zu warnen.
Sogar innerhalb der EU hat man das EU-ALERT System spezifiziert, welches weitgehend mit dem amerikanischen WEA
identisch ist, und auf die gleichen Techniken aufbaut.
Und dann gibt es Länder wie Deutschland, die es seit genauso vielen Jahren vermissen lassen, durch Gesetze
oder Vorschriften
die Netzbetreiber zum Betrieb dieser Broadcast-Technologien in ihrem Netz verpflichtet
die Netzbetreiber zur Bereitstellung von standardisierten Schnittstellen gegenüber den Behörden wie Zivilschutz / Katastrophenschutz zu verpflichten, so das diese selbständig über alle Netzbetreiber Warnungen versenden können
die Gerätehersteller z.B. über Vorschriften des FTEG (Gesetz über Funkanlagen und Telekommunikationsendeinrichtungen) zu Verpflichten, die PWS-Nachrichten anzuzeigen
In den USA, dem vermeintlich viel mehr dem Freien Markt und dem
Kapitalismus anhängenden System ist all dies der Regulierungsbehörde FCC
möglich. In Deutschland mit seiner sozialen Marktwirtschaft ist es
anscheinend unmöglich, den Markt entsprechend zu regulieren. Eine
solche Regulierung schafft man in Deutschland nur für wirklich wichtige
Themen wie zur Durchsetzung der Bereitstellung von Schnittstellen für
die Telekommunikationsüberwachung. Bei so irrelevanten Themen wie dem
Katastrophenschutz und der Alarmierung der Bevölkerung braucht man den
Markt nicht zu regulieren. Wenn die Netzbetreiber kein PWS anbieten
wollen, dann ist das einfach so Gottgegeben, und man kann da ja nichts
machen.
Falls jemand sich SMSCB und PWS technisch näher ansehen will: In 2019
haben wir im Osmocom-Projekt eine Open Source Implementation des
kompletten Systems von BTS über BSC bis zum CBC, sowie der dazwischen
befindlichen Protokolle wie CBSP vorgenommen. Dies wurde
freundlicherweise durch den Prototype Fund mit EUR 35k finanziert. Ja,
so günstig kann man die nötige Technik zumindest für eine einzelne
Mobilfunkgeneration entwickeln...
Man kann also in einem selbst betriebenen Labor-Mobilfunknetz,
welches auf Open Source Software basiert mehr in Punkt standardkonformer
Notfallalarmierung, als die Deutsche Politik, Verwaltung und
Netzbetreiber zusammen hinbekommen.
Wir haben in Deutschland Leute, die diese Standards in und auswendig
kennen, sogar daran mitgearbeitet haben. Wir haben Entwickler, die
diese Standards implementiert haben. Aber wir schaffen es nicht, das
auch mal selbst praktisch zu benutzen - das überlassen wir lieber den
anderen Ländern. Wir lassen lieber zuerst die ganze
Katastrophenalarmierung mittels Sirenen vergammeln, machen den
Netzbetreibern keine Vorgaben, entwicklen komische Apps, die Anwender
extra installieren müssen, die prinzipbedingt nicht skalieren und beim
Test (WarnTag) nicht funktionieren.
Was für eine Glanzleistung für den hochentwickelten Techhologie-Standort Deutschland.
After many years of being involved in various projects at the annual
Chaos Communication Congress (starting from the audio/vidoe recording
team at 15C3), I've finally also departed the GSM team, i.e. the people
who operate (Osmocom based) cellular networks at CCC events.
The CCC Camp in August 2019 was
slightly different: Instead of helping an Osmocom based 2G/3G network, I
decided to put up a nextepc-based LTE network and make that use the
2G/3G HLR (osmo-hlr) via a newly-written DIAMETER-to-GSUP proxy. After lots of hacking
on that proxy and fixing various bugs in nextepc (see my
laforge/cccamp2019 branch here)
this was working rather fine.
For 36C3 in December 2019 I had
something different in mind: It was supposed to be the first actual
demo of the retronetworking / bbs-revival setup I've been working on
during past months. This setup in turn is sort-of a continuation of my
talk at 34C3 two years ago: BBSs and early Intenet access in the 1990ies.
Rather than just talking about it, I wanted to be able to show people
the real thing: Actual client PCs running (mainly) DOS, dialling over
analog modems and phone lines as well as ISDN-TAs and ISDN lines into
BBSs, together with early Interent access using SLIP and PPP over the
same dial-up lines.
What took most of the time was - interestingly - mainly two topics:
A 1U rack-mount system with four E1 ports. I had lots of old Sangoma
Quad-E1 cards in PCI form-factor available, but wanted to use a PC
with a more modern/faster CPU than those old first-generation Atom
boxes that still had actual PCI slots. Those new mainboards don't
have PCI but PCIe. There are plenty of PCIe to PCI bridges and
associated products on the market, which worked fine with virtually
any PCI card I could find, but not with the Sangoma AFT PCI cards I
wanted to use. Seconds to minutes after boot, the PCI-PCIe bridges
would always forget their secondary bus number. I suspected
excessive power consumption or glitches, but couldn't find anything
wrong when looking at the power rails with a scope. Adding
additional capacitors on every rail also didn't change it. The
!RESET line is also clean. It remains a mystery. I then finally
decided to but a new (expensive) DAHDI 4-port E1 PCIe card to move
ahead. What a waste of money if you have tons of other E1 cards
around.
Various trouble with FreeSWITCH. All I wanted/needed was some simple
emulation of a PSTN/ISDN switch, operating in NT mode towards both
the Livingston Portmaster 3 RAS and the Auerswald PBX. I would have
used lcr, but it supports neither
DAHDI nor Sangoma, but only mISDN - and there are no mISDN cards with
four E1 ports :( So I decided to go for FreeSWITCH, knowing it has
had a long history of ISDN/PRI/E1 support. However, it was a big
disappointment. First, there were some segfaults due to a classic pointer deref before NULL-check.
Next, libpri and FreeSWITCH have a different idea how channel (timeslot) numbers are structured,
rendering any call attempt to fail. Finally, FreeSWITCH decided to
blindly overwrite any bearer capabilities IE with 'speech',
even if an ISDN dialup call (unrestricted digital information) was
being handled. The FreeSWITCH documentation contains tons of
references on channel input/output variables related to that - but it
turns out their libpri integration doesn't set any of those,
nor use any of them on the outbound side.
Anyway, after a lot more time than expected the setup was operational,
and we could establish modem calls as well as ISDN dialup calls between
the clients and the Portmaster3. The PM3 in turn then was configured to
forward the dialup sessions via telnet to a variety of BBSs around the
internet. Some exist still (or again) on the public internet.
Some others were explicitly (re)created by 36C3 participants for this
very BBS-Revival setup.
My personal favorite was finding ACiD Underworld 2.0, one
of the few BBSs out there today who support RIPscrip, a protocol used to
render vector graphics, text and even mouse-clickable UI via modem
connection to a DOS/EGA client program called RIPterm. So we had one
RIPterm installation on Novell DOS7 that was just used for dialling into
ACiD Underworld 2.0.
Among other things we also tested interoperability between the 1980ies
CCC DIY accoustic coupler "Datenklo" and the Portmaster, and confirmed
that Windows 2000 could establish multilink-PPP not only over two
B-channels (128 kbps) but also over 3 B-Channels (192).
Running this setup for four days meant 36C3 was a quite different
experience than many previous CCC congresses:
I was less stressed as I wasn't involved in operating a service that
many people would want to use (GSM).
I got engaged with many more people with whom I would normally not
have entered a conversation, as they were watching the exhibits/demos
and we got to chat about the technology involved and the 'good old
days'.
Sylvain "tnt" Munaut for spending a lot of time with me at the
retronetworking assembly. The fact that I had an E1 interface around
was a good way for him to continue development on his ICE40 based
bi-directional E1 wiretap. He also helped with setup and teardown.
miaoski and evanslify for reviving two of their old BBSs from Taiwan
so we could use them at this event
The retronetworking setup is intended to operate at many other future
events, whether CCC related, Vintage Computing or otherwise. It's
relatively small and portable.
I'm very much looking forward to the next incarnations. Until then, I
will hopefully have more software configured and operational, including
a variety of local BBSs (running in VMs/containers), together with the
respective networking (FTN, ZConnect, ...) and point software like
CrossPoint.
If you are interested in helping out with this project: I'm very much
looking for help. It doesn't matter if you're old and have had BBS
experience back in the day, or if you're a younger person who wants to
learn about communications history. Any help is appreciated. Please
reach out to the bbs-revival@lists.osmocom.org mailing list, or directly
to me via e-mail.
The SIM card talk was surprisingly successful, both in terms of a full
audience on-site, as well as in terms of the number of viewers of the
recordings on media.ccc.de. SIM cards are a rather niche topic in the
wider IT industry, and my talk was not covering any vulnerabilities or
the like. Also, there was nothing novel in the talk: SIM cards have
been around for decades, and not much has changed (except maybe eSIM and
TLS) in recent years.
In any case, I'm of course happy that it was well received. So far I've
received lots of positive feedback.
As I'm working [more than] full time in cellular technology for almost
15 years now, it's sometimes hard to imagine what kind of topics people
might be interested in. If you have some kind of suggestion on what
kind of subject within my area of expertise you'd like me to talk about,
please don't hesitate to reach out.
The Mitel DECT talk also went quite well. I covered about 10 minutes of
technical details regarding the reverse engineering of the firmware and
the communication protocols of the device. Thanks again to Dieter
Spaar for helping with that. He is and remains
the best reverse engineer I have met, and it's always a privilege to
collaborate on any project. It was of course also nice to see what
kind of useful (and/or fun) things the eventphone team have built on
top of the knowledge that was gained by protocol-level reverse
engineering.
If only I had more time, I would love to work on improving the lack of
Free / Open Source Software realted to the DECT protocol family.
There's the abandoned deDECTed.org, and the
equally abandoned dect.osmocom.org
project. The former only deals with the loewst levels of DECT
(PHY/MAC). The latter is to a large extent implemented as part of an
ancient version of the Linux kernel (I would say this should all run in
userspace, like we run all of GSM/UMTS/LTE in userspace today).
If anyone wants to help out, I still think working on the DECT DLC and
NWK dissectors for wireshark is the best way to start. It will create a
tool that's important for anyone working with the DECT protocols, and it
will be more or less a requirement for development and debugging should
anyone ever go further in terms of implementing those protocols on
either the PP or FP side. You can find my humble beginnings of the
related dissectors in the laforge/dect branch of osmocom.org/wireshark.git.
Recently I had the pleasure of being part of the 3rd incarnation of a
new podcast series by the Free Software Foundation Europe: The
Software Freedom Podcast.
In episode 3, Matthias
and Bonnie of the FSFE are interviewing me about various high-level
topics related to [the lack of] Free Software in cellular telephony, as
well as some of the projects that I was involved in (Openmoko, Osmocom).
We've also touched the current mainstream / political debate about Huawei and 5G networks,
where my position can only be summarized as: It doesn't matter much
in which country the related proprietary software is being developed.
What we need is Free / Open Source software that can be publicly
audited, and we need a method by which the operator can ensure that
a given version of that FOSS software is actually executing on his
equipment.
Thanks to the FSFE for covering such underdeveloped areas of Free
Software, and to use their podcast to distribute related information and
ideas.
I'm currently working on the firmware for a new project, an 8-slot smart
card reader. I will share more about the architecture and design ideas
behind this project soon, but today I'll simply write about how hard it
sometimes is to actually get software development done. Seemingly trivial
things suddenly take ages. I guess everyone writing code knows this, but
today I felt like I had to share this story.
Chapter 1 - Introduction
As I'm quite convinced of test-driven development these days, I don't
want to simply write firmware code that can only execute in the target,
but I'm actually working on a USB CCID (USb Class for Smart Card
readers) stack which is hardware-independent, and which can also run
entirely in userspace on a Linux device with USB gadget (device)
controller. This way it's much easier to instrument, trace, introspect
and test the code base, and tests with actual target board hardware are
limited to those functions provided by the board.
So the current architecture for development of the CCID implementation
looks like this:
Implement the USB CCID device using FunctionFS
(I did this some months ago, and in fact developing this was a similarly
much more time consuming task than expected, maybe I find time to expand on that)
Attach this USB gadget to a virtual USB bus + host controller using
the Linux kernel dummy_hcd module
Talk to a dumb phoenix style serial SIM card reader attached to a
USB UART, which is connected to an actual SIM card (or any smart card,
for that matter)
By using a "stupid" UART based smart card reader, I am very close to the
target environment on a Cortex-M microcntroller, where I also have to
talk to a UART and hence implement all the beauty of ISO 7816-3. Hence,
the test / mock / development environment is as close as possible to the
target environment.
So I implemented the various bits and pieces and ended up at a point
where I wanted to test. And I'm not getting any response from the UART
/ SIM card at all. I check all my code, add lots of debugging, play
around with various RTS / DTR / ... handshake settings (which sometimes
control power) - no avail.
In the end, after many hours of trial + error I actually inserted a
different SIM card and finally, I got an ATR from the card. In more
than 20 years of working with smart cards and SIM cards, this is the
first time I've actually seen a SIM card die in front of me, with no
response whatsoever from the card.
Chapter 2 - Linux is broken
Anyway, the next step was to get the T=0 protocol of ISO 7816-3 going.
Since there is only one I/O line between SIM card and reader for both
directions, the protocol is a half-duplex protocol. This is unlike
"normal" RS232-style UART communication, where you have a separate Rx
and Tx line.
On the hardware side, this is most often implemented by simply
connecting both the Rx and Tx line of the UART to the SIM I/O pin. This
in turn means that you're always getting an echo back for every byte you
write.
One could discard such bytes, but then I'm targeting a microcontroller,
which should be running eight cards in parallel, at preferably
baud-rates up to ~1 megabit speeds, so having to read and discard all
those bytes seems like a big waste of resources.
The obvious solution around that is to disable the receiver inside the
UART before you start transmitting, and re-enable it after you're done
transmitting. This is typically done rather easily, as most UART
registers in hardware provide some way to selectively enable transmitter
and/or receiver independently.
But since I'm working in Linux userspace in my development environment:
How do I approximate this kind of behavior? At least the older readers
of this blog will remember something called the CREAD flag of termios. Clearing that
flag will disable the receiver. Back in the 1990ies, I did tons of work
with serial ports, and I remembered there was such a flag.
So I implement my userspace UART backend
and somehow it simply doesn't want to work. Again of course I assume I
must be doing something wrong. I'm using strace, I'm single-stepping
through code - no avail.
In the end, it turns out that I've just found a bug in the Linux kernel, one that appears
to be there at least ever since the git history of
linux-2.6.git started. Almost all USB serial device drivers do not
implement CREAD, and there is no sotware fall-back implemented in the
core serial (or usb-serial) handling that would discard any received
bytes inside the kernel if CREAD is cleared. Interestingly, the non-USB
serial drivers for classic UARTs attached to local bus, PCI, ... seem to
support it.
The problem would be half as much of a problem if the syscall to clear
CREAD would actually fail with an error. But no, it simply returns
success but bytes continue to be received from the UART/tty :/
So that's the second big surprise of this weekend...
Chapter 3 - Again a broken card?
So I settle for implementing the 'receive as many characters as you
wrote' work-around. Once that is done, I continue to test the code.
And what happens? Somehow my state machine (implemented using osmo-fsm,
of course) for reading the ATR (code found here)
somehow never wants to complete. The last byte of the ATR always is
missing. How can that be?
Well, guess what, the second SIM card I used is sending a broken,
non-spec compliant ATR where the header indicates 9 historical bytes are
present, but then in reality only 8 bytes are sent by the card.
Of course every reader has a timeout at that point, but that timeout was
not yet implemented in my code, and I also wasn't expecting to hit that
timeout.
So after using yet another SIM card (now a sysmoUSIM-SJS1, not sure why
I didn't even start with that one), it suddenly works.
After a weekend of detours, each of which I would not have assumed at
all before, I finally have code that can obtain the ATR and exchange T=0
TPDUs with cards. Of course I could have had that very easily if I
wanted (we do have code in pySim for this, e.g.) but not in the
architecture that is as close as it gets to the firmware environment of
the microcontroller of my target board.
Back in December 2014 at 31C3, bunnie and xobs presented about their exciting Fernvale project,
how they reverse engineered parts of the MT6260 ARM SoC, which also
happens to contain a Mediatek GSM baseband.
Thousands (at least hundreds) of people have seen that talk live. To date,
2506 people (or AIs?) have watched the recordings on youtube, 4859 more people
on media.ccc.de.
Given that Fernvale was the
closest you could get to having a hackable baseband processor / phone
chip, I expected at least as much interest into this project as we
received four years earlier with OsmocomBB.
As a result, in early 2015, sysmocom decided to order 50 units of Fernvale DVT2
evaluation kits from bunnie, and to offer them in the sysmocom webshop
to ensure the wider community would be able to get the boards they need
for research into widely available, inexpensive 2G baseband chips.
This decision was made purely for the perceived benefit of the
community: Make an exciting project available for anyone. With that
kind of complexity and component density, it's unlikely anyone would
ever solder a board themselves. So somebody has to build some and make
it available. The mark-up sysmocom put on top of bunnie's manufacturing
cost was super minimal, only covering customs/import/shipping fees to
Germany, as well as minimal overhead for packing/picking and accounting.
Now it's almost four years after bunnie + xobs' presentation, and of
those 50 Fernvale boards, we still have 34 (!) units in stock. That means,
only 16 people on this planet ever had an interest in playing with what
at the time I thought was one of the most exciting pieces of equipment
to play with.
So we lost somewhere on the order of close to 3600 EUR in dead
inventory, for something that never was supposed to be a business
anyway. That sucks, but I still think it was worth it.
In case you're thinking "oh, let's wait some more time, until they hand
them out for free", let me tell you: If money is the issue that
prevents you from playing with a Fernvale, then please contact me with
the details about what you'd want to do with it, and we can see about
providing them for free or at substantially reduced cost.
In the worst case, it was ~ 3600 EUR we could have invested in
implementing more Osmocom software, which is sad. But would I do it
again if I saw a very exciting project? Definitely!
The lesson learned here is probably that even a technically very exciting
project backed by world-renowned hackers like bunnie doesn't mean that
anyone will actually ever do anything with it, unless they get
everything handed on a silver plate, i.e. all the software/reversing
work is already done for them by others. And that actually makes
me much more sad than the loss of those ~ 3600 EUR in sysmocom's balance
sheet.
I also feel even more sorry for bunnie + xobs. They've invested time,
money and passion into a project that nobody really seemed to want to
get involved and/or take further. ("nobody" is meant figuratively. I
know there were/are some enthusiasts who did pick up. I'm talking about
the big picture). My condolences to bunnie + xobs!
I recently was reading 3GPP TS 48.049, the specification for the CBSP (Cell
Broadcast Service Protocol), which is the protocol between the BSC (Base
Station Controller) and the CBC (Cell Broadcast Centre). It is how the
CBC according to spec is instructing the BSCs to broadcast the various
cell broadcast messages to their respective geographic scope.
While OsmoBTS and OsmoBSC do have support for SMSCB on the CBCH, there
is no real interface in OsmoBSC yet on how any external application
would instruct it tot send cell broadcasts. The only existing interface
is a VTY command, which is nice for testing and development, but hardly
a scalable solution.
So I was reading up on the specs, discovered CBSP and thought one good
way to get familiar with it is to write a wireshark dissector for it.
You can find the result at https://code.wireshark.org/review/#/c/29745/
Now my main problem is that as usual there appear to be no open source
implementations of this protocol, so I cannot generate any traces
myself. More surprising is that it's not even possible to find any
real-world CBSP traces out there. So I'm facing a chicken-and-egg
problem. I can only test / verify my wireshark dissector if I find some
traces.
So if you happen to have done any work on cell broadcast in 2G network
and have a CBSP trace around (or can generate one): Please send it to me, thanks!
Alternatively, you can of course also use the patch linked above, build
your own wireshark from scratch, test it and provide feedback. Thanks
in either case!
created autoamtic testing of pySim-prog and sysmo-usim-util
extended our osmo-gsm-tester based automatic testing setup to include multi-TRX nanoBTS setups
In terms of other topic,
my wife and I have been to a three week motorbike tour all over the Alps in July
I've done tons of servicing (brake piston fittings, brake tubes, fuel line, fixing rust/paint, replacing clutch cable,
choke cable, transmission chain, replacing several rusted/worn-out needle bearings, and much more) on my 22year old
BMW F650ST to prepare it for many more yers to come. As some type-specific spare parts (mostly plastic
parts) are becoming rarer, it was best to take care of replacements sooner than later
some servicing/repairs to my 19 year old Audi A4 car (which passed German mandatory inspection without any
deficiency at the first attempt!)
some servicing of my Yamaha FZ6
repaired my Fairphone 2 by swapping the microphone module (mike was mute)
I've re-vamped a lot of the physical/hardware infrastructure for gnumonks.org and other sites I run, which was triggered by having to move racks
Some time after Openmoko went out of business, they
made available their USB Vendor IDs and IEEE OUI (Ethernet MAC address prefix)
available to Open Source Hardware / FOSS projects.
After maintaining that for some years myself, I was unable to find time to continue
the work and I had handed it over some time ago to two volunteers. However, as
things go, those volunteers also stopped to respond to PID / OUI requests, and
we're now launching the third attempt of continuing this service.
As the openmoko.org wiki will soon be moved into an archive of static web pages only,
we're also moving the list of allocated PID and OUIs into a git repository.
Since git.openmoko.org is also about to be decommissioned, the repository is now
at https://github.com/openmoko/openmoko-usb-oui, next to all the archived openmoko.org
repository mirrors.
This also means that in addition to sending an e-mail application for getting an allocation
in those ranges, you can now send a pull-request via github.
Thanks to cuvoodoo for volunteering to maintain the
Openmoko USB PID and IEEE OUI allocations from now on!
One of the difficulties with OsmoCon2017
last year was that almost nobody submitted talks / discussions within
the deadline, early enough to allow for proper planning.
This lad to the situation where the sysmocom team had to come up with a
schedule/agenda on their own. Later on much after the CfP
deadline,people then squeezed in talks, making the overall schedule too
full.
It is up to you to avoid this situation again in 2018 at OsmoCon2018 by
submitting your talk RIGHT NOW. We will be very strict regarding late
submissions. So if you would like to shape the Agenda of OsmoCon 2018,
this is your chance. Please use it.
We will have to create a schedule soon, as [almost] nobody will register
to a conference unless the schedule is known. If there's not sufficient
contribution in terms of CfP response from the wider community, don't
complain later that 90% of the talks are from sysmocom team members and
only about the Cellular Network Infrastructure topics.
Unfortunately, since about 11:30 am CEST on MAy 24, openmoko.org is down due to some
power outage related issues at Hetzner, the hosting
company at which openmoko.org has been hosting for more than a decade now.
The problem seems to have caused quite a lot of fall-out tom many
servers (Hetzner is hosting some 200k machines, not sure how many
affected, though), and Hetzner is anything but verbose when it comes to
actually explaining what the issue is.
All they have published is https://www.hetzner-status.de/en.html#8842 -
which is rather tight lipped about some power grid issues. But then,
what do you have UPSs for if not for "a strong voltage reduction in the
local power grid"?
The openmoko.org archive machine is running in Hetzner DC10, by the way.
This is where they've had the largest number of tickets.
In any case, we'll have to wait for them to resolve their tickets.
They appear to be working day and night on that.
I have a number of machines hosted at Hetzner, and I'm actually rather
happy that none of the more important systems were affected that long.
Some machines simply lost their uplink connectivity for some minutes,
while some others were rebooted (power outage). The openmoko.org
archive is the only machine that didn't automatically boot after the
outage, maybe the power supply needs replacement.
In any case, I hope the service will be back up again soon.
btw: Guess who's been paying for hosting costs ever since Openmoko, Inc.
has shut down? Yes, yours truly. It was OK for something like 9 years,
but I want to recursively pull the dynamic content through some cache,
which can then be made permanent. The resulting static archive can then
be moved to some VM somewhere, without requiring a dedicated root
server. That should reduce the costs down to almost nothing.
Recently I've encountered several occasions in which a FOSS project would
have been interested in some reliable, independent mailing list hosting for
their project communication.
I was surprised how difficult it was to find anyone running such a service.
From the user / FOSS project point of view, the criteria that I would have are:
operated by some respected entity that is unlikely to turn hostile,
discontinue the service or go out of business altogether
free of any type of advertisements (we all know how annoying those are)
cares about privacy, i.e. doesn't sell the subscriber lists or
non-public archives
use FOSS to run the service itself, such as GNU mailman, listserv,
ezmlm, ...
an easy path to migrate away to another service (or self-hosting) as
they grow or their requirements change. A simple mail forward to that
new address for the related addresses is typically sufficient for that
If you think mailing lists serve no purpose these days anyways, and
everyone is on github: Please have a look at the many thousands of FOSS
project mailing lists out there still in use. Not everyone wants to
introduce a dependency to the whim of a proprietary
software-as-a-service provider.
I never had this problem as I always hosted my own mailman instance on
lists.gnumonks.org anyway, and all the entities that I've been involved
in (whether non-profit or businesses) had their own mailing list hosts.
From franken.de in the 1990ies to netfilter.org, openmoko.org and now
osmocom.org, we all pride oursevles in self-hosting.
But then there are plenty of smaller projects that neither have the
skills nor the funding available. So they go to yahoo groups or some
other service that will then hold them hostage without a way to switch
their list archives from private to public, without downloadable
archives or forwarding in the case they want to move away :(
Of course the larger FOSS projects also have their own list servers,
starting from vger.kernel.org to Linux distributions like Debian
GNU/Linux. But what if your FOSS project is not specifically Linux
related?
The sort-of obvious candidates that I found all don't really fit:
https://lists.nongnu.org/ is for projects on Savannah (which is much more than just a
mailing list service, many projects don't need or want a "Forge")
Now don't get me wrong, I'm of course not expecting that there are
commercial entities operating free-of charge list hosting services where
you neither pay with money, nor your data, nor by becoming a spam
receiver.
But still, in the wider context of the Free Software community, I'm
seriously surprised that none of the various non-for-profit /
non-commercial foundations or associations are offering a public mailing
list hosting service for FOSS projects.
One can of course always ask any from the above list and ask for a
mailing list even though it's strictly speaking off-topic to them. But
who will do that, if he has to ask uninvited for a favor?
I think there's something missing. I don't have the time to set up a
related service, but I would certainly want to contribute in terms of
funding in case any existing FOSS related legal entity wanted to
expand. If you already have a legal entity, abuse contacts, a team of
sysadmins, then it's only half the required effort.
One week ago, the annual Osmocom developer meeting (OsmoDevCon 2018) concluded
after four long and intense days with old and new friends (schedule can be seen
here).
It was already the 7th incarnation of OsmoDevCon, and I have to say
that it's really great to see the core Osmocom community come together
every year, to share their work and experience with their fellow hackers.
Ever since the beginning we've had the tradition that we look beyond our
own projects. In 2012, David Burgess was presenting on OpenBTS. In
2016, Ismael Gomez presented about srsUE + srsLTE, and this year we've
had the pleasure of having Sukchan Kim coming all the way from Korea to
talk to us about his nextepc project (a FOSS
implementation of the Evolved Packet Core, the 4G core network).
What has also been a regular "entertainment" part in recent years are
the field trip reports to various [former] satellite/SIGINT/... sites
by Dimitri Stolnikov.
All in all, the event has become at least as much about the people than
about technology. It's a community of like-minded people that to some
part are still working on joint projects, but often work independently
and scratch their own itch - whether open source mobile comms related or
not.
After some criticism last year, the so-called "unstructured" part of
OsmoDevCon has received more time again this year, allowing for exchange
among the participants irrespective of any formal / scheduled talk or
discussion topic.
In 2018, with the help of c3voc, for the first
time ever, we've recorded most of the presentations on video. The
results are still in the process of being cut, but are starting to
appear at https://media.ccc.de/c/osmodevcon2018.
If you want to join a future OsmoDevCon in person: Make sure you start
contributing to any of the many Osmocom member projects now to become
eligible. We need you!
Now the sad part is that it will take one entire year until we'll
reconvene. May the Osmocom Developer community live long and prosper.
I want to meet you guys for many more years at OsmoDevCon!
There is of course the user-oriented OsmoCon 2018 in
October, but that's a much larger event with a different audience.
Nevertheless, I'm very much looking forward to that, too.
The OsmoCon 2018 Call for Participation is still running. Please
consider submitting talks if you have anything open source mobile
communications related to share!
Yesterday, during OsmoDevCon 2018,
Steve Markgraf released osmo-fl2k,
a new Osmocom member project which enables the use of FL2000 USB-VGA adapters as
ultra-low-cost SDR transmitters.
How does it work?
A major part of any VGA card has always been a rather fast DAC which
generates the 8-bit analog values for (each) red, green and blue at the
pixel clock. Given that fast DACs were very rare/expensive (and still
are to some extent), the idea of (ab)using the VGA DAC to transmit radio
has been followed by many earlier, mostly proof-of-concept projects,
such as Tempest for Eliza in 2001.
However, with osmo-fl2k,
for the first time it was possible to completely disable the horizontal
and vertical blanking, resulting in a continuous stream of pixels
(samples). Furthermore, as the supported devices have no frame buffer
memory, the samples are streamed directly from host RAM.
As most USB-VGA adapters appear to have no low-pass filters on their DAC
outputs, it is possible to use any of the harmonics to transmit signals
at much higher frequencies than normally possible within the baseband of
the (max) 157 Mega-Samples per seconds that can be achieved.
osmo-fl2k and rtl-sdr
Steve is the creator of the earlier, complementary rtl-sdr software, which since
2012 transforms USB DVB adapters into general-purpose SDR receivers.
Today, six years later, it is hard to think of where SDR would be
without rtl-sdr. Reducing the entry cost of SDR receivers nearly down
to zero has done a lot for democratization of SDR technology.
There is hence a big chance that his osmo-fl2k project will attain a
similar popularity. Having a SDR transmitter for as little as USD 5 is
an amazing proposition.
free riders
Please keep in mind that Steve has done rtl-sdr just for fun, to scratch
his own itch and for the "hack value". He chose to share his work with
the wider public, in source code, under a free software license. He's
a very humble person, he doesn't need to stand in the limelight.
Many other people since have built a business around rtl-sdr. They have
grabbed domains with his project name, etc. They are now earning money
based on what he has done and shared selflessly, without ever
contributing back to the pioneering developers who brought this to all
of us in the first place.
So, do we want to bet if history repeats itself? How long will it take
for vendors showing up online advertising the USB VGA dongles as "SDR
transmitter", possibly even with a surcharge? How long will it take for
them to include Steve's software without giving proper attribution? How
long until they will violate the GNU GPL by not providing the complete
corresponding source code to derivative versions they create?
If you want to thank Steve for his amazing work
reach out to him personally
contribute to his work, e.g.
help to maintain it
package it for distributions
send patches (via osmocom-sdr mailing list)
register an osmocom.org account and update the wiki with more information
And last, but not least, carry on the spirit of "hack value" and
democratization of software defined radio.
Thank you, Steve! After rtl-sdr and osmo-fl2k,
it's hard to guess what will come next :)
When developing applications that exchange data over sockets, every so
often you'd like to analyze exactly what kind of data is exchanged over
the socket.
For TCP/UDP/SCTP/DCCP or other IP-based sockets, this is rather easy by
means of libpcap and tools like tcpdump, tshark or wireshark. However,
for unix domain socket, unfortunately no such general capture/tracing
infrastructure exists in the Linux kernel.
Interestingly, even after searching for quite a bit I couldn't find
any existing tools for this. This is surprising, as unix domain sockets
are used by a variety of programs, from sql servers to bind8 ndc all
the way to the systemctl tool to manage systemd.
In absence of any kernel support, the two technologies I can think of to
implement this is either systemtap or
a LD_PRELOAD wrapper.
However, I couldn't find an example for using either of those two to get
traces of unix domain soocket communications.
Ok, so I get to write my own. My first idea hence was to implement
something based on top of systemtap, the Linux kernel tracing framework.
Unfortunately, systemtap was broken in Debian unstable (which I use for
decades) at the time, so I went back to the good old LD_PRELOAD shim
library / wrapper approach.
Below is a copy+paste of its README file. Let's hope this tool is
useful to other developers, too:
udtrace - Unix Domain socket tracing
This is a LD_PRELOAD wrapper library which can be used to trace the data
sent and/or received via unix domain sockets.
Unlike IP based communication that can be captured/traced with pcap
programs like tcpdump or wireshark, there is no similar mechanism
available for unix domain sockets.
This LD_PRELOAD library intercepts the C library function calls of
dynamically linked programs. It will detect all file descriptors
representing unix domain sockets and will then print traces of all
data sent/received via the socket.
Usage
Simply build libudtrace.so using the make command, and then
start your to-be-traced program with
LD_PRELOAD=libudtrace.os
e.g.
LD_PRELOAD=libudtrace.so systemctl status
which will produce output like this:
>>> UDTRACE: Unix Domain Socket Trace initialized (TITAN support DISABLED)
>>> UDTRACE: Adding FD 4
>>> UDTRACE: connect(4, "/run/dbus/system_bus_socket")
4 sendmsg W 00415554482045585445524e414c20
4 sendmsg W 3331333033303330
4 sendmsg W 0d0a4e45474f54494154455f554e49585f46440d0a424547494e0d0a
[...]
Output Format
Currently, udtrace will produc the following output:
At time a FD for a unix domain socket is created:
>>> UDTRACE: Adding FD 8
At time a FD for a unix domain socket is closed:
>>> UDTRACE: Removing FD 8
At time a FD for a unix domain socket is bound or connected:
>>> UDTRACE: connect(9, "/tmp/mncc")
When data is read from the socket:
9 read R 00040000050000004403000008000000680000001c0300002c03000000000000
When data is written to the socket:
9 write W 00040000050000004403000008000000680000001c0300002c03000000000000
Where
9 is the file dsecriptor on which the event happened
read/write is the name of the syscall, could e.g. also be sendmsg / readv / etc.
R|W is Read / Write (from the process point of view)
followed by a hex-dump of the raw data. Only data successfully
written (or read) will be printed, not the entire buffer passed to
the syscall. The rationale is to only print data that was actually
sent to or received from the socket.
TITAN decoder support
Getting hex-dumps is nice and fine, but normally one wants to have a
more detailed decode of the data that is being passed on the socket.
For TCP based protocols, there is wireshark. But most protocols on unix
domain sockets don't follow inter-operable / public standards, so even
if one was to pass the traces into wireshark somehow, there would be no
decoder.
In the Osmocom project, we already had some type
definitions and decoders for our protocols written in the TTCN-3
programming language, using Eclipse TITAN.
In order to build those decoders fro MNCC and PCUIF, please use
make ENABLE_TITAN=1
when building the code.
Please note that this introduces a run-time dependency to
libttcn3-dynamic.so, which is (at least on Debian GNU/Linux) not
installed in a default library search path, so you will have to use
something like:
LD_LIBRARY_PATH=/usr/lib/titan LD_PRELOAD=libudtrace.so systemctl status
I am not in any way legally involved in the lawsuit on either the
plaintiff or the defendant side. However, as a fellow (former) Linux
kernel developer myself, and a long-term Free Software community member
who strongly believes in the copyleft model, I of course am very
interested in this case.
History of the Case
This case is about GPL infringements in consumer electronics devices
based on a GNU/Linux operating system, including the Linux kernel and at
least some devices netfilter/iptables. The specific devices in question
are a series of satellite TV receivers built by a Shenzhen (China) based
company Geniatech, which is represented in Europe by Germany-based
Geniatech Europe GmbH.
The Geniatech Europe CEO has openly admitted (out of court) that they
had some GPL incompliance in the past, and that there was failure on
their part that needed to be fixed. However, he was not willing to
accept an overly wide claim in the preliminary injunction against his
company.
The history of the case is that at some point in July 2017, Patrick
McHardy has made a test purchase of a Geniatech Europe product, and
found it infringing the GNU General Public License v2. Apparently no
source code (and/or written offer) had been provide alongside the
binary - a straight-forward violation of the license terms and hence a
violation of copyright. The plaintiff then asked the regional court
of Cologne to issue a preliminary injunction against the defendant,
which was granted on September 8th,2017.
In terms of legal procedure, in Germany, when a plaintiff applies for a
preliminary injunction, it is immediately granted by the court after
brief review of the filing, without previously hearing the defendant in
an oral hearing. If the defendant (like in this case) wishes to appeal
the preliminary injunction, it files an appeal which then results in an
oral hearing. This is what happened, after which the district court of
cologne (Landgericht Koeln) on October 20, 2017 issued ruling 14 O 188/17
partially upholding the injunction.
All in all, nothing particularly unusual about this. There is no
dispute about a copyright infringement having existed, and this
generally grants any of the copyright holders the right to have the
infringing party to cease and desist from any further infringement.
However, this injunction has a very wide scope, stating that the
defendant was to cease and desist not only from ever publishing,
selling, offering for download any version of Linux (unless being
compliant to the license). It furthermore asked the defendant to
cease and desist
from putting hyperlinks on their website to any version of Linux
from asking users to download any version of Linux
unless the conditions of the GPL are met, particularly the clauses
related to providing the complete and corresponding source code.
The appeals case at OLG Cologne
The defendant now escalated this to the next higher court, the higher
regional court of Cologne (OLG Koeln), asking to withdraw the earlier
ruling of the lower court, i.e. removing the injunction with its current
scope.
The first very positive surprise at the hearing was the depth in which
the OLG court has studied the subject matter of the dispute prior to the
hearing. In the many GPL related court cases that I witnessed so far, it
was by far the most precise analysis of how Linux kernel development
works, and this despite the more than 1000 pages of filings that parties
had made to the court to this point.
Just to give you some examples:
the court understood that Linux was created by Linus Torvalds in 1991 and
released under GPL to facilitate the open and collaborative development
the court recognized that there is no co-authorship / joint authorship
(German: Miturheber) in the Linux kernel as a whole, as it was not a
group of people planning+developing a given program together, but it
is a program that has been released by Linus Torvalds and has since
been edited by more than 15.000 developers without any "grand joint
plan" but rather in successive iterations. This situation constitutes
"editing authorship" (German: Bearbeiterurheber)
the court further recognized that being listed as "head of the
netfilter core team" or a "subsystem maintainer" doesn't necessarily
mean that one is contributing copyrightable works. Reviewing
thousands of patches doesn't mean you own copyright on them, drawing
an analogy to an editorial office at a publisher.
the court understood there are plenty of Linux versions that may not
even contain any of Patric McHardy's code (such as older versions)
After about 35 minutes of the presiding judge explaining the court's
understanding of the case (and how kernel development works), it went on
to summarize the summary of their internal elaboration at the court
prior to the meeting.
In this summary, the presiding judge stated very clearly that they
believe there is some merit to the arguments of the defendant, and that
they would be inclined in a ruling favorable to the defendant based on
their current understanding of the case.
He cited the following main reasons:
The Linux kernel development model does not support the claim of
Patrick McHardy having co-authored Linux. In so far, he is only
an editing author (Bearbeiterurheber), and not a co-author.
Nevertheless, even an editing author has the right to ask for cease
and desist, but only on those portions that he authored/edited, and
not on the entire Linux kernel.
The plaintiff did not sufficiently show what exactly his contributions
were and how they were forming themselves copyrightable works
The plaintiff did not substantiate what copyrightable contributions he
has made outside of netfilter/iptables. His mere listing as general
networking subsystem maintainer does not clarify what his
copyrightable contributions were
The plaintiff being a member of the netfilter core team or even the
head of the core team still doesn't support the claim of being a
co-author, as netfilter substantially existed since 1999, three years
before Patrick's first contribution to netfilter, and five years
before joining the core team in 2004.
So all in all, it was clear that the court also thought the ruling on
all of Linux was too far-fetching.
The court suggested that it might be better to
have regular main proceedings, in which expert witnesses can be called
and real evidence has to be provided, as opposed to the constraints of
the preliminary procedure that was applied currently.
Some other details that were mentioned somewhere during the hearing:
Patrick McHardy apparently unilaterally terminated the license to his
works in an e-mail dated 26th of July 2017 towards the defendant.
According to the defendant (and general legal opinion, including my
own position), this is in turn a violation of the GPLv2, as it
only allowed plaintiff to create and publish modified versions of
Linux under the obligation that he licenses his works under GPLv2 to
any third party, including the defendant. The defendant believes
this is abuse of his rights (German: Rechtsmissbraeuchlich).
sworn affidavits of senior kernel developer Greg Kroah-Hartman and
current netfilter maintainer Pablo Neira were presented in support of
some of the defendants claims. The contents of those are
unfortunately not public, neither is the contents of the sworn
affidavists presented by the plaintiff.
The defendant has made substantiated claims in his filings that Patrick
McHardy would perform his enforcement activities not with the primary
motivation of achieving license compliance, but as a method to
generate monetary gain. Such claims include that McHardy has acted in
more than 38 cases, in at least one of which he has requested a
contractual penalty of 1.8 million EUR. The total amount of monies
received as contractual penalties was quoted as over 2 million EUR to
this point. Please note that those are claims made by the defendant,
which were just reproduced by the court. The court has not
assessed their validity. However, the presiding judge explicitly
stated that he received a phone calls about this case from a lawyer
known to him personally, who supported that large contractual
penalties are being paid in other related cases.
One argument by the plaintiff seems to center around being listed as
a general kernel networking maintainer until 2017 (despite his latest
patches being from 2015, and those were netfilter only)
Withdrawal by Patrick McHardy
At some point, the court hearing was temporarily suspended to provide the
legal representation of the plaintiff with the opportunity to have a
Phone call with the plaintiff to decide if they would want to continue
with their request to uphold the preliminary injunction. After a few
minutes, the hearing was resumed, with the plaintiff withdrawing their
request to uphold the injunction.
As a result, the injunction is now withdrawn, and the plaintiff has to
bear all legal costs (court fees, lawyers costs on both sides).
Personal Opinion
For me, this is all of course a difficult topic. With my history of
being the first to enforce the GNU GPLv2 in (equally German) court,
it is unsurprising that I am in favor of license enforcement being
performed by copyright holders.
I believe individual developers who have contributed to the Linux
kernel should have the right to enforce the license, if needed. It is
important to have distributed copyright, and to avoid a situation where
only one (possibly industry friendly) entity would be able to take
[legal] action.
I'm not arguing for a "too soft" approach. It's almost 15 years since
the first court cases on license violations on (embedded) Linux, and the
fact that the problem still exists today clearly shows the industry is
very far from having solved a seemingly rather simple problem.
On the other hand, such activities must always be oriented to
compliance, and compliance only. Collecting huge amounts of contractual
penalties is questionable. And if it was necessary to collect such huge
amounts to motivate large corporations to be compliant, then this must
be done in the open, with the community knowing about it, and the
proceeds of such contractual penalties must be donated to free software
related entities to prove that personal financial gain is not a
motivation.
The rumors of Patrick performing GPL enforcement for personal financial
gain have been around for years. It was initially very hard for me to
believe. But as more and more about this became known, and Patrick
would refuse to any contact requests by his former netfilter team-mates
as well as the wider kernel community make it hard to avoid drawing
related conclusions.
We do need enforcement, both out of court and in court. But we need it
to happen out of the closet, with the community in the picture, and
without financial gain to individuals. The "principles of community
oriented enforcement" of the Software Freedom Conservancy as well as the
more recent (but much less substantial) kernel enforcement statement
represent the most sane and fair approach for how we as a community
should deal with license violations.
So am I happy with the outcome? Not entirely. It's good that an
over-reaching injunction was removed. But then, a lot of money and
effort was wasted on this, without any verdict/ruling. It would have
been IMHO better to have a court ruling published, in which the
injunction is substantially reduced in scope (e.g. only about netfilter,
or specific versions of the kernel, or specific products, not about
placing hyperlinks, etc.). It would also have been useful to have some
of the other arguments end up in a written ruling of a court, rather
than more or less "evaporating" in the spoken word of the hearing today,
without advancing legal precedent.
Lessons learned for the developer community
In the absence of detailed knowledge on computer programming, legal folks
tend to look at "metadata" more, as this is what they can understand.
It matters who has which title and when. Should somebody not be
an active maintainer, make sure he's not listed as such.
If somebody ceases to be a maintainer or developer of a project,
remove him or her from the respective lists immediately, not just
several years later.
Copyright statements do matter. Make sure you don't merge any patches
adding copyright statements without being sure they are actually valid.
Lessons learned for the IT industry
There may be people doing GPL enforcement for not-so-noble motives
Defending yourself against claims in court can very well be worth it,
as opposed to simply settling out of court (presumably for some
money). The Telefonica case in 2016 <>_ has shown this, as has this
current Geniatech case. The legal system can work, if you give it a
chance.
Nevertheless, if you have violated the license, and one of the
copyright holders makes a properly substantiated claim, you still will
get injunctions granted against you (and rightfully so). This was
just not done in this case (not properly substantiated, scope of
injunction too wide/coarse).
Dear Patrick
For years, your former netfilter colleagues and friends wanted to have a
conversation with you. You have not returned our invitation so far.
Please do reach out to us. We won't bite, but we want to share our
views with you, and show you what implications your actions have not
only on Linux, but also particularly on the personal and professional
lives of the very developers that you worked hand-in-hand with for
a decade. It's your decision what you do with that information afterwards,
but please do give us a chance to talk. We would greatly appreciate if
you'd take up that invitation for such a conversation. Thanks.
As 2017 has just concluded, let's have a look at the major events and improvements in the Osmocom Cellular
Infrastructure projects (i.e. those projects dealing with building protocol stacks and network elements for
mobile network infrastructure.
I've prepared a detailed year 2017 summary at the osmocom.org website,
but let me write a bit about the most note-worthy topics here.
NITB Split
Once upon a time, we implemented everything needed to operate a GSM network inside a single process called
OsmoNITB. Those days are now gone, and we have separate OsmoBSC, OsmoMSC, OsmoHLR, OsmoSTP processes, which
use interfaces that are interoperable with non-Osmocom implementations (which is what some of our users
require).
This change is certainly the most significant change in the close-to-10-year history of the project. However,
we have tried to make it as non-intrusive as possible, by using default point codes and IP addresses which
will make the individual processes magically talk to each other if installed on a single machine.
We'll continue to improve the user experience, to re-introduce some of the features lost in the split, such as
the ability to attach names to the subscribers.
Testing
We have osmo-gsm-tester together with the two physical setups at the sysmocom office, which continuously run
the latest Osmocom components and test an entire matrix of different BTSs, software configurations and modems.
However, this tests at super low load, and it tests only signalling so far, not user plane yet. Hence,
coverage is limited.
We also have unit tests as part of the 'make check' process, Jenkins based build verification before merging
any patches, as well as integration tests for some of the network elements in TTCN-3. This is much more than
we had until 2016, but still by far not enough, as we had just seen at the fall-out from the sub-optimal 34C3
event network.
OsmoCon
2017 also marks the year where we've for the first time organized a user-oriented event. It was a huge
success, and we will for sure have another OsmoCon incarnation in 2018 (most likely in May or June). It will
not be back-to-back with the developer conference OsmoDevCon this time.
SIGTRAN stack
We have a new SIGTRAN stakc with SUA, M3UA and SCCP as well as OsmoSTP. This has been lacking a long time.
OsmoGGSN
We have converted OpenGGSN into a true member of the Osmocom family, thereby deprecating OpenGGSN which we had
earlier adopted and maintained.
At the 34th annual Chaos Communication Congress,
a team of Osmocom folks continued the many years old tradition of operating
an experimental Osmocom based GSM network at the event. Though I've originally
started that tradition, I'm not involved in installation and/or operation
of that network, all the credits go to Lynxis, neels, tsaitgaist and the
larger team of volunteers surrounding them. My involvement was only
to answer the occasional technical question and to look at bugs that
show up in the software during operation, and if possible fix them on-site.
34C3 marks two significant changes in terms of its cellular network:
the new post-nitb Osmocom stack was used, with OsmoBSC, OsmoMSC and OsmoHLR
both an GSM/GPRS network (on 1800 MHz) was operated ,as well as (for
the first time) an UMTS network (in the 850 MHz band)
The good news is: The team did great work building this network from
scratch, in a new venue, and without relying on people that have
significant experience in network operation. Definitely, the team was
considerably larger and more distributed than at the time when I was
still running that network.
The bad news is: There was a seemingly endless number of bugs that were discovered
while operating this network. Some shortcomings were known before, but
the extent and number of bugs uncovered all across the stack was quite
devastating to me. Sure, at some point from day 2 onwards we had a
network that provided [some level of] service, and as far as I've
heard, some ~ 23k calls were switched over it. But that was after more
than two days of debugging + bug fixing, and we still saw unexplained
behavior and crashes later on.
This is such a big surprise as we have put a lot of effort into testing
over the last years. This starts from the osmo-gsm-tester
software and continuously running test setup, and continues with the
osmo-ttcn3-hacks
integration tests that mainly I wrote during the last few months. Both
us and some of our users have also (successfully!) performed
interoperability testing with other vendors' implementations such as
MSCs. And last, but not least, the individual Osmocom developers had
been using the new post-NITB stack on their personal machines.
So what does this mean?
I'm sorry about the sub-standard state of the software and the
resulting problems we've experienced in the 34C3 network. The extent
of problems surprised me (and I presume everyone else involved)
I'm grateful that we've had the opportunity to discover all those
bugs, thanks to the GSM team at 34C3, as well as Deutsche Telekom for
donating 3 ARFCNs from their spectrum, as well as the German
regulatory authority Bundesnetzagentur for
providing the experimental license in the 850 MHz spectrum.
We need to have even more focus on automatic testing than we had so
far. None of the components should be without exhaustive test coverage
on at least the most common transactions, including all their failure
modes (such as timeouts, rejects, ...)
My preferred method of integration testing has been by using TTCN-3 and
Eclipse TITAN to
emulate all the interfaces surrounding a single of the Osmocom programs
(like OsmoBSC) and then test both valid and invalid transactions. For
the BSC, this means emulating MS+BTS on Abis; emulating MSC on A;
emulating the MGW, as well as the CTRL and VTY interfaces.
I currently see the following areas in biggest need of integration
testing:
OsmoHLR (which needs a GSUP implementation in TTCN-3, which I've
created on the spot at 34C3)
where we e.g. discovered that updates to the subscriber via VTY/CTRL would
surprisingly not result in an InsertSubscriberData to VLR+SGSN
OsmoMSC, particularly when used with external MNCC handlers,
which was so far blocked by the lack of a MNCC implementation in
TTCN-3, which I've been working on both on-site and after returning
back home.
user plane testing for OsmoMGW and other components. We currently
only test the control plane (MGCP), but not the actual user plane
e.g. on the RTP side between the elements
UMTS related testing on OsmoHNBGW, OsmoMSC and OsmoSGSN. We currently
have no automatic testing at all in these areas.
Even before 34C3 and the above-mentioned experiences, I concluded that
for 2018 we will pursue a test-driven development approach for all new
features added by the sysmocom team to the Osmocom code base. The
experience with the many issues at 34C3 has just confirmed that
approach. In parallel, we will have to improve test coverage on the
existing code base, as outlined above. The biggest challenge will of
course be to convince our paying customers of this approach, but I see
very little alternative if we want to ensure production quality of
our cellular stack.
As the Software Freedom Conservancy (SFC) has publicly disclosed
on their website,
it appears that Software Freedom Law Center (SFLC) has filed for a
trademark infringement lawsuit against SFC.
SFLC has launched SFC in 2006, and SFLC has helped and endorsed SFC in
the past.
This lawsuit is hard to believe. What has this community come to, if
its various members - who used all to be respected equally - start
filing law suits against each other?
It's of course not known what kind of negotiations might have happened
out-of-court before an actual lawsuit has been filed. Nevertheless,
one would have hoped that people are able to talk to each other,
and that the mutual respect for working at different aspects and with
possibly slightly different strategies would have resulted in a less
confrontational approach to resolving any dispute.
To me, this story just looks like there can only be losers on all sides,
by far not just limited to the two entities in question.
On lwn.net some people, including
high-ranking members of the FOSS community have started to spread
conspiracy theories as to whether there's any secret scheming behind the
scenes, particularly from the Linux Foundation towards SFLC to cause
trouble towards the SFC and their
possibly-not-overly-enjoyed-by-everyone enforcement activities.
I think this is complete rubbish. Neither have I ever had the
impression that the LF is completely opposed to license enforcement
to begin with, nor do I have remotely enough phantasy to see them
engage in such malicious scheming.
What motivates SFLC and/or Eben to attack their former offspring
is however unexplainable to the bystander. One hopes there is no
connection to his departure from FSF about one year ago,
where he served as general counsel for more than two decades.
The enforcement statement then specifically expresses the view of the
signatories on the specific aspect of the license termination.
Particularly in the US, among legal scholars there is a strong opinion
that if the rights under the GPLv2 are terminated due to non-compliance,
the infringing entity needs an explicit reinstatement of rights from the
copyright holder. The enforcement statement now basically states that
the signatories believe the rights should automatically be re-instated
if the license violation ceases within 30 days of being notified of the
license violation
To people like me living in the European (and particularly German) legal
framework, this has very little to no implications. It has been the
major legal position that any user, even an infringing user can
automatically obtain a new license as soon as he no longer violates. He
just (really or imaginary) obtains a new copy of the source code, at
which time he again gets a new license from the copyright holders, as
long as he fulfills the license conditions.
So my personal opinion as a non-legal person active in GPL compliance on
the reinstatement statement is that it changes little to nothing
regarding the jurisdiction that I operate in. It merely expresses that
other developers express their intent and interest to a similar approach
in other jurisdictions.
Sometimes one is finding an interesting problem and is surprised that
there is not a multitude of blog post, stackoverflow answers or the like
about it.
A (I think) not so uncommon problem when working with datagram sockets
is that you may want to know the local IP address that the OS/kernel
chooses when sending a packet to a given destination.
In an unbound UDP socket, you basically send and receive packets with
any number of peers from a single socket. When sending a packet to
destination Y, you simply pass the destination address/port into the
sendto() socket function, and the OS/kernel will figure out which of
its local IP addresses will be used for reaching this particular
destination.
If you're a dumb host with a single default router, then the answer to
that question is simple. But in any reasonably non-trivial use case,
your host will have a variety of physical and/or virtual network devices
with any number of addresses on them.
Why would you want to know that address? Because maybe you need to
encode that address as part of a packet payload. In the current use
case that we have, it is the OsmoMGW, implementing the IETF
MGCP Media Gateway Control Protocol.
So what can you do? You can actually create a new "trial" socket, not
bind it to any specific local address/port, but connect() it to the
destination of your IP packets. Then you do a getsockname(), which
will give you the local address/port the kernel has selected for this
socket. And that's exactly the answer to your question. You can now
close the "trial" socket and have learned which local IP address the
kernel would use if you were to send a packet to that destination.
At least on Linux, this works. While getsockname() is standard BSD
sockets API, I'm not sure how portable it is to use it on a socket that
has not been explicitly bound by a prior call to bind().
First of all, I wouldn't have expected netfilter to be that relevant
next to all the other [core] networking topics at netdevconf. Secondly,
I've not been doing any work on netfilter for about a decade now, so my
memory is a bit rusty by now ;)
Speaking of Rusty: Timing wise there is apparently a nice coincidence that
I'll be able to meet up with him in Berlin later this month, i.e. hopefully
we can spend some time reminiscing about old times and see what kind of useful
input he has for the keynote.
I'm also asking my former colleagues and successors in the netfilter
project to share with me any note-worthy events or anecdotes,
particularly also covering the time after my retirement from the core
team. So if you have something that you believe shouldn't miss in a
keynote on netfilter project history: Please reach out to me by e-mail
ASAP and let me know about it.
To try to fend off the elder[ly] statesmen image that goes along with
being invited to give keynotes about the history of projects you were
working on a long time ago, I also submitted an actual technical talk:
TTCN-3 and Eclipse Titan for testing protocol stacks, in
which I'll cover my recent journey into TTCN-3 and TITAN land, and how I
think those tools can help us in the Linux [kernel] networking
community to productively produce tests for the various protocols.
As usual for netdevconf, there are plenty of other exciting talks in
the schedule
I'm very much looking forward to both visiting Seoul again, as well as
meeting lots of the excellent people involved in the Linux networking
subsystems. See ya!
On this occasion, a number of the key people managed to gather for an
anniversary dinner in Taipei. Thanks for everyone who could make it, it
was very good to see them together again. Sadly, by far not everyone
could attend. You have been missed!
The award for the most crazy attendee of the meeting goes out to my
friend Milosch, who has actually flown from
his home in the UK to Taiwan, only to meet up with old friends and
attend the anniversary dinner.
In case you're wondering about the lack of activity not only on this
blog but also in git repositories, mailing lists and the like: I've
been on vacation since September 13. It's my usual "one month in Taiwan"
routine, during which I spend some time in Taipei, but also take several
long motorbike tours around mostly rural Taiwan.
There's a new project currently undergoing crowd funding that might be
of interest to the former Openmoko community: The Purism Librem 5
campaign.
Similar to Openmoko a decade ago, they are
aiming to build a FOSS based smartphone built on GNU/Linux without any
proprietary drivers/blobs on the application processor, from
bootloader to userspace.
Furthermore (just like Openmoko) the baseband processor is fully
isolated, with no shared memory and with the Linux-running application
processor being in full control.
They go beyond what we wanted to do at Openmoko in offering hardware
kill switches for camera/phone/baseband/bluetooth. During Openmoko days
we assumed it is sufficient to simply control all those bits from the
trusted Linux domain, but of course once that might be compromised, a
physical kill switch provides a completely different level of security.
I wish them all the best, and hope they can leave a better track record
than Openmoko. Sure, we sold some thousands of phones, but the company
quickly died, and the state of software was far from end-user-ready. I
think the primary obstacles/complexities are verification of the
hardware design as well as the software stack all the way up to the UI.
The budget of ~ 1.5 million seems extremely tight from my point of view,
but then I have no information about how much Puri.sm is able to invest
from other sources outside of the campaign.
If you're a FOSS developer with a strong interest in a Free/Open
privacy-first smartphone, please note that they have several job openings, from
Kernel Developer to
OS Developer
to UI Developer.
I'd love to see some talents at work in that area.
It's a bit of a pity that almost all of the actual technical details are
unspecified at this point (except RAM/flash/main-cpu). No details on
the cellular modem/chipset used, no details on the camera, neither on
the bluetooth chipset, wifi chipset, etc. This might be an indication
of the early stage of their plannings. I would have expected that one
has ironed out those questions before looking for funding - but then,
it's their campaign and they can run it as they see it fit!
I for my part have just put in a pledge for one phone. Let's see what
will come of it. In case you feel motivated by this post to join in:
Please keep in mind that any crowdfunding campaign bears significant
financial risks. So please make sure you made up your mind and don't
blame my blog post for luring you into spending money :)
Cellular modems have existed for decades and come in many shapes and kinds. They contain the cellular
baseband processor, RF frontend, protocol stack software and anything else required to communicate with a
cellular network. Basically a phone without display or input.
During the last decade or so, the vast majority of cellular modems come as LGA modules, i.e. a small PCB with
all components on the top side (and a shielding can), which has contact pads on the bottom so you can solder
it onto your mainboard. You can obtain them from vendors such as Sierra Wireless, u-blox, Quectel, ZTE,
Huawei, Telit, Gemalto, and many others.
In most cases, the vendors now also solder those modules to small adapter boards to offer the same product
in mPCIe form-factor. Other modems are directly manufactured in mPCIe or NGFF aka m.2 form-factor.
As long as those modems were still 2G / 2.5G / 2.75G, the main interconnection with the host (often some
embedded system) was a serial UART. The Audio input/output for voice calls was made available as analog
signals, ready to connect a microphone and spekaer, as that's what the cellular chipsets were designed for in
the smartphones. In the Openmoko phones we also interfaced the audio of the cellular modem in analog, exactly
for that reason.
From 3G onwards, the primary interface towards the host is now USB, with the modem running as a USB device.
If your laptop contains a cellular modem, you will see it show up in the lsusb output.
From that point onwards, it would have made a lot of sense to simply expose the audio also via USB. Simply
offer a multi-function USB device that has both whatever virutal serial ports for AT commands and network
device for IP, and add a USB Audio device to it. It would simply show up as a "USB sound card" to the host,
with all standard drivers working as expected. Sadly, nobody seems to have implemented this, at least not in
a supported production version of their product
Instead, what some modem vendors have implemented as an ugly hack is the transport of 8kHz 16bit PCM samples
over one of the UARTs. See for example the Quectel UC-20 or the Simcom SIM7100 which implement such a method.
All the others ignore any acess to the audio stream from software to a large part. One wonders why that is.
From a software and systems architecture perspective it would be super easy. Instead, what most vendors do,
is to expose a digital PCM interface. This is suboptimal in many ways:
there is no mPCIe standard on which pins PCM should be exposed
no standard product (like laptop, router, ...) with mPCIe slot will have anything connected to those PCM
pins
Furthermore, each manufacturer / modem seems to support a different subset of dialect of the PCM interface in
terms of
voltage (almost all of them are 1.8V, while mPCIe signals normally are 3.3V logic level)
master/slave (almost all of them insist on being a clock master)
sample format (alaw/ulaw/linear)
clock/bit rate (mostly 2.048 MHz, but can be as low as 128kHz)
frame sync (mostly short frame sync that ends before the first bit of the sample)
endianness (mostly MSB first)
clock phase (mostly change signals at rising edge; sample at falling edge)
It's a real nightmare, when it could be so simple. If they implemented USB-Audio, you could plug a cellular
modem into any board with a mPCIe slot and it would simply work. As they don't, you need a specially designed
mainboard that implements exactly the specific dialect/version of PCM of the given modem.
By the way, the most "amazing" vendor seems to be u-blox. Their Modems support PCM audio, but only the
solder-type version. They simply didn't route those signals to the mPCIe slot, making audio impossible to use
when using a connectorized modem. How inconvenient.
Summary
If you want to access the audio signals of a cellular modem from software, then you either
have standard hardware and pick one very specific modem model and hope this is available sufficiently long during your application, or
build your own hardware implementing a PCM slave interface and then pick + choose your cellular modem
On the Osmocom mpcie-breakout board and the sysmocom
QMOD board we have exposed the PCM related pins on
2.54mm headers to allow for some separate board to pick up that PCM and offer it to the host system. However,
such separate board hasn't been developed so far.
For many years I've been fascinated by the XMOS XCore architecture. It offers a surprisingly refreshing alternative
virtually any other classic microcontroller architectures out there. However, despite reading a lot about it
years ago, being fascinated by it, and even giving a short informal presentation about it once, I've so far
never used it. Too much "real" work imposes a high barrier to spending time learning about new architectures,
languages, toolchains and the like.
Introduction into XCore
Rather than having lots of fixed-purpose built-in "hard core" peripherals for interfaces such as SPI, I2C,
I2S, etc. the XCore controllers have a combination of
I/O ports for 1/4/8/16/32 bit wide signals, with SERDES, FIFO, hardware strobe generation, etc
Clock blocks for using/dividing internal or external clocks
hardware multi-threading that presents 8 logical threads on each core
xCONNECT links that can be used to connect multiple processors over 2 or 5 wires per direction
channels as a means of communication (similar to sockets) between threads, whether on the same xCORE or
a remote core via xCONNECT
an extended C (xC) programming language to make use of parallelism, channels and the I/O ports
In spirit, it is like a 21st century implementation of some of the concepts established first with
Transputers.
My main interest in xMOS has been the flexibility that you get in implementing not-so-standard electronics
interfaces. For regular I2C, UART, SPI, etc. there is of course no such need. But every so often one
encounters some interface that's very rately found (like the output of an E1/T1 Line Interface Unit).
Also, quite often I run into use cases where it's simply impossible to find a microcontroller with a
sufficient number of the related peripherals built-in. Try finding a microcontroller with 8 UARTs, for
example. Or one with four different PCM/I2S interfaces, which all can run in different clock domains.
The existing options of solving such problems basically boil down to either implementing it in hard-wired
logic (unrealistic, complex, expensive) or going to programmable logic with CPLD or FPGAs. While the latter
is certainly also quite interesting, the learning curve is steep, the tools anything but easy to use and the
synthesising time (and thus development cycles) long. Furthermore, your board design will be more complex as
you have that FPGA/CPLD and a microcontroller, need to interface the two, etc (yes, in high-end use cases
there's the Zynq, but I'm thinking of several orders of magnitude less complex designs).
Of course one can also take a "pure software" approach and go for high-speed bit-banging. There are some ARM
SoCs that can toggle their pins. People have reported rates like 14 MHz being possible on a Raspberry Pi.
However, when running a general-purpose OS in parallel, this kind of speed is hard to do reliably over long
term, and the related software implementations are going to be anything but nice to write.
So the XCore is looking like a nice alternative for a lot of those use cases. Where you want a
microcontroller with more programmability in terms of its I/O capabilities, but not go as far as to go full-on
with FPGA/CPLD development in Verilog or VHDL.
My current use case
My current use case is to implement a board that can accept four independent PCM inputs (all in slave mode,
i.e. clock provided by external master) and present them via USB to a host PC. The final goal is to have
a board that can be combined with the sysmoQMOD and
which can interface the PCM audio of four cellular modems concurrently.
While XMOS is quite strong in the Audio field and you can find existing examples and app notes for I2S and
S/PDIF, I couldn't find any existing code for a PCM slave of the given requirements (short frame sync, 8kHz
sample rate, 16bit samples, 2.048 MHz bit clock, MSB first).
I wanted to get a feeling how well one can implement the related PCM slave. In order to test the slave, I
decided to develop the matching PCM master and run the two against each other. Despite having never written
any code for XMOS before, nor having used any of the toolchain, I was able to implement the PCM master and PCM
slave within something like ~6 hours, including simulation and verification. Sure, one can certainly do that
in much less time, but only once you're familiar with the tools, programming environment, language, etc. I
think it's not bad.
The biggest problem was that the clock phase for a clocked output port cannot be configured, i.e. the XCore
insists on always clocking out a new bit at the falling edge, while my use case of course required the
opposite: Clocking oout new signals at the rising edge. I had to use a second clock block to generate the
inverted clock in order to achieve that goal.
I found the various pieces of documentation extremely useful and very well written.
Fast progress
I was able to make fast progress in solving the first task using the XMOS / Xcore approach.
Soft Cores developed in public, with commit log
You can find plenty of soft cores that XMOS has been developing on github at https://github.com/xcore,
including the full commit history.
This type of development is a big improvement over what most vendors of smaller microcontrollers like
Atmel are doing (infrequent tar-ball code-drops without commit history). And in the case of the classic uC
vendors, we're talking about drivers only. In the XMOS case it's about the entire logic of the peripheral!
You can for example see that for their I2C core, the very active commit history goes back to January 2011.
xSIM simulation extremely helpful
The xTIMEcomposer IDE (based on Eclipse) contains extensive tracing support and an extensible near cycle
accurate simulator (xSIM). I've implemented a PCM mater and PCM slave in xC and was able to simulate the
program while looking at the waveforms of the logic signals between those two.
The bad parts
Unfortunately, my extremely enthusiastic reception of XMOS has suffered quite a bit over time. Let me explain
why:
Hard to get XCore chips
While the product portfolio on on the xMOS website looks
extremely comprehensive, the vast majority of the parts is not available from stock at distributors. You
won't even get samples, and lead times are 12 weeks (!). If you check at digikey, they have listed a total of
302 different XMOS controllers, but only 35 of them are in stock. USB capable are 15. With other distributors
like Farnell it's even worse.
I've seen this with other semiconductor vendors before, but never to such a large extent. Sure, some
packages/configurations are not standard products, but having only 11% of the portfolio actually available is
pretty bad.
In such situations, where it's difficult to convince distributors to stock parts, it would be a good idea for
XMOS to stock parts themselves and provide samples / low quantities directly. Not everyone is able to order
large trays and/or capable to wait 12 weeks, especially during the R&D phase of a board.
Extremely limited number of single-bit ports
In the smaller / lower pin-count parts, like the XU[F]-208 series in QFN/LQFP-64, the number of usable,
exposed single-bit ports is ridiculously low. Out of the total 33 I/O lines available, only 7 can be used as
single-bit I/O ports. All other lines can only be used for 4-, 8-, or 16-bit ports. If you're dealing
primarily with serial interfaces like I2C, SPI, I2S, UART/USART and the like, those parallel ports are of no
use, and you have to go for a mechanically much larger part (like XU[F]-216 in TQFP-128) in order to have a
decent number of single-bit ports exposed. Those parts also come with twice the number of cores, memory, etc-
which you don't need for slow-speed serial interfaces...
Insufficient number of exposed xLINKs
The smaller parts like XU[F]-208 only have one xLINK exposed. Of what use is that? If you don't have at least
two links available, you cannot daisy-chain them to each other, let alone build more complex structures like
cubes (at least 3 xLINKs).
So once again you have to go to much larger packages, where you will not use 80% of the pins or resources,
just to get the required number of xLINKs for interconnection.
Change to a non-FOSS License
XMOS deserved a lot of praise for releasing all their soft IP cores as Free / Open Source Software
on github at https://github.com/xcore. The License has basically been a 3-clause BSD license. This was a good move, as it meant that anyone
could create derivative versions, whether proprietary or FOSS, and there would be virtually no license
incompatibilities with whatever code people wanted to write.
However, to my very big disappointment, more recently XMOS seems to have changed their policy on this.
New soft cores (released at https://github.com/xmos as opposed to the old https://github.com/xcore) are made
available under a non-free license. This license
is nothing like BSD 3-clause license or any other Free Software or Open Source license. It restricts the
license to use the code together with an XMOS product, requires the user to contribute fixes back to XMOS and
contains references to importand export control. This license is incopatible with probably any FOSS license
in existance, making it impossible to write FOSS code on XMOS while using any of the new soft cores released
by XMOS.
But even beyond that license change, not even all code is provided in source code format anymore. The new
USB library (lib_usb) is provided as binary-only library, for example.
If you know anyone at XMOS management or XMOS legal with whom I could raise this topic of license change
when transitioning from older sc_* software to later lib_* code, I would appreciate this a lot.
Proprietary Compiler
While a lot of the toolchain and IDE is based on open source (Eclipse, LLVM, ...), the actual xC compiler is
proprietary.
So far, in many Osmocom projects we have unit tests next to the code.
Those unit tests are executing test on a per-C-function basis, and
typically use the respective function directly from a small test
program, executed at make check time. The actual main program (like
OsmoBSC or OsmoBTS) is not executed at that time.
We also have VTY testing, which specifically tests that the VTY
has proper documentation for all nodes of all commands.
Then there's a big gap, and we have osmo-gsm-tester for testing a full
cellular network end-to-end. It includes physical GSM modesm, coaxial
distribution network, attenuators, splitter/combiners, real BTS hardware
and logic to run the full network, from OsmoBTS to the core - both for
OsmoNITB and OsmoMSC+OsmoHLR based networks.
However, I think a lot of testing falls somewhere in between, where you
want to run the program-under-test (e.g. OsmoBSC), but you don't want to
run the MS, BTS and MSC that normally surroudns it. You want to test it
by emulating the BTS on the Abis sid and the MSC on the A side, and just
test Abis and A interface transactions.
For this kind of testing, I have recently started to investigate
available options and tools.
OsmoSTP (M3UA/SUA)
Several months ago, during the development of OsmoSTP, I disovered that
the Network Programming Lab of Münster University of Applied Sciences led by
Michael Tuexen had released implementations of the ETSI test suite for the M3UA
and SUA members of the SIGTRAN protocol family.
The somewhat difficult part is that they are implemented in scheme,
using the guile interpreter/compiler, as well as a C-language based
execution wrapper, which then is again called by another guile wrapper
script.
I've also cleaned up the Dockerfiles and related image generation for
the osmo-stp-master, m3ua-test and sua-test images, as well
as some scripts to actually execute them on one of the Builders. You
can find related Dockerfiles as well as associtaed Makfiles in
http://git.osmocom.org/docker-playground
The end result after integration with Osmocom jenkins can be seen in the
following examples on jenkins.osmocom.org
for M3UA
and for SUA
Triggering the builds is currently periodic once per night, but we could
of course also trigger them automatically at some later point.
I've also packaged the GGSN and the test cases each into separate Docker
containers called osmo-ggsn-latest and ggsn-test. Related
Dockerfiles and Makefiles can again be found in
http://git.osmocom.org/docker-playground - together with a Eclipse
TITAN Docker base image using Debian Stretch called debian-stretch-titan
I've built some infrastructure for Gb (NS/BSSGP), VirtualUm and other
testing, but yet have to build Docker images and related jenkins
integration for it. Stay tuned about that. Also, lots more actual
tests cases are required. I'm very much looking forward to any
contributions.
Cellular systems ever since GPRS are using a tunnel based architecture to provide IP
connectivity to cellular terminals such as phones, modems, M2M/IoT devices and the
like. The MS/UE establishes a PDP context between itself and the GGSN on the other
end of the cellular network. The GGSN then is the first IP-level router, and the
entire cellular network is abstracted away from the User-IP point of view.
This architecture didn't change with EGPRS, and not with UMTS, HSxPA and even
survived conceptually in LTE/4G.
While the concept of a PDP context / tunnel exists to de-couple the
transport layer from the structure and type of data inside the tunneled
data, the primary user plane so far has been IPv4.
In Osmocom, we made sure that there are no impairments / assumptions
about the contents of the tunnel, so OsmoPCU and OsmoSGSN do not care at
all what bits and bytes are transmitted in the tunnel.
The only Osmocom component dealing with the type of tunnel and its
payload structure is OpenGGSN.
The GGSN must allocate the address/prefix assigned to each individual
MS/UE, perform routing between the external IP network and the cellular
network and hence is at the heart of this. Sadly, OpenGGSN was an
abandoned project for many years until Osmocom adopted it, and it only
implemented IPv4.
This is actually a big surprise to me. Many of the users of the Osmocom
stack are from the IT security area. They use the Osmocom stack to
test mobile phones for vulnerabilities, analyze mobile malware and the
like. As any penetration tester should be interested in analyzing all
of the attack surface exposed by a given device-under-test, I would have
assumed that testing just on IPv4 would be insufficient and over the
past 9 years, somebody should have come around and implemented the
missing bits for IPv6 so they can test on IPv6, too.
In reality, it seems nobody appears to have shared line of thinking and
invested a bit of time in growing the tools used. Or if they did, they
didn't share the related code.
In June 2017, Gerrie Roos submitted a patch for OpenGGSN IPv6 support that raised hopes about soon
being able to close that gap. However, at closer sight it turns out
that the code was written against a more than 7 years old version of
OpenGGSN, and it seems to primarily focus on IPv6 on the outer
(transport) layer, rather than on the inner (user) layer.
OpenGGSN IPv6 PDP Context Support
So in July 2017, I started to work on IPv6 PDP support in OpenGGSN.
Initially I thought How hard can it be? It's not like IPv6 is new to
me (I joined 6bone under 3ffe prefixes back in the 1990ies and worked on
IPv6 support in ip6tables ages ago. And aside
from allocating/matching longer addresses, what kind of complexity does
one expect?
After my initial attempt of implementation, partially mislead by the
patch that was contributed against that 2010-or-older version of
OpenGGSN, I'm surprised how wrong I was.
In IPv4 PDP contexts, the process of establishing a PDP context is
simple:
Request establishment of a PDP context, set the type to IETF IPv4
Receive an allocated IPv4 End User Address
Optionally use IPCP (part of PPP) to reques and receive DNS Server
IP addresses
So I implemented the identical approach for IPv6. Maintain a pool of
IPv6 addresses, allocate one, and use IPCP for DNS. And nothing worked.
IPv6 PDP contexts assign a /64 prefix, not a single address or a
smaller prefix
The End User Address that's part of the Signalling plane of Layer 3
Session Management and GTP is not the actual address, but just serves
to generate the interface identifier portion of a link-local IPv6
address
IPv6 stateless autoconfiguration is used with this link-local IPv6
address inside the User Plane, after the control plane signaling to
establish the PDP context has completed. This means the GGSN needs to
parse ICMPv6 router solicitations and generate ICMPV6 router
advertisements.
To make things worse, the stateless autoconfiguration is modified in
some subtle ways to make it different from the normal SLAAC used on
Ethernet and other media:
the timers / lifetimes are different
only one prefix is permitted
only a prefix length of 64 is permitted
A few days later I implemented all of that, but it still didn't work.
The problem was with DNS server adresses. In IPv4, the 3GPP protocols
simply tunnel IPCP frames for this. This makes a lot of sense, as IPCP
is designed for point-to-point interfaces, and this is exactly what a
PDP context is.
In IPv6, the corresponding IP6CP protocol does not have the capability
to provision DNS server addresses to a PPP client. WTF? The IETF
seriously requires implementations to do DHCPv6 over PPP, after
establishing a point-to-point connection, only to get DNS server
information?!? Some people suggested an IETF draft to change this
butthe draft has expired in 2011 and we're still stuck.
While 3GPP permits the use of DHCPv6 in some scenarios, support in
phones/modems for it is not mandatory. Rather, the 3GPP has come up
with their own mechanism on how to communicate DNS server IPv6
addresses during PDP context activation: The use of containers as part
of the PCO Information Element used in L3-SM and GTP (see Section
10.5.6.3 of 3GPP TS 24.008. They by the
way also specified the same mechanism for IPv4, so there's now two
competing methods on how to provision IPv4 DNS server information: IPCP
and the new method.
In any case, after some more hacking, OpenGGSN can now also provide
DNS server information to the MS/UE. And once that was implemented,
I had actual live uesr IPv6 data over a full Osmocom cellular stack!
Summary
We now have working IPv6 User IP in OpenGGSN. Together with the rest
of the Osmocom stack you can operate a private GPRS, EGPRS, UMTS or
HSPA network that provide end-to-end transparent, routed IPv6
connectivity to mobile devices.
All in all, it took much longer than nneeded, and the following
questions remain in my mind:
why did the IETF not specify IP6CP capabilities to configure DNS
servers?
why the complex two-stage address configuration with PDP EUA
allocation for the link-local address first and then stateless
autoconfiguration?
why don't we simply allocate the entire prefix via the End User
Address information element on the signaling plane? For sure next
to the 16byte address we could have put one byte for prefix-length?
why do I see duplication detection flavour neighbour solicitations
from Qualcomm based phones on what is a point-to-point link with
exactly two devices: The UE and the GGSN?
why do I see link-layer source address options inside the ICMPv6
neighbor and router solicitation from mobile phones, when that option
is specifically not to be used on point-to-point links?
why is the smallest prefix that can be allocated a /64? That's such a
waste for a point-to-point link with a single device on the other end,
and in times of billions of connected IoT devices it will just
encourage the use of non-public IPv6 space (i.e. SNAT/MASQUERADING)
while wasting large parts of the address space
Some of those choices would have made sense if one would have made it
fully compatible with normal IPv6 like e.g. on Ethernet. But
implementing ICMPv6 router and neighbor solicitation without getting
any benefit such as ability to have multiple prefixes, prefixes of
different lengths, I just don't understand why anyone ever thought
You can find the code at http://git.osmocom.org/openggsn/log/?h=laforge/ipv6
and the related ticket at https://osmocom.org/issues/2418
During the last couple of days, I've been working on completing, cleaning up and merging a Virtual Um interface
(i.e. virtual radio layer) between OsmoBTS and OsmocomBB. After I started with the implementation and left it in an early
stage in January 2016, Sebastian Stumpf has been completing it around early 2017, with now some subsequent
fixes and improvements by me. The combined result allows us to run a complete GSM network with 1-N BTSs and
1-M MSs without any actual radio hardware, which is of course excellent for all kinds of testing scenarios.
The Virtual Um layer is based on sending L2 frames (blocks) encapsulated via GSMTAP UDP multicast packets.
There are two separate multicast groups, one for uplink and one for downlink. The multicast nature simulates
the shared medium and enables any simulated phone to receive the signal from multiple BTSs via the downlink
multicast group.
In OsmoBTS, this is implemented via the new osmo-bts-virtual BTS model.
In OsmocomBB, this is realized by adding virtphy virtual L1, which speaks the same
L1CTL protocol that is used between the
real OsmcoomBB Layer1 and the Layer2/3 programs such as mobile and the like.
Now many people would argue that GSM without the radio and actual handsets is no fun. I tend to agree, as I'm
a hardware person at heart and I am not a big fan of simulation.
Nevertheless, this forms the basis of all kinds of possibilities for automatized (regression) testing in a way
and for layers/interfaces that osmo-gsm-tester cannot cover as it uses a black-box proprietary mobile phone
(modem). It is also pretty useful if you're traveling a lot and don't want to carry around a BTS and phones
all the time, or get some development done in airplanes or other places where operating a radio transmitter is
not really a (viable) option.
If you're curious and want to give it a shot, I've put together some setup instructions at the
Virtual Um page of the Osmocom Wiki.
It was exciting times, and there was a lot of pioneering spirit:
Building a Linux based smartphone with a 100% FOSS software stack on the
application processor, including all drivers, userland, applications -
at a time before Android was known or announced. As history shows, we'd
been working in parallel with Apple on the iPhone, and Google on
Android. Of course there's little chance that a small taiwanese company
can compete with the endless resources of the big industry giants, and
the many Neo1973 delays meant we had missed the window of opportunity to
be the first on the market.
It's sad that Openmoko (or similar projects) have not survived even as a
special-interest project for FOSS enthusiasts. Today, virtually all
options of smartphones are encumbered with way more proprietary blobs
than we could ever imagine back then.
In any case, the tenth anniversary of trying to change the amount of
Free Softwware in the smartphone world is worth some celebration. I'm
reaching out to old friends and colleagues, and I guess we'll have
somewhat of a celebration party both in Germany and in Taiwan (where
I'll be for my holidays from mid-September to mid-October).
Given that the Free and Open Source movement has been around at least
since the 1980ies, it puzzles me that people still seem to have such
fundamental misconceptions about it.
Something that really triggered me was an article at LightReading[1]
which quotes Ulf Ewaldsson, a leading Ericsson excecutive with
"I have yet to understand why we would open source something we think is
really good software"
This completely misses the point. FOSS is not about making a charity
donation of a finished product to the planet.
FOSS is about sharing the development costs among multiple players, and
avoiding that everyone has to reimplement the wheel.
Macro-Economically, it is complete and utter nonsense that each 3GPP
specification gets implemented two dozens of times, by at least a dozen
of different entities. As a result, products are way more expensive
than needed.
If large Telco players (whether operators or equipment manufacturers)
were to collaboratively develop code just as much as they
collaboratively develop the protocol specifications, there would be no
need for replicating all of this work.
As a result, everyone could produce cellular network elements at reduced
cost, sharing the R&D expenses, and competing in key areas, such as who
can come up with the most energy-efficient implementation, or can
produce the most reliable hardware, the best receiver sensitivity, the
best and most fair scheduling implementation, or whatever else. But
some 80% of the code could probably be shared, as e.g. encoding and
decoding messages according to a given publicly released 3GPP
specification document is not where those equipment suppliers actually
compete.
So my dear cellular operator executives: Next time you're cursing about
the prohibitively expensive pricing that your equipment suppliers quote
you: You only have to pay that much because everyone is reimplementing
the wheel over and over again.
Equally, my dear cellular infrastructure suppliers: You are all dying
one by one, as it's hard to develop everything from scratch. Over the
years, many of you have died. One wonders, if we might still have more
players left, if some of you had started to cooperate in developing FOSS
at least in those areas where you're not competing. You could replicate
what Linux is doing in the operating system market. There's no need in
having a phalanx of different proprietary flavors of Unix-like OSs. It's
way too expansive, and it's not an area in which most companies need to
or want to compete anyway.
Management Summary
You don't first develop and entire product until it is finished and
then release it as open source. This makes little economic sense in
a lot of cases, as you've already invested into developing 100% of it.
Instead, you actually develop a new product collaboratively as FOSS in
order to not have to invest 100% but maybe only 30% or even less. You
get a multitude of your R&D investment back, because you're not only
getting your own code, but all the other code that other community
members implemented. You of course also get other benefits, such as
peer review of the code, more ideas (not all bright people work inside
one given company), etc.
that article is actually a heavily opinionated post by somebody
who appears to be pushing his own anti-FOSS agenda for some time. The
author is misinformed about the fact that the TIP has always included
projects under both FRAND and FOSS terms. As a TIP member I can
attest to that fact. I'm only referencing it here for the purpose of
that that Ericsson quote.
Keep in mind: Osmocom is a much larger umbrella project, and beyond
the Networks-side cellular stack is home many different community-based
projects around open source mobile communications. All of those have
started more or less as just for fun projects, nothing serious,
just a hobby[1]
The projects implementing the network-side protocol stacks and network
elements of GSM/GPRS/EGPRS/UMTS cellular networks are somewhat the
exception to that, as they have evolved to some extent professionalized.
We call those projects collectively the Cellular Infrastructure
projects inside Osmocom. This post is about that part of Osmocom only
History
From late 2008 through 2009, People like Holger and I were working on
bs11-abis and later OpenBSC only in our spare time. The name Osmocom
didn't even exist back then. There was a strong technical community with
contributions from Sylvain Munaut, Andreas Eversberg, Daniel Willmann,
Jan Luebbe and a few others. None of this would have been possible if
it wasn't for all the help we got from Dieter Spaar with the BS-11 [2].
We all had our dayjob in other places, and OpenBSC work was really
just a hobby. People were working on it, because it was where no
FOSS hacker has gone before. It was cool. It was a big and pleasant
challenge to enter the closed telecom space as pure autodidacts.
Holger and I were doing freelance contract development work on Open
Source projects for many years before. I was mostly doing Linux related
contracting, while Holger has been active in all kinds of areas
throughout the FOSS software stack.
In 2010, Holger and I saw some first interest by companies into OpenBSC,
including Netzing AG and On-Waves ehf. So we were able to spend at
least some of our paid time on OpenBSC/Osmocom related contract work,
and were thus able to do less other work. We also continued to spend
tons of spare time in bringing Osmocom forward. Also, the amount of
contract work we did was only a fraction of the many more hours of spare
time.
In 2011, Holger and I decided to start the company sysmocom in order to generate more funding for the
Osmocom GSM projects by means of financing software development by
product sales. So rather than doing freelance work for companies who
bought their BTS hardware from other places (and spent huge amounts of
cash on that), we decided that we wanted to be a full solution
supplier, who can offer a complete product based on all hardware and
software required to run small GSM networks.
The only problem is: We still needed an actual BTS for that. Through
some reverse engineering of existing products we figured out who one of
the ODM suppliers for the hardware + PHY layer was, and decided to
develop the OsmoBTS software to
do so. We inherited some of the early code from work done by Andreas
Eversberg on the jolly/bts branch of OsmocomBB (thanks), but much was
missing at the time.
What follows was Holger and me working several years for free [3], without
any salary, in order to complete the OsmoBTS software, build an embedded
Linux distribution around it based on OE/poky, write documentation, etc.
and complete the first sysmocom product: The
sysmoBTS 1002
We did that not because we want to get rich, or because we want to run a
business. We did it simply because we saw an opportunity to generate
funding for the Osmocom projects and make them more sustainable and
successful. And because we believe there is a big, gaping, huge vacuum
in terms of absence of FOSS in the cellular telecom sphere.
Funding by means of sysmocom product sales
Once we started to sell the sysmoBTS products, we were able to fund
Osmocom related development from the profits made on hardware /
full-system product sales. Every single unit sold made a big
contribution towards funding both the maintenance as well as the ongoing
development on new features.
This source of funding continues to be an important factor today.
Funding by means of R&D contracts
The probably best and most welcome method of funding Osmocom related
work is by means of R&D projects in which a customer funds our work to
extend the Osmocom GSM stack in one particular area where he has a
particular need that the existing code cannot fulfill yet.
This kind of project is the ideal match, as it shows where the true
strength of FOSS is: Each of those customers did not have to fund the
development of a GSM stack from scratch. Rather, they only had to fund
those bits that were missing for their particular application.
Our reference for this is and has been On-Waves, who have been funding
development of their required features (and bug fixing etc.) since 2010.
We've of course had many other projects from a variety of customers over
over the years. Last, but not least, we had a customer who willingly
co-funded (together with funds from NLnet foundation and lots of unpaid
effort by sysmocom) the 3G/3.5G support in the Osmocom stack.
The problem here is:
we have not been able to secure anywhere nearly as many of those R&D
projects within the cellular industry, despite believing we have a
very good foundation upon which we can built. I've been writing many
exciting technical project proposals
you almost exclusively get funding only for new features. But it's
very hard to get funding for the core maintenance work. The
bug-fixing, code review, code refactoring, testing, etc.
So as a result, the profit margin you have on selling R&D projects is
basically used to (do a bad job of) fund those bits and pieces that
nobody wants to pay for.
Funding by means of customer support
There is a way to generate funding for development by providing support
services. We've had some success with this, but primarily alongside the
actual hardware/system sales - not so much in terms of pure
software-only support.
Also, providing support services from a R&D company means:
either you distract your developers by handling support inquiries.
This means they will have less time to work on actual code, and likely
get side tracked by too many issues that make it hard to focus
or you have to hire separate support staff. This of course means that
the size of the support business has to be sufficiently large to not
only cover the cots of hiring + training support staff, but also still
generate funding for the actual software R&D.
We've tried shortly with the second option, but fallen back to the first
for now. There's simply not sufficient user/admin type support business
to rectify dedicated staff for that.
Funding by means of cross-subsizing from other business areas
sysmocom also started to do some non-Osmocom projects in order to
generate revenue that we can feed again into Osmocom projects. I'm not
at liberty to discuss them in detail, but basically we've been doing
pretty much anything from
custom embedded Linux board designs
M2M devices with GSM modems
consulting gigs
public tendered research projects
Profits from all those areas went again into Osmocom development.
Last, but not least, we also operate the sysmocom webshop. The profit we make on those products
also is again immediately re-invested into Osmocom development.
Funding by grants
We've had some success in securing funding from NLnet Foundation for specific features. While this is useful, the
size of their projects grants of up to EUR 30k is not a good fit for the
scale of the tasks we have at hand inside Osmocom. You may think that's
a considerable amount of money? Well, that translates to 2-3 man-months
of work at a bare cost-covering rate. At a team size of 6 developers,
you would theoretically have churned through that in two weeks. Also,
their focus is (understandably) on Internet and IT security, and not so
much cellular communications.
There are of course other options for grants, such as government
research grants and the like. However, they require long-term planning,
they require you to match (i.e. pay yourself) a significant portion,
and basically mandate that you hire one extra person for doing all the
required paperwork and reporting. So all in all, not a particularly
attractive option for a very small company consisting of die hard engineers.
Funding by more BTS ports
At sysmocom, we've been doing some ports of the OsmoBTS + OsmoPCU
software to other hardware, and supporting those other BTS vendors with
porting, R&D and support services.
If sysmocom was a classic BTS vendor, we would not help our
"competition". However, we are not. sysmocom exists to help Osmocom,
and we strongly believe in open systems and architectures, without a
single point of failure, a single supplier for any component or any type
of vendor lock-in.
So we happily help third parties to get Osmocom running on their
hardware, either with a proprietary PHY or with OsmoTRX.
However, we expect that those BTS vendors also understand their
responsibility to share the development and maintenance effort of the
stack. Preferably by dedicating some of their own staff to work in
the Osmocom community. Alternatively, sysmocom can perform that work as
paid service. But that's a double-edged sword: We don't want to be a
single point of failure.
Osmocom funding outside of sysmocom
Osmocom is of course more than sysmocom. Even for the cellular
infrastructure projects inside Osmocom is true: They are true,
community-based, open, collaborative development projects. Anyone can
contribute.
Over the years, there have been code contributions by e.g.
Fairwaves. They, too, build GSM base station hardware and use that as a
means to not only recover the R&D on the hardware, but also to
contribute to Osmocom. At some point a few years ago, there was a lot
of work from them in the area of OsmoTRX, OsmoBTS and OsmoPCU.
Unfortunately, in more recent years, they have not been able to keep up
the level of contributions.
There are other companies engaged in activities with and around Osmcoom.
There's Rhizomatica, an NGO helping
indigenous communities to run their own cellular networks. They have
been funding some of our efforts, but being an NGO helping rural regions
in developing countries, they of course also don't have the deep
pockets. Ideally, we'd want to be the ones contributing to them, not
the other way around.
State of funding
We're making some progress in securing funding from players we cannot
name [4] during recent years. We're also making occasional progress in
convincing BTS suppliers to chip in their share. Unfortunately there
are more who don't live up to their responsibility than those who do.
I might start calling them out by name one day. The wider community and
the public actually deserves to know who plays by FOSS rules and who
doesn't. That's not shaming, it's just stating bare facts.
Which brings us to:
sysmocom is in an office that's actually too small for the team,
equipment and stock. But we certainly cannot afford more space.
we cannot pay our employees what they could earn working at similar
positions in other companies. So working at sysmocom requires
dedication to the cause :)
Holger and I have invested way more time than we have ever paid us,
even more so considering the opportunity cost of what we would have
earned if we'd continued our freelance Open Source hacker path
we're [just barely] managing to pay for 6 developers dedicated to
Osmocom development on our payroll based on the various funding
sources indicated above
Nevertheless, I doubt that any such a small team has ever implemented an
end-to-end GSM/GPRS/EGPRS network from RAN to Core at
comparative feature set. My deepest respects to everyone involved. The
big task now is to make it sustainable.
Summary
So as you can see, there's quite a bit of funding around. However, it
always falls short of what's needed to implement all parts properly, and
even not quite sufficient to keep maintaining the status quo in a proper
and tested way. That can often be frustrating (mostly to us but
sometimes also to users who run into regressions and oter bugs).
There's so much more potential. So many things we wanted to add or
clean up for a long time, but too little people interested in joining
in, helping out - financially or by writing code.
On thing that is often a challenge when dealing with traditional
customers: We are not developing a product and then selling a ready-made
product. In fact, in FOSS this would be more or less suicidal: We'd
have to invest man-years upfront, but then once it is finished, everyone
can use it without having to partake in that investment.
So instead, the FOSS model requires the customers/users to chip in
early during the R&D phase, in order to then subsequently harvest the
fruits of that.
I think the lack of a FOSS mindset across the cellular / telecom
industry is the biggest constraining factor here. I've seen that some
20-15 years ago in the Linux world. Trust me, it takes a lot of
dedication to the cause to endure this lack of comprehension so many
years later.
while you will not find a lot of commits from Dieter in the code, he has been playing a key role in doing a lot of prototyping, reverse engineering and debugging!
sysmocom is 100% privately held by Holger and me, we intentionally have no external investors and are proud to never had to take a bank loan. So all we could invest was our own money and, most of all, time.
In an all-IP GSM network, where we use Abis, A and other interfaces
within the cellular network over IP transport, the audio of voice calls
is transported inside RTP frames. The codec payload in those RTP frames
is the actual codec frame of the respective cellular voice codec. In
GSM, there are four relevant codecs: FR, HR, EFR and AMR.
Every so often during the (meanwhile many years of ) development of
Osmocom cellular infrastructure software it would have been useful to be
able to quickly play back the audio for analysis of given issues.
However, until now we didn't have that capability. The reason is
relatively simple: In Osmocom, we genally don't do transcoding but
simply pass the voice codec frames from left to right. They're only
transcoded inside the phones or inside some external media gateway (in
case of larger networks).
Chapter 1: GSM Audio Pocket Knife
Back in 2010, when we were very actively working on OsmocomBB, the
telephone-side GSM protocol stack implementation, Sylvain Munaut wrote
the GSM Audio Pocket Knife (gapk) in order to be able to
convert between different formats (representations) of codec frames. In
cellular communcations, everyoe is coming up with their own
representation for the codec frames: The way they look on E1 as a TRAU
frame is completely different from how RTP payload looks like, or what
the TI Calypso DSP uses internally, or what a GSM Tester like the Racal
61x3 uses. The differences are mostly about data types used,
bit-endinanness as well as padding and headers. And of course those
different formats exist for each of the four codecs :/
In 2013 I first added simplistic RTP support for FR-GSM to gapk,
which was sufficient for my debugging needs back then. Still, you had
to save the decoded PCM output to a file and play that back, or use a
pipe into aplay.
Last week, I picked up this subject again and added a long series of
patches to gapk:
support for variable-length codec frames (required for AMR support)
support for AMR codec encode/decode using libopencore-amrnb
support of all known RTP payload formats for all four codecs
support for direct live playback to a sound card via ALSA
All of the above can now be combined to make GAPK bind to a specified
UDP port and play back the RTP codec frames that anyone sends to that
port using a command like this:
$ gapk -I 0.0.0.0/30000 -f rtp-amr -A default -g rawpcm-s16le
I've also merged a chance to OsmoBSC/OsmoNITB which allows the
administrator to re-direct the voice of any active voice channel towards
a user-specified IP address and port. Using that you can simply
disconnect the voice stream from its normal destination and play
back the audio via your sound card.
Chapter 2: Bugs in OsmoBTS GSM-HR
While going through the exercise of implementing the above extension to
gapk, I had lots of trouble to get it to work for GSM-HR.
After some more digging, it seems there are two conflicting
specification on how to format the RTP payload for half-rate GSM:
In Osmocom, we claim to implement RFC5993, but it turned out that (at
least) osmo-bts-sysmo (for sysmoBTS) was actually implementing the
ETSI format instead.
And even worse, osmo-bts-sysmo gets event the ETSI format wrong. Each
of the codec parameters (which are unaligned bit-fields) are in the
wrong bit-endianness :(
Both the above were coincidentially also discovered by Sylvain Munaut
during operating of the 32C3 GSM network in December 2015 and resulted
the two following "work around" patches:
* HACK for HR
* HACK: Fix the bit order in HR frames
Those merely worked around those issues in the rtp_proxy of OsmoNITB,
rather than addressing the real issue. That's ok, they were "quick"
hacks to get something working at all during a four-day conference. I'm
now working on "real" fixes in osmo-bts-sysmo. The devil is of course
in the details, when people upgrade one BTS but not the other and want
to inter-operate, ...
It yet remains to be investigated how osmo-bts-trx and other osmo-bts
ports behave in this regard.
Chapter 3: Conclusions
Most definitely it is once again a very clear sign that more testing is
required. It's tricky to see even wih osmo-gsm-tester, as GSM-HR
works between two phones or even two instances of osmo-bts-sysmo, as
both sides of the implementation have the same (wrong) understanding of
the spec.
Given that we can only catch this kind of bug together with the hardware
(the DSP runs the PHY code), pure unit tests wouldn't catch it. And the
end-to-end test is also not very well suited to it. It seems to call
for something in betewen. Something like an A-bis interface level test.
We need more (automatic) testing. I cannot say that often enough. The
big challenge is how to convince contributors and customers that they
should invest their time and money there, rather than
yet-another (not automatically tested) feature?
Every so often I happen to be involved in designing electronics
equipment that's supposed to run reliably remotely in inaccessible
locations,without any ability for "remote hands" to perform things like
power-cycling or the like. I'm talking about really remote locations,
possible with no but limited back-haul, and a very high cost of ever
sending somebody there for remote maintenance.
Given that a lot of computer peripherals (chips, modules, ...) use USB
these days, this is often some kind of an embedded ARM (rarely x86) SoM
or SBC, which is hooked up to a custom board that contains a USB hub
chip as well as a line of peripherals.
One of the most important lectures I've learned from experience is:
Never trust reset signals / lines, always include power-switching
capability. There are many chips and electronics modules available on
the market that have either no RESET, or even might claim to have a
hardware RESET line which you later (painfully) discover just to be a
GPIO polled by software which can get stuck, and hence no way to really
hard-reset the given component.
In the case of a USB-attached device (even though the USB
might only exist on a circuit board between two ICs), this is typically
rather easy: The USB hub is generally capable of switching the power of
its downstream ports. Many cheap USB hubs don't implement this at all,
or implement only ganged switching, but if you carefully select your USB
hub (or in the case of a custom PCB), you can make sure that the given
USB hub supports individual port power switching.
Now the next step is how to actually use this from your (embedded) Linux
system. It turns out to be harder than expected. After all, we're
talking about a standard feature that's present in the USB
specifications since USB 1.x in the late 1990ies. So the expectation is
that it should be straight-forward to do with any decent operating
system.
I don't know how it's on other operating systems, but on Linux I
couldn't really find a proper way how to do this in a clean way. For
more details, please read my post to the linux-usb mailing list.
Why am I running into this now? Is it such a strange idea? I mean,
power-cycling a device should be the most simple and straight-forward
thing to do in order to recover from any kind of "stuck state" or other
related issue. Logical enabling/disabling of the port, resetting the
USB device via USB protocol, etc. are all just "soft" forms of a reset
which at best help with USB related issues, but not with any other part
of a USB device.
And in the case of e.g. an USB-attached cellular modem, we're actually
talking about a multi-processor system with multiple built-in
micro-controllers, at least one DSP, an ARM core that might run another
Linux itself (to implement the USB gadget), ... - certainly enough
complex software that you would want to be able to power-cycle it...
I'm curious what the response of the Linux USB gurus is.
I've always been extremely skeptical of suddenly emerging over-hyped
technologies, particularly if they advertise to solve problems by adding
yet another layer to systems that are already sufficiently complex
themselves.
There are of course many issues with containers, ranging from replicated
system libraries and the basic underlying statement that you're giving
up on the system packet manager to properly deal with dependencies.
I'm also highly skeptical of FOSS projects that are primarily driven by
one (VC funded?) company. Especially if their offering includes a
so-called cloud service which they can stop to operate at any given
point in time, or (more realistically) first get everybody to use and
then start charging for.
But well, despite all the bad things I read about it over the years, on
one day in May 2017 I finally thought let's give it a try. My problem
to solve as a test balloon is fairly simple.
My basic use case
The plan is to start OsmoSTP, the m3ua-testtool and the sua-testtool,
which both connect to OsmoSTP. By running this setup inside containers
and inside an internal network, we could then execute the entire
testsuite e.g. during jenkins test without having IP address or port
number conflicts. It could even run multiple times in parallel on one
buildhost, verifying different patches as part of the continuous
integration setup.
This application is not so complex. All it needs is three containers,
an internal network and some connections in between. Should be a piece
of cake, right?
But enter the world of buzzword-fueled web-4000.0 software-defined
virtualised and orchestrated container NFW + SDN vodoo: It turns out to
be impossible, at least not with the preferred tools they advertise.
Dockerfiles
The part that worked relatively easily was writing a few Dockerfiles to
build the actual containers. All based on debian:jessie from the
library.
As m3ua-testsuite is written in guile, and needs to build some guile
plugin/extension, I had to actually include guile-2.0-dev and other
packages in the container, making it a bit bloated.
I couldn't immediately find a nice example Dockerfile recipe that would
allow me to build stuff from source outside of the container, and then
install the resulting binaries into the container. This seems to be a
somewhat weak spot, where more support/infrastructure would be helpful.
I guess the idea is that you simply install applications via package
feeds and apt-get. But I digress.
So after some tinkering, I ended up with three docker containers:
one running OsmoSTP
one running m3ua-testtool
one running sua-testtool
I also managed to create an internal bridged network between the
containers, so the containers could talk to one another.
However, I have to manually start each of the containers with ugly long
command line arguments, such as docker run --network sigtran --ip
172.18.0.200 -it osmo-stp-master. This is of course sub-optimal, and
what Docker Services + Stacks should resolve.
Services + Stacks
The idea seems good: A service defines how a given container is run,
and a stack defines multiple containers and their relation to each
other. So it should be simple to define a stack with three
services, right?
Well, it turns out that it is not. Docker documents that you can
configure a static ipv4_address[1] for each service/container, but it
seems related configuration statements are simply silently
ignored/discarded [2], [3], [4].
This seems to be related that for some strange reason stacks can (at
least in later versions of docker) only use overlay type networks,
rather than the much simpler bridge networks. And while bridge
networks appear to support static IP address allocations, overlay
apparently doesn't.
I still have a hard time grasping that something that considers itself a
serious product for production use (by a company with estimated value
over a billion USD, not by a few hobbyists) that has no support for
running containers on static IP addresses. that. How many applications
out there have I seen that require static IP address configuration? How
much simpler do setups get, if you don't have to rely on things like
dynamic DNS updates (or DNS availability at all)?
So I'm stuck with having to manually configure the network between my
containers, and manually starting them by clumsy shell scripts, rather
than having a proper abstraction for all of that. Well done :/
Exposing Ports
Unrelated to all of the above: If you run some software inside
containers, you will pretty soon want to expose some network services
from containers. This should also be the most basic task on the planet.
However, it seems that the creators of docker live in the early 1980ies,
where only TCP and UDP transport protocols existed. They seem to have
missed that by the late 1990ies to early 2000s, protocols like SCTP or DCCP were invented.
Now some of the readers may think 'who uses SCTP anyway'. I will give
you a straight answer: Everyone who has a mobile phone uses SCTP. This
is due to the fact that pretty much all the connections inside cellular
networks (at least for 3G/4G networks, and in reality also for many 2G
networks) are using SCTP as underlying transport protocol, from the
radio access network into the core network. So every time you switch
your phone on, or do anything with it, you are using SCTP. Not on your
phone itself, but by all the systems that form the network that you're
using. And with the drive to C-RAN, NFV, SDN and all the other
buzzwords also appearing in the Cellular Telecom field, people should
actually worry about it, if they want to be a part of the software stack
that is used in future cellular telecom systems.
Summary
After spending the better part of a day to do something that seemed like
the most basic use case for running three networked containers using
Docker, I'm back to step one: Most likely inventing some custom
scripts based on unshare to run my three
test programs in a separate network namespace for isolated execution of
test suite execution as part of a Jenkins CI setup :/
It's also clear that Docker apparently don't care much about playing a
role in the Cellular Telecom world, which is increasingly moving away
from proprietary and hardware-based systems (like STPs) to virtualised,
software-based systems.
This is a much smaller group, typically about 20 people, and is limited
to actual developers who have a past record of contributing to any of
the many Osmocom projects.
We had a large number of presentation and discussions. In fact, so
large that the schedule of talks extended from 10am to midnight on some
days. While this is great, it also means that there was definitely too
little time for more informal conversations, chatting or even actual
work on code.
We also have such a wide range of topics and scope inside Osmocom, that
the traditional ad-hoch scheduling approach no longer seems to be
working as it used to. Not everyone is interested in (or has time for)
all the topics, so we should group them according to their topic/subject
on a given day or half-day. This will enable people to attend only
those days that are relevant to them, and spend the remaining day in an
adjacent room hacking away on code.
It's sad that we only have OsmoDevCon once per year. Maybe that's
actually also something to think about. Rather than having 4 days once
per year, maybe have two weekends per year.
My former gpl-violations.org colleague Armijn Hemel and Shane Coughlan
(former coordinator of the FSFE Legal Network) have written a book on
practical GPL compliance issues.
I've read through it (in the bath tub of course, what better place to
read technical literature), and I can agree wholeheartedly with its
contents. For those who have been involved in GPL compliance
engineering there shouldn't be much new - but for the vast majority of
developers out there who have had little exposure to the
bread-and-butter work of providing complete an corresponding source
code, it makes an excellent introductory text.
The book focuses on compliance with GPLv2, which is probably not too
surprising given that it's published by the Linux foundation, and Linux
being GPLv2.
Given the subject matter is Free Software, and the book is written by
long-time community members, I cannot help to notice a bit of a surprise
about the fact that the book is released in classic copyright under All
rights reserved with no freedom to the user.
Considering the sensitive legal topics touched, I can understand the
possible motivation by the authors to not permit derivative works. But
then, there still are licenses such as CC-BY-ND which prevent derivative
works but still permit users to make and distribute copies of the work
itself. I've made that recommendation / request to Shane, let's see
if they can arrange for some more freedom for their readers.
It's already one week past the event, so I really have to sit down and
write some rewview on the first public Osmocom Conference ever:
OsmoCon 2017.
The event was a huge success, by all accounts.
We've not only been sold out, but we also had to turn down some last
minute registrations due to the venue being beyond capacity (60
seats). People traveled from Japan, India, the US, Mexico and many
other places to attend.
We've had an amazing audience ranging from commercial operators to
community cellular operators to professional developers doing work
relate to osmocom, academia, IT security crowds and last but not least
enthusiasts/hobbyists, with whom the project[s] started.
I've received exclusively positive feedback from many attendees
We've had a great programme. Some part of it was of introductory
nature and probably not too interesting if you've been in Osmocom for
a few years. However, the work on 3G as well as the current roadmap
was probably not as widely known yet. Also, I really loved to see
Roch's talk about Running a commercial cellular network with Osmocom
software
as well as the talk on Facebook's OpenCellular BTS hardware
and the Community Cellular Manager.
We have very professional live streaming + video recordings courtesy
of the C3VOC team. Thanks a lot for your
support and for having the video recordings of all talks online already at
the next day after the event.
We also received some requests for improvements, many of which we will
hopefully consider before the next Osmocom Conference:
have a multiple day event. Particularly if you're traveling
long-distance, it is a lot of overhead for a single-day event. We of
course fully understand that. On the other hand, it was the first
Osmocom Conference, and hence it was a test balloon where it was
initially unclear if we'll be able to get a reasonable number of
attendees interested at all, or not. And organizing an event with
venue and talks for multiple days if in the end only 10 people attend
would have been a lot of effort and financial risk. But now that we
know there are interested folks, we can definitely think of a multiple
day event next time
Signs indicating venue details on the last meters. I agree, this cold
have been better. The address of the venue was published, but we
could have had some signs/posters at the door pointing you to the
right meeting room inside the venue. Sorry for that.
Better internet connectivity. This is a double-edged sword. Of
course we want our audience to be primarily focused on the talks and
not distracted :P I would hope that most people are able to survive
a one day event without good connectivity, but for sure we will have
to improve in case of a multiple-day event in the future
In terms of my requests to the attendees, I only have one
Participate in the discussions on the schedule/programme while it is
still possible to influence it. When we started to put together the
programme, I posted about it on the openbsc mailing list and invited
feedback. Still, most people seem to have missed the time window
during which talks could have been submitted and the schedule still
influenced before finalizing it
Register in time. We have had almost no registrations until about two
weeks ahead of the event (and I was considering to cancel it), and
then suddenly were sold out in the week ahead of the event. We've had
people who first booked their tickets, only to learn that the tickets
were sold out. I guess we will introduce early bird pricing and add
a very expensive last minute ticket option next year in order to
increase motivation to register early and thus give us flexibility
regarding venue planning.
Thanks again to everyone involved in OsmoCon 2017!
When I was still doing Linux kernel work with netfilter/iptables in the
early 2000's, I was somebody who actually regularly had a look at the
new RFCs that came out. So I saw the SCTP RFCs, SIGTRAN RFCs, SIP and
RTP, etc. all released during those years. I was quite happy to see
that for new protocols like SCTP and later DCCP, Linux quickly received
a mainline implementation.
Now most people won't have used SCTP so far, but it is a protocol used
as transport layer in a lot of telecom protocols for more than a decade
now. Virtually all protocols that have traditionally been spoken over
time-division multiplex E1/T1 links have been migrated over to SCTP
based protocol stackings.
Working on various Open Source telecom related projects, i of course
come into contact with SCTP every so often. Particularly some years
back when implementing the Erlang SIGTAN code in erlang/osmo_ss7 and most recently
now with the introduction of libosmo-sigtran with its OsmoSTP, both part
of the libosmo-sccp repository.
I've also hard to work with various proprietary telecom equipment over
the years. Whether that's some eNodeB hardware from a large brand
telecom supplier, or whether it's a MSC of some other vendor. And they
all had one thing in common: Nobody seemed to use the Linux kernel SCTP
code. They all used proprietary implementations in userspace, using RAW
sockets on the kernel interface.
I always found this quite odd, knowing that this is the route that you
have to take on proprietary OSs without native SCTP support, such as
Windows. But on Linux? Why? Based on rumors, people find the Linux
SCTP implementation not mature enough, but hard evidence is hard to come
by.
Sure, software always has bugs and will have bugs. But we at Osmocom
are 10-15 years "late" with our implementations of higher-layer
protocols compared to what the mainstream telecom industry does. So if
we find something, and we find it even already during R&D of some
userspace code, not even under load or in production, then that seems a
bit unsettling.
One would have expected, with all their market power and plenty of
Linux-based devices in the telecom sphere, why did none of those large
telecom suppliers invest in improving the mainline Linux SCTP code? I
mean, they all use UDP and TCP of the kernel, so it works for most of
the other network protocols in the kernel, but why not for SCTP? I
guess it comes back to the fundamental lack of understanding how open
source development works. That it is something that the given
industry/user base must invest in jointly.
For quite some time I was seeing some erratic behavior when at some
point the STP would not receive/process a given message sent by one of
the clients (ASPs) connected. I tried to ignore the problem initially
until the code matured more and more, but the problems remained.
It became even more obvious when using Michael Tuexen's m3ua-testtool,
where sometimes even the most basic test cases consisting of sending +
receiving a single pair of messages like ASPUP -> ASPUP_ACK was failing.
And when the test case was re-tried, the problem often disappeared.
Also, whenever I tried to observe what was happening by meas of strace,
the problem would disappear completely and never re-appear until strace
was detached.
Of course, given that I've written several thousands of lines of new
code, it was clear to me that the bug must be in my code. Yesterday I
was finally prepare to accept that it might actually be a Linux SCTP
bug. Not being able to reproduce that problem on a FreeBSD VM also
pointed clearly into this direction.
Now I could simply have collected some information and filed a bug
report (which some kernel hackers at RedHat have thankfully invited me
to do!), but I thought my use case was too complex. You would have to
compile a dozen of different Osmocom libraries, configure the STP, run
the scheme-language m3ua-testtool in guile, etc. - I guess nobody
would have bothered to go that far.
So today I tried to implement a test case that reproduced the problem in
plain C, without any external dependencies. And for many hours, I
couldn't make the bug to show up. I tried to be as close as possible to
what was happening in OsmoSTP: I used non-blocking mode on client and
server, used the SCTP_NODELAY socket option, used the sctp_rcvmsg()
library wrapper to receive events, but the bug was not reproducible.
Some hours later, it became clear that there was one setsockopt() in
OsmoSTP (actually, libosmo-netif) which enabled all existing SCTP
events. I did this at the time to make sure OsmoSTP has the maximum
insight possible into what's happening on the SCTP transport layer, such
as address fail-overs and the like.
As it turned out, adding that setsockopt for SCTP_FLAGS to my test code
made the problem reproducible. After playing around which of the flags,
it seems that enabling the SENDER_DRY_EVENT flag makes the bug appear.
Inside the Osmocom world, luckily we can live without the
SENDER_DRY_EVENT and a corresponding work-around has been submitted and
merged as https://gerrit.osmocom.org/#/c/2386/
With that work-around in place, suddenly all the m3ua-testtool and sua-testtool test cases are reliably green
(PASSED) and OsmoSTP works more smoothly, too.
What do we learn from this?
Free Software in the Telecom sphere is getting too little attention.
This is true even those small portions of telecom relevant protocols
that ended up in the kernel like SCTP or more recently the GTP module I
co-authored. They are getting too little attention in development, even
more lack of attention in maintenance, and people seem to focus more on
not using it, rather than fixing and maintaining what is there.
It makes me really sad to see this. Telecoms is such a massive
industry, with billions upon billions of revenue for the classic telecom
equipment vendors. Surely, they would be able to co-invest in some
basic infrastructure like proper and reliable testing / continuous
integration for SCTP. More recently, we see millions and more millions
of VC cash burned by buzzword-flinging companies doing "NFV" and
"SDN". But then rather reimplement network stacks in userspace than to
fix, complete and test those little telecom infrastructure components
which we have so far, like the SCTP protocol :(
Where are the contributions to open source telecom parts from Ericsson,
Nokia (former NSN), Huawei and the like? I'm not even dreaming about
the actual applications / network elements, but merely the maintenance
of something as basic as SCTP. To be fair, Motorola was involved early
on in the Linux SCTP code, and Huawei contributed a long series of fixes
in 2013/2014. But that's not the kind of long-term maintenance
contribution that one would normally expect from the primary interest
group in SCTP.
Finally, let me thank to the Linux SCTP maintainers. I'm not
complaining about them! They're doing a great job, given the arcane code
base and the fact that they are not working for a company that has
SCTP based products as their core business. I'm sure the would love
more support and contributions from the Telecom world, too.
As I blogged in my blog post in Fabruary, I was working towards a more
fully-featured SIGTRAN stack in the Osmocom (C-language) universe.
The trigger for this is the support of 3GPP compliant AoIP (with a
BSSAP/SCCP/M3UA/SCTP protocol stacking), but it is of much more general
nature.
The code has finally matured in my development branch(es) and is now
ready for mainline inclusion. It's a series of about 77 (!) patches,
some of which already are the squashed results of many more incremental
development steps.
The result is as follows:
General SS7 core functions maintaining links, linksets and routes
xUA functionality for the various User Adaptations (currently SUA and M3UA supported)
MTP User SAP according to ITU-T Q.701 (using osmo_prim)
management of application servers (AS)
management of application server processes (ASP)
ASP-SM and ASP-TM state machine for ASP, AS-State Machine (using osmo_fsm)
server (SG) and client (ASP) side implementation
validated against ETSI TS 102 381 (by means of Michael Tuexen's m3ua-testtool)
support for dynamic registration via RKM (routing key management)
osmo-stp binary that can be used as Signal Transfer Point, with
the usual "Cisco-style" command-line interface that all Osmocom
telecom software has.
SCCP implementation, with strong focus on Connection Oriented SCCP (as
that's what the A interface uses).
osmo_fsm based state machine for SCCP connection, both incoming and
outgoing
SCCP User SAP according to ITU-T Q.711 (osmo_prim based)
Interfaces with underlying SS7 stack via MTP User SAP (osmo_prim based)
Support for SCCP Class 0 (unit data) and Class 2 (connection oriented)
All SCCP + SUA Address formats (Global Title, SSN, PC, IPv4 Address)
SCCP and SUA share one implementation, where SCCP messages are
transcoded into SUA before processing, and re-encoded into SCCP after
processing, as needed.
I have already done experimental OsmoMSC and OsmoHNB-GW over to libosmo-sigtran.
They're now all just M3UA clients (ASPs) which connect to osmo-stp
to exchange SCCP messages back and for the between them.
What's next on the agenda is to
finish my incomplete hacks to introduce IPA/SCCPlite as an alternative
to SUA and M3UA (for backwards compatibility)
port over OsmoBSC to the SCCP User SAP of libosmo-sigtran
validate with SSCPlite lower layer against existing SCCPlite MSCs
implement BSSAP / A-interface procedures in OsmoMSC, on top of the
SCCP-User SAP.
If those steps are complete, we will have a single OsmoMSC that can talk
both IuCS to the HNB-GW (or RNCs) for 3G/3.5G as well as AoIP towards
OsmoBSC. We will then have fully SIGTRAN-enabled the full Osmocom
stack, and are all on track to bury the OsmoNITB that was devoid of such
interfaces.
If any reader is interested in interoperability testing with other
implementations, either on M3UA or on SCCP or even on A or Iu interface
level, please contact me by e-mail.
April 21st is approaching fast, so here some updates. I'm particularly
happy that we now have travel grants available. So if the travel
expenses were preventing you from attending so far: This excuse is no
longer valid!
Get your ticket now, before it is too late. There's a limited number of
seats available.
OsmoCon 2017 Schedule
The list of talks
for OsmoCon 2017 has been
available for quite some weeks, but today we finally published the first
actual schedule.
As you can see, the day is fully packed with talks about Osmocom
cellular infrastructure projects. We had to cut some talk slots short
(30min instead of 45min), but I'm confident that it is good to cover a
wider range of topics, while at the same time avoiding fragmenting the
audience with multiple tracks.
OsmoCon 2017 Travel Grants
We are happy to announce that we have received donations to permit for
providing travel grants!
This means that any attendee who is otherwise not able to cover their
travel to OsmoCon 2017 (e.g. because their interest in Osmocom is not
related to their work, or because their employer doesn't pay the travel
expenses) can now apply for such a travel grant.
Tech Talks are nice and fine, but what many people enjoy even more at
conferences is the informal networking combined with good food. For
this, we have the social event at night, which is open to all attendees.
Instead of ordering more of the old (v2) design, I decided to do some
improvements in the next version:
add mounting holes so the PCB can be mounted via M3 screws
add U.FL and SMA sockets, so the modems are connected via a short U.FL
to U.FL cable, and external antennas or other RF components can be
attached via SMA. This provides strain relief for the external
antenna or cabling and avoids tearing off any of the current loose
U.FL to SMA pigtails
flip the SIM slot to the top side of the PCB, so it can be accessed
even after mounting the board to some base plate or enclosure via the
mounting holes
more meaningful labeling of the silk screen, including the purpose of
the jumpers and the input voltage.
A software rendering of the resulting v3 PCB design files that I just
sent for production looks like this:
Like before, the design of the board (including schematics and PCB
layout design files) is available as open hardware under CC-BY-SA
license terms. For more information see
http://osmocom.org/projects/mpcie-breakout/wiki
It will take some expected three weeks until I'll see the first
assembled boards.
I'm also planning to do a M.2 / NGFF version of it, but haven't found
the time to get around doing it so far.
As I just wrote in my post about TelcoSecDay, I sometimes
worry about the choices I made with Osmocom, particularly when I see
all the great stuff people doing in fields that I previously was working
in, such as applied IT security as well as Linux Kernel development.
History
When people like Dieter, Holger and I started to play with what later
became OpenBSC, it was just for fun. A challenge to master. A closed
world to break open and which to attack with the tools, the mindset and
the values that we brought with us.
Later, Holger and I started to do freelance development for commercial
users of Osmocom (initially basically only OpenBSC, but then OsmoSGSN,
OsmoBSC, OsmoBTS, OsmoPCU and all the other bits on the infrastructure
side). This lead to the creation of sysmocom in 2011, and ever since we
are trying to use revenue from hardware sales as well as development
contracts to subsidize and grow the Osmocom projects. We're investing
most of our earnings directly into more staff that in turn works on
Osmocom related projects.
NOTE
It's important to draw the distinction betewen the Osmocom cellular
infrastructure projects
which are mostly driven by commercial users and sysmocom these days,
and all the many other pure juts-for-fun community projects under
the Osmocom umbrella, like OsmocomTETRA, OsmocomGMR, rtl-sdr, etc.
I'm focussing only on the cellular infrastructure projects, as they
are in the center of my life during the past 6+ years.
In order to do this, I basically gave up my previous career[s] in IT
security and Linux kernel development (as well as put things like
gpl-violations.org on hold). This is a big price to pay for crating
more FOSS in the mobile communications world, and sometimes I'm a bit
melancholic about the "old days" before.
Financial wealth is clearly not my primary motivation, but let me be
honest: I could have easily earned a shitload of money continuing to do
freelance Linux kernel development, IT security or related consulting.
There's a lot of demand for related skills, particularly with some
experience and reputation attached. But I decided against it, and
worked several years without a salary (or almost none) on Osmocom
related stuff [as did Holger].
But then, even with all the sacrifices made, and the amount of revenue
we can direct from sysmocom into Osmocom development: The complexity of
cellular infrastructure vs. the amount of funding and resources is always
only a fraction of what one would normally want to have to do a proper
implementation. So it's constant resource shortage, combined with lots
of unpaid work on those areas that are on the immediate short-term
feature list of customers, and that nobody else in the community feels
like he wants to work on. And that can be a bit frustrating at times.
Is it worth it?
So after 7 years of OpenBSC, OsmocomBB and all the related projects, I'm
sometimes asking myself whether it has been worth the effort, and
whether it was the right choice.
It was right from the point that cellular technology is still an area
that's obscure and unknown to many, and that has very little FOSS
(though Improving!). At the same time, cellular networks are becoming
more and more essential to many users and applications. So on an
abstract level, I think that every step in the direction of FOSS for
cellular is as urgently needed as before, and we have had quite some
success in implementing many different protocols and network elements.
Unfortunately, in most cases incompletely, as the amount of funding
and/or resources were always extremely limited.
Satisfaction/Happiness
On the other hand, when it comes to metrics such as personal
satisfaction or professional pride, I'm not very happy or satisfied.
The community remains small, the commercial interest remains limited,
and as opposed to the Linux world, most players have a complete lack of
understanding that FOSS is not a one-way road, but that it is important
for all stakeholders to contribute to the development in terms of
development resources.
Project success?
I think a collaborative development project (which to me is what FOSS is
about) is only then truly successful, if its success is not related to
a single individual, a single small group of individuals or a single
entity (company). And no matter how much I would like the above to be
the case, it is not true for the Osmocom cellular infrastructure
projects. Take away Holger and me, or take away sysmocom, and I think
it would be pretty much dead. And I don't think I'm exaggerating here.
This makes me sad, and after all these years, and after knowing quite a
number of commercial players using our software, I would have hoped that
the project rests on many more shoulders by now.
This is not to belittle the efforts of all the people contributing to
it, whether the team of developers at sysmocom, whether those in the
community that still work on it 'just for fun', or whether those
commercial users that contract sysmocom for some of the work we do.
Also, there are known and unknown donors/funders, like the NLnet
foundation for some parts of the work. Thanks to all of you, and
clearly we wouldn't be where we are now without all of that!
But I feel it's not sufficient for the overall scope, and it's not [yet]
sustainable at this point. We need more support from all sides,
particularly those not currently contributing. From vendors of BTSs and
related equipment that use Osmocom components. From operators that use
it. From individuals. From academia.
Yes, we're making progress. I'm happy about new developments like the
Iu and Iuh support, the OsmoHLR/VLR split and 2G/3G authentication that Neels just blogged about. And
there's progress on the SIMtrace2 firmware with card emulation and MITM,
just as well as there's progress on libosmo-sigtran (with a more
complete SUA, M3UA and connection-oriented SCCP stack), etc.
But there are too little people working on this, and those people are
mostly coming from one particular corner, while most of the [commercial]
users do not contribute the way you would expect them to contribute in
collaborative FOSS projects. You can argue that most people in the
Linux world also don't contribute, but then the large commercial
beneficiaries (like the chipset and hardware makers) mostly do, as are
the large commercial users.
All in all, I have the feeling that Osmocom is as important as it
ever was, but it's not grown up yet to really walk on its own feet. It
may be able to crawl, though ;)
So for now, don't panic. I'm not suffering from burn-out, mid-life
crisis and I don't plan on any big changes of where I put my energy: It
will continue to be Osmocom. But I also think we have to have a more
open discussion with everyone on how to move beyond the current
situation. There's no point in staying quiet about it, or to claim that
everything is fine the way it is. We need more commitment. Not from
the people already actively involved, but from those who are not [yet].
If that doesn't happen in the next let's say 1-2 years, I think it's
fair that I might seriously re-consider in which field and in which way
I'd like to dedicate my [I would think considerable] productive energy and
focus.
I'm just on my way back from the Telecom Security Day 2017
<https://www.troopers.de/troopers17/telco-sec-day/>, which is an
invitation-only event about telecom security issues hosted by ERNW
back-to-back with their Troopers 2017 <https://www.troopers.de/troopers17/>
conference.
I've been presenting at TelcoSecDay in previous years and hence was
again invited to join (as attendee). The event has really gained quite
some traction. Where early on you could find lots of IT security /
hacker crowds, the number of participants from the operator (and to
smaller extent also equipment maker) industry has been growing.
The quality of talks was great, and I enjoyed meeting various familiar
faces. It's just a pity that it's only a single day - plus I had to
head back to Berlin still today so I had to skip the dinner + social
event.
When attending events like this, and seeing the interesting hacks that
people are working on, it pains me a bit that I haven't really been
doing much security work in recent years. netfilter/iptables was at
least somewhat security related. My work on OpenPCD / librfid was
clearly RFID security oriented, as was the work on airprobe,
OsmocomTETRA, or even the EasyCard payment system hack
I have the same feeling when attending Linux kernel development related
events. I have very fond memories of working in both fields, and it was
a lot of fun. Also, to be honest, I believe that the work in Linux
kernel land and the general IT security research was/is appreciated much
more than the endless months and years I'm now spending my time with
improving and extending the Osmocom cellular infrastructure stack.
Beyond the appreciation, it's also the fact that both the IT security
and the Linux kernel communities are much larger. There are more
people to learn from and learn with, to engage in discussions and
ping-pong ideas. In Osmocom, the community is too small (and I have the
feeling, it's actually shrinking), and in many areas it rather seems
like I am the "ultimate resource" to ask, whether about 3GPP specs or
about Osmocom code structure. What I'm missing is the feeling of being
part of a bigger community. So in essence, my current role in the "Open
Source Cellular" corner can be a very lonely one.
But hey, I don't want to sound more depressed than I am, this was
supposed to be a post about TelcoSecDay. It just happens that attending
IT Security and/or Linux Kernel events makes me somewhat gloomy for the
above-mentioned reasons.
Meanwhile, if you have some interesting projcets/ideas at the border
between cellular protocols/systems and security, I'd of course love to
hear if there's some way to get my hands dirty in that area again :)
One thing to keep in mind: The Linux Foundation is an industry
association, it exists to act in the joint interest of it's paying
members. It is not a charity, and it does not act for the public good.
I know and respect that, while some people sometimes appear to be
confused about its function.
I wouldn't have any issue if VMware would (prior to joining LF) have
said: Ok, we had some bad policies in the past, but now we fully comply
with the license of the Linux kernel, and we release all
derivative/collective works in source code. This would be a positive
spin: Acknowledge past issues, resolve the issues, become clean and then
publicly underlining your support of Linux by (among other things)
joining the Linux Foundation. I'm not one to hold grudges against
people who accept their past mistakes, fix the presence and then move
on. But no, they haven't fixed any issues.
They are having one of the worst track records in terms of intentional
GPL compliance issues for many years, showing outright disrespect for Linux,
the GPL and ultimately the rights of the Linux developers, not resolving
those issues and at the same time joining the Linux Foundation? What
kind of message sends that?
It sends the following messages:
you can abuse Linux, the GPL and copyleft while still being accepted
amidst the Linux Foundation Members
it means the Linux Foundations has no ethical concerns whatsoever
about accepting such entities without previously asking them to become
clean
it also means that VMware has still not understood that Linux and FOSS
is about your actions, particularly the kind of choices you make how
to technically work with the community, and not against it.
So all in all, I think this move has seriously damaged the image of both
entities involved. I wouldn't have expected different of VMware, but I
would have hoped the Linux Foundation had some form of standards as to
which entities they permit amongst their ranks. I guess I was being
overly naive :(
It's a slap in the face of every developer who writes code not because
he gets paid, but because it is rewarding to know that copyleft will
continue to ensure the freedom of related code.
UPDATE (March 8, 2017):
I was mistaken in my original post in that VMware didn't just join,
but was a Linux Foundation member already before, it is "just" their
upgrade from silver to gold that made the news recently. I stand
corrected. Still doesn't make it any better that the are involved
inside LF while engaging in stepping over the lines of license
compliance.
UPDATE2 (March 8, 2017):
As some people pointed out, there is no verdict against VMware. Yes,
that's true. But the mere fact that they rather distribute derivative
works of GPL licensed software and take this to court with an armada
of lawyers (instead of simply complying with the license like everyone
else) is sad enough. By the time there will be a final verdict, the
product is EOL. That's probably their strategy to begin with :/
I always though I understood UMTS AKA (authentication and key
agreement), including the re-synchronization procedure. It's been years
since I wrote tools like osmo-sim-auth which you can use to
perform UMTS AKA with a SIM card inserted into a PC reader, i.e.
simulate what happens between the AUC (authentication center) in a
network and the USIM card.
However, it is only now as the sysmocom team works on 3G support of the
dedicated OsmoHLR (outside of
OsmoNITB!), that I seem to understand all the nasty little details.
I always thought for re-synchronization it is sufficient to simply
increment the SQN (sequence number). It turns out, it isn't as there is
a MSB-portion called SEQ and a lower-bit portion called IND, used for
some fancy array indexing scheme of buckets of highest-used-SEQ within
that IND bucket.
If you're interested in all the dirty details and associated spec
references (the always hide the important parts in some Annex) see the
discussion between Neels and me in Osmocom redmine issue 1965.
For those of you who don't know what the tinkerphones/OpenPhoenux GTA04 is: It is a
'professional hobbyist' hardware project (with at least public
schematics, even if not open hardware in the sense that editable
schematics and PCB design files are published) creating updated
mainboards that can be used to upgrade Openmoko phones. They fit into
the same enclosure and can use the same display/speaker/microphone.
What the GTA04 guys have been doing for many years is close to a miracle
anyway: Trying to build a modern-day smartphone in low quantities,
using off-the-shelf components available in those low quantities, and
without a large company with its associated financial backing.
Smartphones are complex because they are highly integrated devices. A
seemingly unlimited amount of components is squeezed in the tiniest
form-factors. This leads to complex circuit boards with many layers
that take a lot of effort to design, and are expensive to build in low
quantities. The fine-pitch components mandated by the integration
density is another issue.
Building the original GTA01 (Neo1937) and GTA02 (FreeRunner) devices at
Openmoko, Inc. must seem like a piece of cake compared to what the GTA04
guys are up to. We had a team of engineers that were familiar at last
with feature phone design before, and we had the backing of a consumer
electronics company with all its manufacturing resources and expertise.
Nevertheless, a small group of people around Dr. Nikolaus Schaller has
been pushing the limits of what you can do in a small for fun
project, and the have my utmost respect. Well done!
Unfortunately, there are bad news. Manufacturing of their latest
generation of phones (GTA04A5) has been stopped due to massive soldering
problems with the TI OMAP3 package-on-package (PoP).
Those PoPs are basically "RAM chip soldered onto the CPU, and the stack
of both soldered to the PCB". This is used to save PCB footprint and to
avoid having to route tons of extra (sensitive, matched) traces between
the SDRAM and the CPU.
According to the mailing list posts, it seems to be incredibly difficult
to solder the PoP stack due to the way TI has designed the packaging of
the DM3730. If you want more gory details, see
this post
and yet another post.
It is very sad to see that what appears to be bad design choices at TI
are going to bring the GTA04 project to a halt. The financial hit by
having only 33% yield is already more than the small community can take,
let alone unused parts that are now in stock or even thinking about
further experiments related to the manufacturability of those chips.
If there's anyone with hands-on manufacturing experience on the DM3730
(or similar) TI PoP reading this: Please reach out to the GTA04 guys and
see if there's anything that can be done to help them.
UPDATE (March 8, 2017):
In an earlier post I was asserting that the GTA04 is open hardware
(which I actually believed up to that point) until some readers have
pointed out to me that it isn't. It's sad it isn't, but still it has
my sympathies.
During the second half of 2016, the code basically stayed untouched. In
early 2017, several patch series of (at least) three authors have been
published on the netdev mailing list for review and merge.
This poses the very valid question on how do we test those (sometimes
quite intrusive) changes. Setting up a complete cellular network with
either GPRS/EGPRS or even UMTS/HSPA is possible using OsmoSGSN and
related Osmocom components. But it's of course a luxury that not many
Linux kernel networking hackers have, as it involves the availability of
a supported GSM BTS or UMTS hNodeB. And even if that is available,
there's still the issue of having a spectrum license, or a wired setup
with coaxial cable.
So as part of the recent discussions on netdev, I tested and described a
minimal test setup using libgtpnl, OpenGGSN and sgsnemu.
This setup will start a mobile station + SGSN emulator inside a Linux
network namespace, which talks GTP-C to OpenGGSN on the host, as well as
GTP-U to the Linux kernel GTP-U implementation.
This is of course just for manual testing, and for functional (not
performance) testing only. It would be great if somebody would pick up
on my recent mail containing some suggestions about an automatic
regression testing setup for the kernel GTP-U code. I have way
too many spare-time projects in desperate need of some attention to work
on this myself. And unfortunately, none of the telecom operators (who
are the ones benefiting most from a Free Software accelerated GTP-U
implementation) seems to be interested in at least co-funding or
otherwise contributing to this effort :/
I've recently attended a seminar that (among other topics) also covered
RF interference hunting. The speaker was talking about various
real-world cases of RF interference and illustrating them in detail.
Of course everyone who has any interest in RF or cellular will know
about fundamental issues of radio frequency interference. To the
biggest part, you have
cells of the same operator interfering with each other due to too
frequent frequency re-use, adjacent channel interference, etc.
cells of different operators interfering with each other due to
intermodulation products and the like
cells interfering with cable TV, terrestrial TV
DECT interfering with cells
cells or microwave links interfering with SAT-TV reception
all types of general EMC problems
But what the speaker of this seminar covered was actually a cellular
base-station being re-broadcast all over Europe via a commercial
satellite (!).
It is a well-known fact that most satellites in the sky are basically
just "bent pipes", i.e. they consist of a RF receiver on one frequency,
a mixer to shift the frequency, and a power amplifier. So basically
whatever is sent up on one frequency to the satellite gets
re-transmitted back down to earth on another frequency. This is abused
by "satellite hijacking" or "transponder hijacking" and has been covered
for decades in various publications.
Ok, but how does cellular relate to this? Well, apparently some people
are running VSAT terminals (bi-directional satellite terminals) with
improperly shielded or broken cables/connectors. In that case, the RF
emitted from a nearby cellular base station leaks into that cable, and
will get amplified + up-converted by the block up-converter of that VSAT
terminal.
The bent-pipe satellite subsequently picks this signal up and
re-transmits it all over its coverage area!
I've tried to find some public documents about this, an there's
surprisingly little public information about this phenomenon.
However, I could find a slide set from SES, presented at a
Satellite Interference Reduction Group: Identifying Rebroadcast (GSM)
It describes a surprisingly manual and low-tech approach at hunting down
the source of the interference by using an old nokia net-monitor phone
to display the MCC/MNC/LAC/CID of the cell. Even in 2011 there were
already open source projects such as airprobe that could have done the
job based on sampled IF data. And I'm not even starting to consider
proprietary tools.
It should be relatively simple to have a SDR that you can tune to a
given satellite transponder, and which then would look for any
GSM/UMTS/LTE carrier within its spectrum and dump their identities in a
fully automatic way.
But then, maybe it really doesn't happen all that often after all to
rectify such a development...
In the good old days ever since the late 1980ies - and a surprising
amount even still today - telecom signaling traffic is still carried
over circuit-switched SS7 with its TDM lines as physical layer, and not
an IP/Ethernet based transport.
When Holger first created OsmoBSC, the BSC-only version of OpenBSC some
7-8 years ago, he needed to implement a minimal subset of SCCP wrapped
in TCP called SCCP Lite. This was due to the simple fact that the MSC
to which it should operate implemented this non-standard protocol
stacking that was developed + deployed before the IETF SIGTRAN WG
specified M3UA or SUA came around. But even after those were specified
in 2004, the 3GPP didn't specify how to carry A over IP in a standard
way until the end of 2008, when a first A interface over IP study
was released.
As time passese, more modern MSCs of course still implement classic
circuit-switched SS7, but appear to have dropped SCCPlite in favor of
real AoIP as specified by 3GPP meanwhile. So it's time to add this to
the osmocom universe and OsmoBSC.
A couple of years ago (2010-2013) implemented both classic SS7
(MTP2/MTP3/SCCP) as well as SIGTRAN stackings (M2PA/M2UA/M3UA/SUA in
Erlang. The result has been used in some production deployments, but
only with a relatively limited feature set. Unfortunately, this code
has nto received any contributions in the time since, and I have to say
that as an open source community project, it has failed. Also, while
Erlang might be fine for core network equipment, running it on a BSC
really is overkill. Keep in miond that we often run OpenBSC on
really small ARM926EJS based embedded systems, much more resource
constrained than any single smartphone during the late decade.
In the meantime (2015/2016) we also implemented some minimal SUA support
for interfacing with UMTS femto/small cells via Iuh (see OsmoHNBGW).
So in order to proceed to implement the required
SCCP-over-M3UA-over-SCTP stacking, I originally thought well, take
Holgers old SCCP code, remove it from the IPA multiplex below, stack it
on top of a new M3UA codebase that is copied partially from SUA.
However, this falls short of the goals in several ways:
The application shouldn't care whether it runs on top of SUA or SCCP,
it should use a unified interface towards the SCCP Provider.
OsmoHNBGW and the SUA code already introduce such an interface baed on
the SCCP-User-SAP implemented using Osmocom primitives (osmo_prim).
However, the old OsmoBSC/SCCPlite code doesn't have such abstraction.
The code should be modular and reusable for other SIGTRAN stackings
as required in the future
So I found myself sketching out what needs to be done and I ended up
pretty much with a re-implementation of large parts. Not quite fun, but
definitely worth it.
The strategy is:
Implement the SCCP SCOC state machines for connection-oriented SCCP
(of which Iu and A interface are probably the only users) using
Osmcoom Finite State Machines (osmo_fsm).
Migrate the existing SUA code on top of that, maintaining the existing
osmo_prim based SCCP User SAP
Implement M3UA using the xUA ASP and AS FSMs as well as the general
xUA message encoder/decoder,
offering the MTP SAP toward SCCP
And then finally stack all those bits on top of each other, rendering a
fairly clean and modern implementation that can be used with the IuCS of
the virtually unmodified OsmmoHNBGW, OsmoCSCN and OsmoSGSN for testing.
Next steps in the direction of the AoIP are:
Implementation of the MTP-SAP based on the IPA transport
Binding the new SCCP code on top of that
Converting OsmoBSC code base to use the SCCP-User-SAP for its
signaling connection
From that point onwards, OsmoBSC doesn't care anymore whether it
transports the BSSAP/BSSMAP messages of the A interface over
SCCP/IPA/TCP/IP (SCCPlite) SCCP/M3UA/SCTP/IP (3GPP AoIP), or even
something like SUA/SCTP/IP.
However, the 3GPP AoIP specs (unlike SCCPlite) actually modify the
BSSAP/BSSMAP payload. Rather than using Circuit Identifier Codes and
then mapping the CICs to UDP ports based on some secret conventions,
they actually encapsulate the IP address and UDP port information for
the RTP streams. This is of course the cleaner and more flexible
approach, but it means we'll have to do some further changes inside the
actual BSC code to accommodate this.
When implementing any kind of communication protocol, one always dreams
of some existing test suite that one can simply run against the
implementation to check if it performs correct in at least those use
cases that matter to the given application.
Of course in the real world, there rarely are protocols where this is
true. If test specifications exist at all, they are often just very
abstract texts for human consumption that you as the reader should
implement yourself.
For some (by far not all) of the protocols found in cellular networks,
every so often I have seen some formal/abstract machine-parseable test
specifications. Sometimes it was TTCN-2, and sometimes TTCN-3.
If you haven't heard about TTCN-3, it is basically a way to create
functional tests in an abstract description (textual + graphical), and
then compile that into an actual executable tests suite that you can run
against the implementation under test.
However, when I last did some research into this several years ago, I
couldn't find any Free / Open Source tools to actually use those
formally specified test suites. This is not a big surprise, as even
much more fundamental tools for many telecom protocols are missing, such
as good/complete ASN.1 compilers, or even CSN.1 compilers.
To my big surprise I now discovered that Ericsson had released their
(formerly internal) TITAN TTCN3 Toolset
as Free / Open Source Software under EPL 1.0. The project is even part
of the Eclipse Foundation. Now I'm certainly not a friend of Java or
Eclipse by all means, but well, for running tests I'd certainly not
complain.
I haven't yet had time to play with it, but it definitely is rather high
on my TODO list to try.
ETSI provides a couple of test suites in TTCN-3 for protocols like
DIAMETER, GTP2-C, DMR, IPv6, S1AP, LTE-NAS, 6LoWPAN, SIP, and others at
http://forge.etsi.org/websvn/ (It's also the first time I've seen that
ETSI has a SVN server. Everyone else is using git these days, but yes,
revision control systems rather than periodic ZIP files is definitely a
big progress. They should do that for their reference codecs and ASN.1
files, too.
I'm not sure once I'll get around to it. Sadly, there is no TTCN-3 for
SCCP, SUA, M3UA or any SIGTRAN related stuff, otherwise I would want to
try it right away. But it definitely seems like a very interesting
technology (and tool).
Last weekend I had the pleasure of attending FOSDEM 2017. For many years, it is probably the most
exciting event exclusively on Free Software to attend every year.
My personal highlights (next to meeting plenty of old and new friends)
in terms of the talks were:
I was attending but not so excited by Georg Greve's OpenPOWER talk. It was a
great talk, and it is an important topic, but the engineer in me would
have hoped for some actual beefy technical stuff. But well, I was just
not the right audience. I had heard about OpenPOWER quite some time ago
and have been following it from a distance.
For many years, we have had a small, invitation only event by Osmocom
developers for Osmocom developers called OsmoDevCon. This was fine for
the early years of Osmocom, but during the last few years it became
apparent that we also need a public event for our many users. Those
range from commercial cellular operators to community based efforts like
Rhizomatica, and of course include the many
research/lab type users with whom we started.
So now we'll have the public OsmoCon on April 21st, back-to-back with
the invitation-only OsmoDevcon from April 22nd through 23rd.
I'm hoping we can bring together a representative sample of our user
base at OsmoCon 2017 in April. Looking forward to meet you all. I hope
you're also curious to hear more from other users, and of course the
development team.
A few days ago, Autodesk has announecd
that the popular EAGLE electronics design automation (EDA) software is
moving to a subscription based model.
When previously you paid once for a license and could use that
version/license as long as you wanted, there now is a monthly
subscription fee. Once you stop paying, you loose the right to use the
software. Welcome to the brave new world.
I have remotely observed this subscription model as a general trend in
the proprietary software universe. So far it hasn't affected me at all,
as the only two proprietary applications I use on a regular basis
during the last decade are IDA and EAGLE.
I already have ethical issues with using non-free software, but those
two cases have been the exceptions, in order to get to the productivity
required by the job. While I can somehow convince my consciousness in
those two cases that it's OK - using software under a subscription model is
completely out of the question, period. Not only would I end up paying
for the rest of my professional career in order to be able to open and
maintain old design files, but I would also have to accept software that
"calls home" and has "remote kill" features. This is clearly not
something I would ever want to use on any of my computers. Also, I
don't want software to be associated with any account, and it's not the
bloody business of the software maker to know when and where I use my
software.
For me - and I hope for many, many other EAGLE users - this move is
utterly unacceptable and certainly marks the end of any business between
the EAGLE makers and myself and/or my companies. I will happily use
my current "old-style" EAGLE 7.x licenses for the near future, and theS
see what kind of improvements I would need to contribute to KiCAD or
other FOSS EDA software in order to eventually migrate to those.
As expected, this doesn't only upset me, but many other customers, some
of whom have been loyal to using EAGLE for many years if not decades,
back to the DOS version. This is reflected by some media reports (like
this one at hackaday
or user posts at element14.com or eaglecentral.ca
who are similarly critical of this move.
Rest in Peace, EAGLE. I hope Autodesk gets what they deserve: A new
influx of migrations away from EAGLE into the direction of Open Source
EDA software like KiCAD.
In fact, the more I think about it, I'm actually very much inclined to
work on good FOSS migration tools / converters - not only for my own
use, but to help more people move away from EAGLE. It's not that I
don't have enough projects at my hand already, but at least I'm
motivated to do something about this betrayal by Autodesk. Let's see
what (if any) will come out of this.
So let's see it that way: What Autodesk is doing is raising the level
off pain of using EAGLE so high that more people will use and contribute
FOSS EDA software. And that is actually a good thing!
I've just had the pleasure of attending all four days of 33C3 and have returned
home with somewhat mixed feelings.
I've been a regular visitor and speaker at CCC events since 15C3 in
1998, which among other things
means I'm an old man now. But I digress ;)
The event has come extremely far in those years. And to be honest, I
struggle with the size. Back then, it was a meeting of like-minded
hackers. You had the feeling that you know a significant portion of the
attendees, and it was easy to connect to fellow hackers.
These days, both the number of attendees and the size of the event make
you feel much rather that you're in general public, rather than at some
meeting of fellow hackers. Yes, it is good to see that more people are
interested in what the CCC (and the selected speakers) have to say, but
somehow it comes at the price that I (and I suspect other old-timers)
feel less at home. It feels too much like various other technology
related events.
One aspect creating a certain feeling of estrangement is also the venue
itself. There are an incredible number of rooms, with a labyrinth of
hallways, stairs, lobbies, etc. The size of the venue simply makes it
impossible to simply _accidentally_ running into all of your fellow
hackers and friends. If I want to meet somebody, I have to make an
explicit appointment. That is an option that exits most of the rest of
the year, too.
While fefe is happy about the many small children attending
the event, to me this seems
somewhat alien and possibly inappropriate. I guess from teenage years
onward it certainly makes sense, as they can follow the talks and
participate in the workshop. But below that age?
The range of topics covered at the event also becomes wider, at least I
feel that way. Topics like IT security, data protection, privacy,
intelligence/espionage and learning about technology have always been
present during all those years. But these days we have bloggers sitting
on stage and talking about bottles of wine (seriously?).
Contrary to many, I also really don't get the excitement about shows
like 'Methodisch Inkorrekt'. Seems to me like mainstream
compatible entertainment in the spirit of the 1990ies Knoff Hoff Show without much
potential to make the audience want to dig deeper into (information)
technology.
This presentation covers some of our recent explorations into a specific
type of 3G/4G cellular modems, which next to the regular modem/baseband
processor also contain a Cortex-A5 core that (unexpectedly) runs Linux.
We want to use such modems for building self-contained M2M devices that
run the entire application inside the modem itself, without any external
needs except electrical power, SIM card and antenna.
Next to that, they also pose an ideal platform for testing the Osmocom
network-side projects for running GSM, GPRS, EDGE, UMTS and HSPA
cellular networks.
You can find the Slides
and the Video recordings
in case you're interested in more details about our work.
The results of our reverse engineering can be found in the wiki at
http://osmocom.org/projects/quectel-modems/wiki together with links to
the various git repositories containing related tools.
As with all the many projects that I happen to end up doing, it would be
great to get more people contributing to them. If you're interested in
cellular technology and want to help out, feel free to register at the
osmocom.org site and start adding/updating/correcting information to the
wiki.
You can e.g. help by
playing with the modem and documenting your findings
reviewing the source code released by Qualcomm + Quectel and
documenting your findings
help us to create a working OE build with our own kernel and rootfs
images as well as opkg package feeds for the modems
help reverse engineering DIAG and QMI protocols as well as the open
source programs to interact with them
In 2016, Osmocom gained initial 3.5G support with osmo-iuh and the Iu
interface extensions of our libmsc and OsmoSGSN code. This means you can run
your own small open source 3.5G cellular network for SMS, Voice and Data
services.
However, the project needs more contributors: Become an active
member in the Osmocom development community and get your nano3G
femtocell for free.
I'm happy to announce that my company sysmocom hereby issues a call for
proposals to the general public. Please describe in a short proposal
how you would help us improving the Osmocom project if you were to
receive one of those free femtocells.
When you work with GSM/cellular systems, the definite resource are the
specifications. They were originally released by ETSI, later by 3GPP.
The problem start with the fact that there are separate numbering
schemes. Everyone in the cellular industry I know always uses the
GSM/3GPP TS numbering scheme, i.e. something like 3GPP TS 44.008.
However, ETSI assigns its own numbers to the specs, like ETSI TS
144008. Now in most cases, it is as simple s removing the '.' and
prefixing the '1' in the beginning. However, that's not always true and
there are exceptions such as 3GPP TS 01.01 mapping to ETSI TS
101855. To make things harder, there doesn't seem to be a
machine-readable translation table between the spec numbers, but there's
a website for spec number conversion at http://webapp.etsi.org/key/queryform.asp
When I started to work on GSM related topics somewhere between my work
at Openmoko and the start of the OpenBSC project, I manually downloaded
the PDF files of GSM specifications from the ETSI website. This was a
cumbersome process, as you had to enter the spec number (e.g. TS 04.08)
in a search window, look for the latest version in the search results,
click on that and then click again for accessing the PDF file (rather
than a proprietary Microsoft Word file).
At some point a poor girlfriend of mine was kind enough to do this
manual process for each and every 3GPP spec, and then create a
corresponding symbolic link so that you could type something like evince
/spae/openmoko/gsm-specs/by_chapter/44.008.pdf into your command line
and get instant access to the respective spec.
However, of course, this gets out of date over time, and by now almost a
decade has passed without a systematic update of that archive.
To the rescue, 3GPP started at some long time ago to not only provide
the obnoxious M$ Word DOC files, but have deep links to ETSI. So you
could go to http://www.3gpp.org/DynaReport/44-series.htm and then click
on 44.008, and one further click you had the desired PDF, served by
ETSI (3GPP apparently never provided PDF files).
However, in their infinite wisdom, at some point in 2016 the 3GPP
webmaster decided to remove those deep links. Rather than a nice long
list of released versions of a given spec,
http://www.3gpp.org/DynaReport/44008.htm now points to some crappy
JavaScript tabbed page, where you can click on the version number and
then get a ZIP file with a single Word DOC file inside. You can hardly
male it any more inconvenient and cumbersome. The PDF links would open
immediately in modern browsers built-in JavaScript PDF viewer or your
favorite PDF viewer. Single click to the information you want. But no,
the PDF links had to go and replaced with ZIP file downloads that you
first need to extract, and then open in something like LibreOffice,
taking ages to load the document, rendering it improperly in a word
processor. I don't want to edit the spec, I want to read it, sigh.
So since the usability of this 3GPP specification resource had been
artificially crippled, I was annoyed sufficiently well to come up with a
solution:
then use a shell script that utilizes pdfgrep and awk to determine the
3GPP specification number (it is written in the title on the first
page of the document) and creating a symlink. Now I have something
like 44.008-4.0.0.pdf -> ts_144008v040000p.pdf
It's such a waste of resources to have to download all those files and
then write a script using pdfgrep+awk to re-gain the same usability that
the 3GPP chose to remove from their website. Now we can wait for ETSI
to disable indexing/recursion on their server, and easy and quick spec
access would be gone forever :/
Many years ago, in the aftermath of Openmoko shutting down, fellow
former Linux kernel hacker Werner Almesberger
was working on an IEEE 802.15.4 (WPAN) adapter for the
Ben Nanonote.
As a spin-off to that, the ATUSB device was
designed: A general-purpose open hardware (and FOSS firmware + driver)
IEEE 802.15.4 adapter that can be plugged into any USB port.
This adapter has received a mainline Linux kernel driver written by
Werner Almesberger and Stefan Schmidt, which was eventually merged into
mainline Linux in May 2015 (kernel v4.2 and later).
Earlier in 2016, Stefan Schmidt (the current ATUSB Linux driver
maintainer) approached me about the situation that ATUSB hardware was
frequently asked for, but currently unavailable in its
physical/manufactured form. As we run a shop with smaller electronics
items for the wider Osmocom community at sysmocom, and we also
frequently deal with contract manufacturers for low-volume electronics
like the SIMtrace device anyway, it was easy to say "yes, we'll do it".
As a result, ready-built, programmed and tested ATUSB devices are now
finally available from the sysmocom webshop
Note: I was never involved with the development of the ATUSB hardware,
firmware or driver software at any point in time. All credits go to
Werner, Stefan and other contributors around ATUSB.
In a previous life I used to do a lot of IT security work, probably even
at a time when most people had no idea what IT security actually is. I
grew up with the Chaos Computer Club, as it was a great place to meet
people with common interests, skills and ethics. People were hacking
(aka 'doing security research') for fun, to grow their skills, to
advance society, to point out corporate stupidities and to raise
awareness about issues.
I've always shared any results worth noting with the general public.
Whether it was in RFID security, on GSM security, TETRA security, etc.
Even more so, I always shared the tools, creating free software
implementations of systems that - at that time - were very difficult to
impossible to access unless you worked for the vendors of related
device, who obviously had a different agenda then to disclose security
concerns to the general public.
Publishing security related findings at related conferences can be
interpreted in two ways:
On the one hand, presenting at a major event will add to your
credibility and reputation. That's a nice byproduct, but that shouldn't
be the primarily reason, unless you're some kind of a egocentric stage
addict.
On the other hand, presenting findings or giving any kind of
presentation or lecture at an event is a statement of support for that
event. When I submit a presentation at a given event, I think carefully
if that topic actually matches the event.
The reason that I didn't submit any talks in recent years at CCC events
is not that I didn't do technically exciting stuff that I could talk
about - or that I wouldn't have the reputation that would make people
consider my submission in the programme committee. I just thought there
was nothing in my work relevant enough to bother the CCC attendees with.
So when Holger 'zecke' Freyther and I chose to present about our recent
journeys into exploring modern cellular modems at the annual Chaos
Communications Congress, we did so because the CCC Congress is the right
audience for this talk. We did so, because we think the people there
are the kind of community of like-minded spirits that we would like to
contribute to. Whom we would like to give something back, for the many
years of excellent presentations and conversations had.
So far so good.
However, in 2016, something happened that I haven't seen yet in my 17
years of speaking at Free Software, Linux, IT Security and other
conferences: A select industry group (in this case the GSMA) asking me
out of the blue to give them the talk one month in advance at a private
industry event.
I could hardly believe it. How could they? Who am I? Am I spending
sleepless nights and non-existing spare time into security research of
cellular modems to give a free presentation to corporate guys at a
closed industry meeting? The same kind of industries that create the
problems in the first place, and who don't get their act together in
building secure devices that respect people's privacy? Certainly not.
I spend sleepless nights of hacking because I want to share the results
with my friends. To share it with people who have the same passion,
whom I respect and trust. To help my fellow hackers to understand
technology one step more.
If that kind of request to undermine the researcher/authors initial
publication among friends is happening to me, I'm quite sure it must be
happening to other speakers at the 33C3 or other events, too. And that
makes me very sad. I think the initial publication is something that
connects the speaker/author with his audience.
Let's hope the researchers/hackers/speakers have sufficiently strong
ethics to refuse such requests. If certain findings are initially
published at a certain conference, then that is the initial publication.
Period. Sure, you can ask afterwards if an author wants to repeat the
presentation (or a similar one) at other events. But pre-empting the
initial publication? Certainly not with me.
I offered the GSMA that I could talk on the importance of having FOSS
implementations of cellular protocol stacks as enabler for security
research, but apparently this was not to their interest. Seems like all
they wanted is an exclusive heads-up on work they neither commissioned
or supported in any other way.
And btw, I don't think what Holger and I will present about is all that
exciting in the first place. More or less the standard kind of security
nightmares. By now we are all so numbed down by nobody considering
security and/or privacy in design of IT systems, that is is hardly any
news. IoT how it is done so far might very well be the doom of
mankind. An unstoppable tsunami of insecure and privacy-invading
devices, built on ever more complex technology with way too many
security issues. We shall henceforth call IoT the Industry of
Thoughtlessness.
I typically prefer to blog about technical topics, but the occasional
stupidity in every-day (business) life is simply too hard to resist.
Today I updated the shipping pricing / zones in the ERP system of my
company to predict shipping rates based on weight and destination of
the package.
Deutsche Post, the German Postal system is using their DHL brand for
postal packages. They divide the world into four zones:
Zone 1 (EU)
Zone 2 (Europe outside EU)
Zone 3 (World)
You would assume that "World" encompasses everything that's not part of
the other zones. So far so good. However, I then stumbled about Zone 4 (rest of
world). See for yourself:
So the World according to DHL is a very small group of countries
including Libya and Syria, while countries like Mexico are rest of
world
Quite charming, I wonder which PR, communicatoins or marketing guru came
up with such a disqualifying name. Maybe they should hve called id 3rd
world and 4th world instead? Or even discworld?
In 2006 I first visited Taiwan. The reason back then was Sean Moss-Pultz
contacting me about a new Linux and Free Software based Phone that he
wanted to do at FIC in Taiwan. This later became the Neo1973 and
the Openmoko project and finally became part
of both Free Software as well as smartphone history.
Ten years later, it might be worth to share a bit of a retrospective.
It was about building a smartphone before Android or the iPhone existed
or even were announced. It was about doing things "right" from a Free
Software point of view, with FOSS requirements going all the way down to
component selection of each part of the electrical design.
Of course it was quite crazy in many ways. First of all, it was a
bunch of white, long-nosed western guys in Taiwan, starting a company
around Linux and Free Software, at a time where that was not really
well-perceived in the embedded and consumer electronics world yet.
It was also crazy in terms of the many cultural 'impedance mismatches',
and I think at some point it might even be worth to write a book about
the many stories we experienced. The biggest problem here is of course
that I wouldn't want to expose any of the companies or people in the
many instances something went wrong. So probably it will remain a
secret to those present at the time :/
In any case, it was a great project and definitely one of the most
exciting (albeit busy) times in my professional career so far. It was
also great that I could involve many friends and FOSS-compatriots from
other projects in Openmoko, such as Holger Freyther, Mickey Lauer,
Stefan Schmidt, Daniel Willmann, Joachim Steiger, Werner Almesberger,
Milosch Meriac and others. I am happy to still work on a daily basis
with some of that group, while others have moved on to other areas.
I think we all had a lot of fun, learned a lot (not only about Taiwan),
and were working really hard to get the hardware and software into
shape. However, the constantly growing scope, the [for western terms]
quite unclear and constantly changing funding/budget situation and the
many changes in direction have ultimately lead to missing the market
opportunity. At the time the iPhone and later Android entered the
market, it was too late for a small crazy Taiwanese group of
FOSS-enthusiastic hackers to still have a major impact on the landscape
of Smartphones. We tried our best, but in the end, after a lot of hype
and publicity, it never was a commercial success.
What's more sad to me than the lack of commercial success is also the
lack of successful free software that resulted. Sure, there were some
u-boot and linux kernel drivers that got merged mainline, but none of
the three generations of UI stacks (GTK, Qt or EFL based), nor the GSM
Modem abstraction gsmd/libgsmd nor middleware (freesmartphone.org) has
manage to survive the end of the Openmoko company, despite having
deserved to survive.
Probably the most important part that survived Openmoko was the
pioneering spirit of building free software based phones. This spirit
has inspired pure volunteer based projects like
GTA04/Openphoenux/Tinkerphone, who have achieved extraordinary results -
but who are in a very small niche.
What does this mean in practise? We're stuck with a smartphone world in
which we can hardly escape any vendor lock-in. It's virtually
impossible in the non-free-software iPhone world, and it's difficult in
the Android world. In 2016, we have more Linux based smartphones than
ever - yet we have less freedom on them than ever before. Why?
the amount of hardware documentation on the processors and chipsets to
day is typically less than 10 years ago. Back then, you could still
get the full manual for the S3C2410/S3C2440/S3C6410 SoCs. Today,
this is not possible for the application processors of any vendor
the tighter integration of application processor and baseband
processor means that it is no longer possible on most phone designs to
have the 'non-free baseband + free application processor' approach
that we had at Openmoko. It might still be possible if you designed
your own hardware, but it's impossible with any actually existing
hardware in the market.
Google blurring the line between FOSS and proprietary code in the
Android OS. Yes, there's AOSP - but how many features are lacking?
And on how many real-world phones can you install it? Particularly
with the Google Nexus line being EOL'd? One of the popular exceptions
is probably
Fairphone2 with it's alternative AOSP operating system,
even though that's not the default of what they ship.
The many binary-only drivers / blobs, from the graphics stack to wifi
to the cellular modem drivers. It's a nightmare and really scary if
you look at all of that, e.g. at the binary blob downloads for
Fairphone2
to get an idea about all the binary-only blobs on a relatively current
Qualcomm SoC based design. That's compressed 70 Megabytes, probably
as large as all of the software we had on the Openmoko devices back
then...
So yes, the smartphone world is much more restricted, locked-down and
proprietary than it was back in the Openmoko days. If we had been more
successful then, that world might be quite different today. It was a
lost opportunity to make the world embrace more freedom in terms of
software and hardware. Without single-vendor lock-in and proprietary
obstacles everywhere.
During the past 16 years I have been playing a lot with a variety of
embedded devices.
One of the most important tasks for debugging or analyzing embedded
devices is usually to get access to the serial console on the UART of
the device. That UART is often exposed at whatever logic level the main
CPU/SOC/uC is running on. For 5V and 3.3V that is easy, but for ever
more and more unusual voltages I always had to build a custom cable or a
custom level shifter.
In 2016, I finally couldn't resist any longer and built a multi-voltage
USB UART adapter.
This board exposes two UARTs at a user-selectable voltage of 1.8, 2.3,
2.5, 2.8, 3.0 or 3.3V. It can also use whatever other logic voltage
between 1.8 and 3.3V, if it can source a reference of that voltage from
the target embedded board.
Rather than just building one for myself, I released the design as open
hardware under CC-BY-SA license terms. Full schematics + PCB layout
design files are available. For more information see
http://osmocom.org/projects/mv-uart/wiki
There are plenty of cellular modems on the market in the mPCIe form
factor.
Playing with such modems is reasonably easy, you can simply insert them
in a mPCIe slot of a laptop or an embedded device (soekris, pc-engines
or the like).
However, many of those modems actually export interesting signals like
digital PCM audio or UART ports on some of the mPCIe pins, both in
standard and in non-standard ways. Those signals are inaccessible in
those embedded devices or in your laptop.
So I built a small break-out board which performs the basic function of
exposing the mPCIe USB signals on a USB mini-B socket, providing power
supply to the mPCIe modem, offering a SIM card slot at the bottom, and
exposing all additional pins of the mPCIe header on a standard 2.54mm
pitch header for further experimentation.
The design of the board (including schematics and PCB layout design
files) is available as open hardware under CC-BY-SA license terms. For
more information see http://osmocom.org/projects/mpcie-breakout/wiki
Ever since I first came to Taiwan in 2006 (which happens to be more or
less exactly 10 years ago, watch out for a separate blog post about
that), I've been coming back at least once every year until 2014.
Sometimes it's business related ,but one trip per year has always been
about holidays.
I really like the country for a variety of reasons. One of them is the
beautiful landscape from sand beaches to tropical forsts and high
mountains (Taiwan has more than 100 peaks higher than 3000m). This is
also the reason I keep my Yamaha TW-225 motorbike here, as it's
impossible to explore the island without your own transport. And I hate
driving bulky, large cars. Plus, some of the narrow roads have
ascent/descent levels and road conditions that you actually can only
pass them with a motorbike, preferrably using offroad tyres.
But I digress. I like coming to Taiwan, and motorbiking accross the
country is one of the main reasons why.
After the various trips before, including the FIXME last trip circling
the island in 2014, I wanted to do something special this year. The
original plan was to cross the southern cross-island highway
(Provincial Highway 20) from
Taitung to Tainan. Unfortunately that road has been closed for many
years now due to typhoon related damage. Typhonns and Eathquakes (and
associated landslides) unfortunately happen quite often around Taiwan.
I've received some news that a group of motorcyclists managed to pass
the road very recently (despite officially being closed and having
various construction sites enroute). However, the road conditions were
very difficult, having to pass narrow gravel sections at the
construction sites, etc. So I again postponed this plan until a future
year.
Instead, I wanted to travel along provincial highway '7 jia', which
passes alongside a river valley into the central mountain ranges towards
Lishan, and from there take Nantou Lixing Industry road (aka road 89)
further down south towards Ren'ai. This road is of the most narrow
roads that you can find on maps of Taiwan, and leads through very remote
mountain areas with little population. You can find an interesting
report of somebody crazy enough to travel that road on a bicycle in
2015
online. The author describes it as the most epic of all rides in
Taiwan and you can see pictures of the road conditions in it.
Unfortuantely, while on our way, a Typhoon struck the southern
tip of Taiwan, and also brought loads of torrential rain into the
central mountain areas. Given that road 89 is difficult even in dry
conditions, I didn't want to take chances in terms of landslides and
muddy road conditions, and I had to turn back from Lishan to Taipei.
In order to make the return trip a bit different, I went all the way up
to Yilan, and then alongside the north-east coast to Gongliao, before
heading back to Tiapei via Shifen and Pinxi.
Soon after returning to Taipei, the second Typhoon affected the weather
(passing by off the north-east of Taiwan).
In the following week the weather was excellent and there was time for a
second trip. However, due to the rain of the two typhoons I still
didn't want to go for road 89, and decided to go towards the north coast
and continue to explore some of the waterfalls described at the
vonderful http://taiwanswaterfalls.com/ site. Specifically, those
alongside the Keelung river valley, close to road 106 form Taipei
towards the coast, then heading north along the coast, covering the tip
and returning back to Taipei via Dansuei (aka Tamsui).
In summary, the motorbike tours were a lot of fun. What made them more
fun than the previous trips was my strategy of using 'white' roads
(smaller roads) and avoid the provincial highways whenever three was a
'white' alternative on the map. Also, the newly-discovered map of
Taiwans waterfalls was helping a lot to find beautiful sights, and
encourage to go on even uncharted roads at times, a real challenge to
both bike and biker.
So if you ever consider recreational motorbike riding in Taiwan, my
recommendations would be:
get a light-weight motorbike. There's no point for a heavy bike in
Taiwan, other than for locals to show it off. Many mountain roads
typically have speed limits of 30 or 40 kph, and while it is possible
to go 10-20 kph faster, more than that is suicidal. So no need for a
fast bike with large engine.
avoid travel on weekends. Everything is super crowded on weekends. I
prefer to stay home (and even work) on weekends, while using the quiet
weekdays to travel.
always choose mountainous roads over straight roads.
choose smaller 'white' roads over provincial highways. Even though
provincial highways on weekdays have less traffic, they still tend to
attract a fair number of trucks, and are generally more easy and a
less challenging ride.
always carry sufficient water with you.
go in September or October, after the rain season (normally) is over.
The weather is still warm to hot (depending on altitude).
For many years I've always been wanting to do some motorbike riding
across the Alps, but somehow never managed to do so. It seems when in
Germany I've always been too busy - contrary to the many motorbike tours
around and across Taiwan which I did during my frequent holidays there.
This year I finally took the opportunity to combine visiting some
friends in Hungary and Bavaria with a nice tour starting from Berlin
over Prague and Brno (CZ), Bratislava (SK) to Tata and Budapeest (HU),
further along lake Balaton (HU) towards Maribor (SI) and finally across
the Grossglockner High Alpine Road (AT) to Salzburg and Bavaria before heading back to Berlin.
It was eight fun (but sometimes long) days riding. For some strange
turn of luck, not a single drop of rain was encountered during all that
time, traveling across six countries.
The most interesting parts of the tour were:
Along the Elbe river from Pirna (DE) to Lovosice (CZ). Beautiful
scenery along the river valley, most parts of the road immediately on
either side of the river. Quite touristy on the German side, much
more pleasant and quiet on the Czech side.
From Mosonmagyarovar via Gyor to Tata (all HU). Very little traffic
alongside road '1'. Beautiful scenery with lots of agriculture and
forests left and right.
The Northern coast of Lake Balaton, particularly from Tinany to
Keszthely (HU). Way too many tourists and traffic for my taste, but
still very impressive to realize how large/long that lake really is.
From Maribor to Dravograd (SI) alongside the Drau/Drav river valley.
Finally, of course, the Grossglockner High Alpine Road,
which reminded me in many ways of the high mountain tours I did in
Taiwan. Not a big surprise, given that both lead you up to about
2500 meters above sea level.
Finally, I have to say I've been very happy with the performance of my
1996 model BMW F 650ST bike, who has coincidentally just celebrated its
20ieth anniversary. I know it's an odd bike design (650cc
single-cylinder with two spark plugs, ignition coils and two
carburetors) but consider it an acquired taste ;)
In one month from now, I should be reporting from motorbike tours in
Taiwan on the equally trusted small Yamaha TW-225 - which of course
plays in a totally different league ;)
Based on some encouragement from friends as well as my desire to find
more time again to hang out at community events, I decided to attend
Electromagnetic Field 2016 held in
Guildford, UK from August 5th through 7th.
As I typically don't like just attending an event without contributing
to it in some form, I submitted a couple of talks / workshops, all of which were
accepted:
An overview talk about the Osmocom project
A Workshop on running your own cellular network using OpenBSC and related Osmocom software
A Workshop on tracing (U)SIM card communication using Osmocom SIMtrace
I believe the detailed schedule is still in the works, as I haven't yet
been able to find any on the event website.
Looking forward to having a great time at EMF 2016. After attending
Dutch and German hacker camps for almost 20 years, let's see how the
Brits go about it!
