The latest from the UK Home Secretary on "end-to-end encryption" and the responses make me feel the need to explain some things cryptographic and historical.

Modern secure messaging apps and services provide several functions:

1. I can find people - they provide a directory, where often the primary identity is simply the phone number;
2. They provide a private and easy way to exchange encryption keys;
3. They utilise state of the art, and publicly documented encryption protocols between the end points;
4. And, since we lost end-to-end networking for most users, they implement a network forwarding service.

My observation of the popular use of the phrase "ene-to-end encryption" is that often folks mean both functions 2 & 3 - that is both private key exchange as well as use of state of the art encryption between the end points. Hence as a technical pedant, I find myself peeved by much discussion on this topic which confounds and confuses these two functions, so I have felt compelled to write this post!

The directory service is technically the most boring but actually a very useful element of such a service; one route to finding friends is simply that the smart phone app reads your contacts and makes connections by looking for matching phone numbers already using the service. Compared to the suggestions we build a web of trust  it's a heck of a lot simpler to use!

Many of these apps (including WhtsApp) implement a very cunning state of the art key exchange algorithm as developed by Open Whispers Systems and widely available in their open source [1] Signal app. This allows two parties who wish to communicate to share an key with which to encrypt their message without the service provider knowing the key. As noted by in the excellent article by Kieren McCarthy:

...companies like Facebook, Google, Apple and so on could redesign their systems to make it possible to decrypt them. They could even avoid the problem of a simple backdoor by using constantly changing encryption keys – so long as they keep a copy of those keys.

The desired point of intervention is the key exchange protocol, it would be straightforward to arrange that only for those targeted for surveillance, the keys and messages are kept. Doing this on a per service and per target basis is not the end of the Internet, of secure banking, of eCommerce, etc. However, it is a backdoor - bad'uns will try and attack it of course and it is a risk. That said, all the public key and certificate infrastructure that underpins session key exchange for https, and hence the majority of the Internet services we use, rely on keeping the secret half of the public/private key pair secure - that's a even juicer target for bad'uns - grab that and they can subvert the service not just a single conversation...  So we do need some perspective on the hacking risk.

Importantly the point of intervention is not the encryption algorithm itself - here again I find myself peeved when I hear the statement "it is against the laws of mathematics". Not really - we could simply be relying on our mathematical ignorance. Currently we think the cryptanalysis of our modern crypto algorithms is too hard (that is computationally hard and hence expensive) - we haven't proved it is mathematically, maybe we just haven't figured it out yet [2] - indeed therein lies the history of cryptanalysis! We didn't know about differential cryptanalysis for years...

It is worthy of note that where is a need for encryption between two parties but with recoverable key exchange, protocols have been designed specifically to provide this - for example, MIKEY-SAKKE was designed to provide the ability for an organization to deploy end-to-end encryption but also allow for that organization to acquire the keys and decode the messages, specifically where there is a regulatory requirement to do so. Examples cited include emergency services or financial services - or maybe we should just trust the bankers not to collude on price fixing, that has always worked well.
However, the key point of Kieren's article that resonates is that governments have simply shown themselves to be untrustworthy, and on that, there is no going back. To be clear though, as a professional paranoid, I don't understand why we trust the companies either - WhatsApp could be saying they do "end-to-end encryption", but since their code is not open to independent review, how would we know? If you are serious about your privacy check out the EFF Secure Messaging Scorecard - me, I trust Signal.

So far, so current affairs; but I think we should analyse how we got here and where we might go. The root of this evil is in fact that we lost end-to-end networking.

1981 was a key year for the Internet; growing from the experience of years of research on Arpanet, it saw the release of RFC 791 which defined IPv4, the Internet Protocol we still live with today, and indeed for many is the only supported network level protocol [3]. The philosophy was encapsulated in "the End-to-End Argument" as extolled in the seminal paper by Saltzer, Reed and Clark [4]. Importantly underpinning the idea was that computers could all speak to each other if they were connected to the Internet, indeed in defining the IPv4 address:

"Internet addresses distinguish sources and destinations to the host level and provide a protocol field as well.  It is assumed that each protocol will provide for whatever multiplexing is necessary within a host." 

However, we lost the plot on this. In general my 4G/wifi roaming smart phone "on the Internet" cannot speak to my home computer "on the Internet" directly - we built a series of network level mechanisms in the home, in broadband infrastructure and in mobile networks that broke the end-to-end network connectivity. Today we require most interaction between two users to be via "over-to-top" service providers, so nearly everything is now mediated by the behemoths of our era - Google, Facebook, Twitter, ... Furthermore, it is in the commercial interests of these companies to continue be mediate communication and that applies to WhatsApp too. And it's getting worse - most of the IoT kit does the same.

This then leads to the current regulatory situation - if you provide a service, an app, and forward most of the messages, you are going to be seen as a plausible target for regulation, and in particular a single point at which someone might require key escrow on demand.

However, the day will come when someone tweaks and releases the open source Signal app to not use the Signal service, rather a combination of direct SMS messages and an end-to-end network layer like IPv6 [5], and all the regulating of these service providers will have been an exercise in futility.

[1] As is repeatedly pointed out, the cat is out of the bag - the technology is freely available as open source for anyone to use...
[2] I promise not to create any more theorems, so let me put this stake in the ground - I have a proof concerning factoring large numbers but it is too large to fit in the margin.
[3] Recent reports indicate IPv6 is over 15% of all Internet traffic, but it has yet to achieve wide-scale deployment at the edge of the network, and hence be accessible to most end users.
[4] Saltzer, J. H., D. P. Reed, and D. D. Clark (1981) "End-to-End Arguments in System Design". In: Proceedings of the Second International Conference on Distributed Computing Systems. Paris, France. April 8–10, 1981. IEEE Computer Society, pp. 509-512.
[5] Possible 3rd year undergraduate project I think.