Journey to Protected Computing
My journey to Protected Computing arguably began over two decades ago, at the turn of the millennium, when I decided that security & privacy was where I wanted to dedicate my career in computer science and engineering. While there are so many huge impact areas to apply engineering and software skills, cybersecurity really called out to me: I loved the idea of applying math, science, creativity, and hard work towards the goal of protecting folks and thereby unlocking the full power of digital technology to improve lives and the world.
Before we stroll further down memory lane, my informal definition for Protected Computing: technology paradigm wherein users retain exclusive access to their private information processed by systems and services; no other entity - the service provider or anyone else - has access to the plaintext information. I called this Private Computing about a year ago in a Google blog, but the concept was rebranded in the Google I/O 2022 keynote, which focused on shifting how, where, and when data is processed by personal information services.
The promise of Protected Computing dawned on me in a big way in 2014. I was coming to the end of a long stint leading engineering at Green Hills Software. For many years I’d heard security professionals tout the meme: “you can’t retrofit security”. Secure-by-design, private-by-default, etc. were mantras, and any departure seemed heretical. In fact, much of my work had focused on developing software to meet the highest levels of security. Secure-by-design was the core value of the products we were working on, including a real-time hypervisor that met the highest software security certification level in history, with formal mathematical proof of isolation policies. Yet this same secure-by-design approach, elegantly and perhaps ironically, enabled “secure retrofit”: we retrofitted the hypervisor below general purpose operating systems used in embedded and mobile computing systems. The idea was to leverage hypervisor isolation to protect critical workloads, like network encryption (a virtual bump-in-the-wire) and automotive autonomous control systems, rendering them immune to threats within the isolated operating systems (which could not be feasibly retrofitted to the same security level from within). As I departed Green Hills in 2015, end-to-end encrypted messaging apps were going mainstream, and I started wondering about another form of “security retrofit”: why can’t we retrofit a similar kind of data protection in personal information services? I toyed with the idea of starting a company to implement this as a layer on top of social networks; imagine if all your posts were encrypted with a key accessible only to you and your personal network. Your private information would never be accessible to the social network service or any other service provider connected to it, even though the service was never designed for this level of end-to-end privacy.
Then I joined BlackBerry as CSO to work on a privacy-focused Android device and enterprise security software. While at BlackBerry, my journey to Protected Computing was further influenced by the acquisition of WatchDox, which offered end-to-end encrypted files accessible only at the client (and not by the service provider). While at BlackBerry I did a TEDx talk where I spoke about the end-to-end encrypted social network concept and more generally about how we can combat the crisis of confidence in privacy across the rapidly expanding Internet of Things.
Since joining Google in 2017 to lead security and privacy engineering for Android, the journey has continued. At Google I/O 2021, Google announced the “Private Compute Core”, a foundational building block for private processing within the Android operating system. Another building block we’re working on is the next-generation Type-1 hypervisor approach in Linux, called Protected KVM. Other Protected Computing building blocks at Google include federated learning, federated analytics, cloud confidential computing, differential privacy, and edge-to-cloud end-to-end encryption infrastructure used for private Android cloud backups and Chrome’s password manager. Other Protected Computing information services include video chat, messaging, and enterprise private cloud storage.
Protected Computing: the next major computing shift
Like mobile, cloud, and AI-first computing, Protected Computing is one of the most important technological shifts of the digital world. While Protected Computing is in its relative infancy today, progress is accelerating across global scale private information services. It may seem self-evident why Protected Computing is so important: user expectations for privacy are growing quickly and globally. Computing can evolve to deliver helpful and delightful information services without forfeiting the user’s exclusive control over that private information - and therefore computing must evolve.
However, Protected Computing is also going to enable a rather astonishing intelligence paradox. Privacy regulations have created stovepipes, walled gardens, and user consent fatigue. In order to collect and share information between compute entities and systems, users are bombarded with consents and privacy policies they don’t have time to read, written with obscure language. This friction ultimately doesn’t scale. Seamless sharing is reduced, by design, generating a strong headwind against flexible computing needs, such as that required by ambient computing.
In a future ambient world, the devices surrounding us constantly sense what we’re doing and apply intelligent processing to this ambient information to deliver even more helpful experiences. If I’m winding down from a workout at home, my wearable detects my slowing gait and elevated body temperature, shares that analysis with my refrigerator (which knows to pour me a cold glass of water) and my kitchen smart display which knows to play the latest news, reducing volume as I walk closer to the kitchen. But that sharing can’t easily occur across accounts from multiple suppliers because the user must understand and consent to that sharing, which may include pushing sensitive data into multiple clouds (with associated hacking and leaking risks). In a Protected Computing world, sensor data can be processed locally, using secure enclaves where needed, with resulting de-identified inferences forwarded (e.g. “hey fridge, pour a glass of cold water”). While many assume that obscuring data from service providers will cause an untenable loss in intelligence, Protected Computing can actually enable more data to be shared and leveraged to improve our lives, without ever asking the user to give up control of that data (while cutting down on consent overload): the intelligence paradox!
So why is this hard?
Protected Computing is not just about on-device processing and encryption. Certainly there are many situations where we want to process sensitive data on the edge (in our smartphones, wearables, smart displays, home routers, etc.): an important technique in security and privacy is to process and retain sensitive data where it is generated to avoid the risk of moving and storing it elsewhere. But we can’t store all of our photos and email on a phone, can we? So we need Protected Computing from edge to cloud. For example, we can encrypt and process on the edge while using the cloud for bulk storage of encrypted and anonymized data. Or offload processing to the cloud using secure cloud enclaves, confidential compute, and homomorphic encryption to ensure plaintext is never exposed to the service provider. Yes, this all requires rearchitecting existing services and building infrastructure that can meet performance and scalability requirements while ensuring user data remains verifiably private.
Nevertheless, more processing will move inexorably to the edge, creating novel security challenges. For example, machine learning models may contain sensitive intellectual property or perform critical functions that we must protect against reverse engineering. While these models traditionally resided on cloud servers sitting behind many layers of physical security, edge devices lack these moats and are commonly lost or stolen. We need improved edge security and hybrid edge/cloud processing models to account for these differences.
Counter-abuse is another of the most important challenges to address in Protected Computing. Cloud information service providers have spent many years building sophisticated defenses to mitigate all manner of abuse, from malware and financial fraud to the distribution of harmful content. In fact, while governments push technology providers for more privacy, the same governments are asking technology providers to stop or break encryption. The answer to this dilemma is not to weaken encryption; if end-to-end encryption is hamstrung in consumer information services, abusers will simply roll-their-own, because math, while the privacy of billions of consumers is degraded.
Does that mean we just give up on fighting abuse in Protected Computing services? Of course not. In fact, we must fight harder! The work is delicate because we must ensure that processing of data within Protected Computing environments cannot be misused beyond their intended mission. These Protected Computing counter-abuse frameworks must be developed in the open and subject to independent security evaluation and real-time code verifiability using binary transparency. The good news is that we can leverage the same Protected Computing techniques as the information services they seek to protect; for example, abuse detection can be performed locally or in secure enclaves that verifiably cannot exfiltrate private information.
This is an area where the technology world must not only invest much more but also must work collaboratively together, transcending the awkward coopetition we tiptoe through today. Apple and Google’s partnership on Exposure Notifications (another great example of Protected Computing - using on-device contact detection and ephemeral identification to avoid sharing user location or other PII with service providers) to combat the pandemic may serve as a bellwether. Tech needs to truly partner on this. Another critical (and thus far unsolved) problem is enabling end-to-end encrypted communication between users on disparate services (e.g. Google Messages, iMessage, Whatsapp). Consumers deserve better than the walled gardens they face today; if you text and see a green bubble, it means your service provider has failed you on privacy. And the collaboration must extend to a public-private partnership, in which regulators help tech companies navigate a well-lit path.
NGOs also have a critical role to play. We need productive discourse rather than absolute positions that slow us down and actually run at cross purposes to improved privacy for users. For example, while a narrow set of privacy NGOs disparage any form of on-device content processing, the technology world already has a tremendous amount of effective, privacy-preserving processing of E2EE content, including on-device intelligence for autocorrect, autocomplete, smart suggestions, and real-time malware and phishing detection. It would be helpful if NGOs can partner with technology companies to improve Protected Computing approaches, such as researching design flaws, helping tech and regulators align better, pushing for open source and openly auditable infrastructure, hosting binary transparency logs, etc.
How do we know we’re moving in the right direction?
Transparency and control are the two most fundamental principles of digital privacy, and Protected Computing inherently and elegantly provides both. Today, when an information service collects large amounts of personal information, that information is subject to server-side data disclosures of various forms. For example, lawful access data requests are published by tech providers. As private information migrates into Protected Computing systems, those reports will be empty because the service providers simply no longer have access to the data. When we see tech industry data access requests trending down globally, and we’re making excellent progress on counter-abuse in Protected Computing services, we’ll know we’ve turned the corner. I will try to make that day come as soon as possible, and I hope you’ll join me on the journey!
yes yes. this is the best case scenario for the future of computing. i don't like that technology is becoming so much more engraved into our lives for increasingly simple things, but the internet of things is here, convenience is king, and AI fueled by their data will inevitably be used, and if we don't mitigate just how bad the collection of data is by design, while the tech is still being used to gather our data we've failed the next generations who will be living with this tech, and they'll be living like they're in one of those great reset conspiracy theories with a "quantum tattoo" or something.
Technology needs to be uncensorable, private by default, all of those buzzwords that you probably already know if you're reading this article, but it's true.