Imagine if everyone spoke a language that you don’t understand. It’s not a foreign language–it’s been spoken around you since the day you were born–but where as everyone else understands it immediately, for you it means nothing. Others become frustrated with you. Friendships and jobs, just being “normal”, seem to assume fluency. For many with autism, this is the language of emotion. For those on the spectrum, fluency in facial expressions doesn’t come for free as it does for “neurotypicals”–reading facial expressions is a superpower. So, when my son was diagnosed, I reacted not just as a mom–I reacted as a mad scientist and built him a superpower.
But no mad scientist mother would stop there. When he was diagnosed with type 1 diabetes I hacked his insulin pump and built an AI that learned to match his insulin to his emotions and activities. I’ve also explored neurotechnologies to augment sight, hearing, memory, creativity, and emotions. Tiger moms might obsess over the “right” prep schools and extracurriculars for their child, but I say why leave their success up to chance? And if I turn my son into a cyborg and change the definition of what it means to be human, how will your child ever keep up?
If I turn my son into a cyborg and change the definition of what it means to be human, how will your child ever keep up?
Years ago, on my very first machine learning project as an undergrad, I helped build a system for real-time lie detection off of raw video. (It was, of course, sponsored by the CIA. Did you think I was kidding about the “mad scientist” thing?) The AI we developed learned to recognize the facial expressions of people on camera and infer their emotions. Before this project, I assumed I’d spend a long neuroscience career sticking electrodes into brains, but watching our algorithms learn such a foundationally human task hooked me on studying natural and artificial intelligence together.
Fast forward through 10 years of my academic career (neural coding and cyborgs) and my first few startups (AI for education and jobs), and I had built a reputation as the crazy lady seeking to “maximize human potential”. When the ill-fated Google Glass, a wearable smartphone masquerading as a pair of glasses, was launched by throwing some guys out of a blimp, I was invited to explore ideas for what could be done with it beyond social media posts and family videos.
For a woman that wanted to build cyborgs, there was so much potential.
Along with its computing power, Glass had a live camera, a heads up display, and a combination of voice and head-motion controls. Drawing from that old CIA project and my years of machine learning research, I built face and expression recognition systems for Glass. (In truth, the crappy little processor would heat up like a bomb, and so the system required an extra computer strapped to the user’s back–not exactly Iron Man.)
For a woman that wanted to build cyborgs, there was so much potential.
I could read people's faces with these augmented reality glasses—and so many more terrible things. I could scan a room reading expressions and flag false smiles (LA and DC, I’m looking at you), access their credit scores, or pull up their Facebook or Grindr accounts (or Ashley Madison for CFOs). The scene could play out like an episode of Black Mirror with Glass cuing my actions to exploit the emotional vulnerabilities of others. But I wasn’t interested in exploring the questionable or downright terrifying applications of Glass. I just wanted to give kids like my son greater insight into the people around them.
In 2013 I built a proof of concept system. Based on research from one of my academic labs, our system recognized the expression of the face in front of it and then wrote the emotion on the little heads-up screen, allowing an individual with autism to more easily perceive whether the person before them was happy, sad, angry, and more. Simply wearing Glass while continuing everyday social interactions with others allowed these kids to learn that secret language of facial expressions. The state of the art in emotion recognition training for individuals with Autism remains cartoon faces on flashcards. Here’s a smiley face; here’s a frowny face. But our brains process the contextual information of the natural world differently than these static, artificial signals.
My son is rather amazing, and the more I experimented the more I realized that I didn’t want to “cure” his autism. I didn’t want to lose him and his wonderful differences.
The research has continued over the years and overcome many of the original limitations. For many kids, these systems are more than a prosthetic—they actually advance their learning of this secret language. A team at Stanford has shown that these technologies can improve their expression recognition, even when not wearing it. And most surprising was the discovery that it helped develop their empathy. Learning that a smile means happiness from a flashcard teaches nothing about why people are happy. Learning the same from natural social interaction actually helps build theory of mind, another secret language thought to be missing in individuals with autism.
My son is rather amazing, and the more I experimented the more I realized that I didn’t want to “cure” his autism. I didn’t want to lose him and his wonderful differences. SuperGlass became a tool to translate between his experience and ours, a tool to help these kids navigate a sometimes alien world. In an era where jerks like me are building AIs to do an increasing array of complex and sophisticated human tasks, your value to the world is what makes you different. The more different you are, the more valuable. My son is priceless.
Your value to the world is what makes you different. The more different you are, the more valuable.
I want to build a world where everyone has superpowers. I went to grad school telling fellow students “I want to build cyborgs” (and they would literally scoot away from me for fear that my crazy would rub off). I explored what it would mean if prosthetics could directly interface with your brain, a field now known as “neuroprosthetics”. Already today many neuroprosthetics transform people’s lives: cochlear implants for deafness, retinal implants for the blind, motor neuroprosthetics for the paralyzed, and deep brain stimulation for a rather extraordinary array of disorders, including depression and Parkinson's.
My first project in neuroprosthetics came during grad school at Carnegie Mellon. My advisor and I developed a machine learning algorithm that learned how to hear just by “listening” to the sounds we recorded in the parks around Pittsburgh (one of the strangest and most underappreciated cities in the US). As it listened, the algorithm slowly learned to hear more and more, subtly adjusting millions of internal calculations to make greater sense of its’ auditory world.
I began to wonder if we could build an AI-driven cochlear implant, a neuroprosthetic ear that restores hearing to some forms of deafness. Our experiments showed that the algorithm greatly improved speech perception for those using these implants. It was the first time I’d built something that could transform someone’s life and I knew that this was how I’d spend the rest of mine. But it was also an introduction to the messy complexity of what makes a “better” life. As a naive hearing person, it never occured to me that anyone would choose deafness, but I learned that some parts of the deaf community consider cochlear impants to be genocide: of their unique languages, of their way of life, of who they are. And yet, there is also evidence that infants receiving implants not only learn to hear but that it even increases cognitive function. Much like autism, I’m often confronted with the dilemma of “curing” people of who they are versus giving them the tools to share those rich differences with the world.
Much like autism, I’m often confronted with the dilemma of “curing” people of who they are versus giving them the tools to share those rich differences with the world.
This dilemma takes on an even greater urgency when I confess that my particular area of research and development is cognitive neuroprosthetics: devices that directly interface with the brain to improve our memory, attention, emotion, and much more. For many, the idea of computers being jammed into our brains evokes science fiction nightmares like the Borg from Star Trek or the human-like machines of The Terminator. While my own work takes me in very different directions than these dark stories, it’s true that neuroprosthetics are already beginning to change the definition of what it means to be “human”, and the end result of these explorations of humanity are not at all clear.
In many cases what it means to be human is tragic. Children with traumatic brain injuries (TBI) are often devastated by their injuries and suffer long-term mental and physical challenges. Clinical videos of kids and adults tearfully struggling with tasks that used to be trivial are heartbreaking. Many of those with TBIs have trouble with their working memory span, which is roughly how many “chunks” of information a given person can remember in any given moment. Working memory plays a sizeable role in education attainment, life-time income, and even health and longevity. When we know we can make a difference, isn’t allowing a car accident or a fall or even poverty take that future away from a child just as morally perilous as augmentation run amok?
At my mad science incubator, Socos Labs, we are collaborating with neurotech startups like HUMM and Cognixion to make a difference in these kids’ lives. HUMM has developed a wearable headband that sends electrical signals that enhance connections between the prefrontal cortex and the rest of the brain. The technology uses transcranial alternating current electrical stimulation (tACS) to enhance the connection between frontal parts of the cortex (crucial for working memory) and more posterior brain regions. The stimulation drives increases in multitasking performance, attention, and working memory span.
Neuroprosthetics will fundamentally change what it means to be human over the next two decades, but for whom?
In a recent experiment, adults playing the old “Simon” game, where you must remember the order of increasingly long patterns of lights and sounds, increased the length of sequence they could regularly remember by 20% compared to a sham stimulation. And in another recent experiment, similar stimulation improved working memory in seniors experiencing cognitive decline. This technology could have a tremendous impact on a kid with a TBI and others struggling with working memory challenges. If a non-invasive device paired with intense therapy could improve their chances of living longer, richer lives, no loving society could deny them this opportunity. A sustained 20% increase in working memory would change academic and economic life outcomes for so many. As with all of Socos Labs’ projects, we cover all of the costs and give our discoveries away—a reasonable investment in a world in which every child has the chance to write their own life story.
It would be willfully naive to think that neuroprosthetics research ends with these children or with those suffering from dementia. If these technologies can augment functions for these differently-abled populations, they will inevitably one day do the same for the rest of us neurotypicals (a technical term meaning “your brain is boring”). Students in the US already experiment with drugs like ritalin and adderall to improve their academic outcomes, and parents are often willing participants, even though the benefits might be an illusion. The pressure to perform is so great, and the fear of losing our privileged place in the socio-economic hierarchy so intense, that it drives hyper-competition in academics and the workplace.
At Socos Labs, we don’t just invent “superpowers” (as fucking cool as that is!), we investigate how these technologies drive inequality and other unintended consequences. The core question for many inventing AI and neuroprosthetics is usually, “What most idealized use for this technology can I imagine…that will also make me rich?” But the real question must be, “What will actually happen when this technology impacts the real world?” Neuroprosthetics will fundamentally change what it means to be human over the next two decades, but for whom? Though anyone might have access to new neurotechnologies in theory, in reality those most able to take advantage of them are the ones who need them the least. Having wealthy parents already dramatically impacts a child’s outcomes, even affecting working memory. By contrast, simply being born into poverty and stress robs children of their cognitive potential. Imagine these advantages not subtly embedded in the life experience of the wealthy but directly for sale and turned up to 11. Intergenerational social and economic mobility disappears.
I love the idea of a cyborg who’s augmented to explore their emotions. That’s definitely not the Borg.
And working memory is just the start. Research shows that we can augment creativity and emotional control, as well as influence honesty, pleasure, and numerous other foundations of self. As with autism, those with “abnormalities” in these areas can have profound challenges engaging with the world. I’ve worked on systems to predict manic episodes in bipolar sufferers. Groups at MIT are using rhythmic visual or auditory stimuli to reduce Alzheimer's symptoms, and others to detect seizures and depression. Diverse neuroprosthetics beyond my son’s or HUMM’s might allow people to become much more than a tragic diagnosis. In a world that values difference, they might become even more than the neurotypicals around them. And if their augmentation lifts them above the crowd, soon everyone will want to be more than human. Or if these augmentations become a norm, who will choose to be less than human?
You probably have an equalizer on your phone that allow you to amplify the bass and treble in music. Adjusting the slider controls around doesn’t fundamentally change the song, but it emphasizes differing elements, from the clarity of voice in an opera to the big bass drop of the best EDM (is there such a thing?).
Now imagine the app equalizes you. Instead of adjusting the power at different sound frequencies, sliding a controller on this app boosts your attention or dampens your creativity. Add in a boost for memory and you are ready to cram for an exam. Hit the “Date Night” preset to stimulate emotion and focus while dampening cognition. If there’s a bad romantic comedy in your near future, why be too smart to enjoy it? We might augment any of the richly diverse aspects of the human brain. I love the idea of a cyborg who’s augmented to explore their emotions. That’s definitely not the Borg.
Neuroprosthetics shouldn't replace what we can do for ourselves, they should augment who we aspire to be.
Though no one has yet combined these many forms of neurostimulation into a prosthetic, we couldn’t resist experimenting with simulated versions that dynamically adapt to an individual's environment. Combining historical EEG ("brainwave") data with eye movements and heart rate, our AI orchestrates its multidimensional neurostimulation to produce a version of you more suited to the task at hand, dynamically adapting you to problem solving or relaxation or public speaking. Though it's only a theoretical simulation, the system follows my most important technology design rule: you should not only be better when you’re using it, you should be better than where you started when you turn it off. Neuroprosthetics shouldn't replace what we can do for ourselves, they should augment who we aspire to be.
The capabilities of these neurostimulation systems only scratch the surface...literally. They work via wearable devices on the scalp, but invasive neuroprosthetics act inside the brain. This has traditionally been done with ultra-thin wires implanted chronically through the skull (“I need a neuroprosthetic like I need a hole in my head”) but new technologies go far beyond that. These include injectable self-expanding “neural nets”, optogenetic (light-based) interfaces, and one of my favorite up and coming technologies, neural dust. The “dust” consists of nanodevices the size of a grain of sand and powered by ultrasound to stimulate individual nerve fibers. Spreading thousands of these throughout the brain could allow us to create ultraprecise closed-loop prosthetics, simultaneously decoding neural activity and encoding new signals into them. It may sound rather mundane, but combinations of technologies like those above might someday decode visual and speech center patterns allowing you to thinking up a multisensory email, with “attached” emotional content, all while simply walking down the street.
These technologies can give people back their lives, but neural diversity should be embraced as a core to problem solving and innovation.
As science fictional as neuroprosthetic telepathy might sound, profound possibilities are emerging today, from restoring movement to the paralyzed to restoring memory function to injured soldiers. Even Autism might be transformed. One of Autism’s most salient features in the brain is a lack of connectivity and communication between different cortical regions, such as the frontal-posterior communication in working memory. Unable to communicate with one another, each cortical region develops in a kind of isolation, compromising neurotypical brain function. With more invasive technologies like neural dust we might be able to restore some degree of intra-cortical communication, bypassing the biology via AI-mediated wireless communication (like the way much of Africa skipped landlines and went straight to mobile phone networks). Projects like SuperGlass can only help the kids “high-performing” enough to engage with the world around them; however, bringing intracortical communication to life in profoundly autistic brains could completely transform lives. Differing patterns of disrupted cortical connectivity also plays a role in both psychotic illness and emotional disorders. Invasive neuroprosthetics might transform these lives as well.
And with this power will come a profound challenge: restoring a life without erasing what makes it unique. Neuroprosthetics have the ability to change anyone’s life, especially those who are not neurotypical, those experiencing autism more severe than my son, people with traumatic brain injury or bipolar or Parkinson’s or Down's Syndrome. Simply being born into poverty and stress robs children of their cognitive potential. These technologies can give people back their lives, but neural diversity should be embraced as a core to problem solving and innovation.
Individuals with autism see the world how they see it, not with adherence to others’ perspectives, or by conforming to social norms to be the same as everyone else. That alone is a superpower that will benefit my son throughout his life.
Kurt Vonnegut’s “Harrison Bergeron” imagined a world in which prosthetic handicaps make us all equal by removing advantage. It is equally possible to lose our rich differences through over-augmentation as well. If we assume there is only one kind of strength, one kind of beauty or intelligence, then we might “super-normalize” away the rich differences of the human world. I don’t want to “cure” someone of themselves. I want them to be able to share that self with the world. These technologies could and should be used to give people with disabilities, the non-neurotypical, the ability to exist and thrive in a neuro-typical world. In the case of kids with autism, if these types of devices are able to help them on that journey in any way, I hope that they’ll consider using them as a superpower in their back pocket. For everyone else, I admit that I want a superpower too. But these cannot be sweet sixteen gifts from hyper-competitive parents; to use or not use a neuroprosthetic must be a human right.
Individuals with autism see the world how they see it, not with adherence to others’ perspectives, or by conforming to social norms to be the same as everyone else. That alone is a superpower that will benefit my son throughout his life. If allowed the opportunity, kids like him will grow to be every employer’s dream hire. Everyone has value–our job, each one of us, is to help everyone around us find and realize it.