Sabera Talukder
Neuroengineer | Creator | Researcher
   

What is Neuroengineering?

October 29, 2018


Neuroengineering. Neural engineering. Neurotechnology.


 You have heard variations of these words in whispers throughout the internet, but you might not actually know what they mean. And it's no wonder! A simple Google search of the term ‘neuroengineering’ returns an expansive yet unhelpful Wikipedia page and fairly nondescript university landing sites.

 This burgeoning, fuzzily-defined, interdisciplinary field of neuroscience, electrical engineering, biochemistry, and computer science has just started to emerge from the dark hallways of academia. And it will revolutionize the way humans interact with themselves, information, and their surroundings.

If you don't believe me, check out Cathy drinking her morning cup of coffee.


  Cathy, has been paralyzed for over fifteen years, and yet with her brain she can control a robotic arm like it's her arm!... But how?!... We will get to it in more detail, but it all starts with that little gray box affixed to her head. It is also worth mentioning that this work only falls into one of the neuroengineering sub-fields. The work coming out of the other sub-fields is just as amazing.

  Let's demystify some aspects of neuroengineering so that you can approach and understand the field. Fair warning, neuroengineering is so young that many researchers do not even agree on terminology.

 There is so much to cover, but let's start by establishing common ground. We are going to: define neuroengineering by outlining its major divisions. Sounds simple enough!

Neuroengineering Definition

 Neuroengineering can be broken down into the following categories that span basic science to pure engineering:
  • Reading and writing information out of the central or peripheral nervous systems.
  • Controlling neurons to investigate neural circuits.
  • Using the brain to inform the algorithms and architectures used in hardware or software designs.

 Wooh that was a mouthful! Let's break it down.

 Reading and writing information out of the central or peripheral nervous systems is largely encapsulated in the field of neuroprosthetics. One example of a neural prosthesis is the system that allows Cathy to control the robotic arm.

 The devices created within the neuroprosthetics field go by many names such as brain machine interfaces (BMIs), neural prostheses, neural interfaces, brain computer interfaces (BCIs), and many many others. These terms are mostly interchangeable.

 There are a few categories in which prostheses can fall. These include central nervous system (CNS) or peripheral nervous system (PNS) prostheses, and invasive or non-invasive prostheses. categories of neural prostheses: central nervous system, or peripheral nervous system, invasive or non-invasive. Where brain machine interfaces, brain computer interfaces, and neural prosthetics fall

 That's great, but how does a neural prosthesis work? You can divide what a prosthesis does into four main categories: (1) hardware to collect neural activity, (2) software to spike sort, (3) software to perform neural decoding, and finally (4) software and hardware that use control signals to perform some output.

 We are going to dive into these categories in more depth, but before we do, below is an image of the general work flow for a CNS and invasive neural prosthesis. Notice that the diagram represents many types of prosthetics, including motor and communication prostheses. flow flow through a general neural prostheses, aka brain machine interface, brain computer interface, and neural prosthetic

 As we progress through this section reference this image if you can't visualize where in the loop we are. In addition, I encourage you to envision how you think this figure changes if the prosthetic deals with the PNS or is non-invasive. Now to business!

  1.  Hardware to collect neural activity. This usually amounts to a type of electrode array. In Cathy's case and in our diagram the array is invasive, but non-invasive arrays also exist.

     I know what you’re thinking. A non-invasive way to monitor neural activity? Pshh, I’ll believe it when I see it. Well here you go!
    categories of neural prostheses: central nervous system, or peripheral nervous system, invasive or non-invasive. Where brain machine interfaces, brain computer interfaces, and neural prosthetics fall  This bracelet, courtesy of CTRL-labs, uses some combination of electromyographic and mechanomyographic electrode arrays. These types of arrays non invasively measure the electrical and mechanical movements of your muscles and nerves. Non-invasive arrays are amazing, but they pale in comparison to invasive arrays with respect to their signal-to-noise ratio.

     An awesome example of an invasive array is the neuropixel probe, which can record from hundreds of neurons across multiple brain regions. In the neuropixel below, the red arrows indicate the piece of the probe that is inserted into the brain. Picture of the neuropixel probe that is inserted in the brain. The probe is common for many neuroengineering studies.

     The picture is incredibly faint, because the probe has a diameter of only seventy micrometers! For some context, the width of a typical needle that you get a flu vaccination with is about seven hundred and twenty micrometers.

  2.  The next component in our neural prosthesis is: software that can detect and decouple neural spikes. Neural spikes are the temporal record of when a neuron fires an action potential. There are many spike sorting approaches from threshold based, to hoop based, to principal components based. For now, we are only going to dive into the simplest one, the threshold spike sorter.

     In this case, we start with raw spikes from our electrode array, filter them with a high pass filter, set a voltage threshold for each neuron, then sort them based on their threshold. Don't worry, we'll break it down further!

    Picture of the neuropixel probe that is inserted in the brain. The probe is common for many neuroengineering studies.

     In the "High Pass Filtered" sub-image you can see how there are many spikes but only two main voltage levels. The assumption is that each voltage level corresponds to a different neuron. Given this assumption, we set two threshold values which are the red lines in the "Thresholded" sub-image. We then extract the spikes at each threshold value to get the red and green traces in the "Sorted Spikes" sub-image. The red trace corresponds to the waveform of one neuron, and the green trace corresponds to the waveform of the other neuron.

      Spike sorting is ridiculously important because it is virtually impossible to measure just one neuron’s activity from the electrode arrays.

  3.  Once we have our separated spikes we have to read them! We accomplish this with: software that performs neural decoding. Before we jump into neural decoding let's define some terminology. Neural encoding is how we map a behavior to the neural responses the behavior evokes. Neural decoding is how we map from neural responses to the behavior encoded in those neural responses. Picture of the neuropixel probe that is inserted in the brain. The probe is common for many neuroengineering studies.

     You could perform neural encoding in humans, but it is more common to use monkeys that have similar neural architectures to humans. The data generated can then be useful to create neural decoders. In Cathy's case, we can use a neural decoder to control a robotic arm.

     Neural decoders can be broken up into discrete and continuous decoders. Discrete decoders are concerned with decoding the final state of an action, while continuous decoders are concerned with decoding the entire progression of an action. Can you guess which type of decoder was used for Cathy? What about the decoder used in the video below?


     Cathy's decoder was continuous because the robot arms needs an entire, continuous trajectory to bring the coffee to her mouth. The video above is an example of a communication prosthesis, and it was a trick question! This particular communication prosthesis was continuous, you can see the cursor moving on the screen continuously, as well as discrete. However, a discrete decoder by itself would be sufficient for a communication prosthesis if we only care about the final key strike.

     There are bayesian decoders, population vector decoders, FORCE decoders, Wiener filers, Kalman filters, and so many other algorithms that can be used to perform both discrete and continuous decoding. But, we will save the details of those algorithms for future blog posts!

  4.  Finally, there need to be control signals that tell a combination of software and hardware systems what to do with our decoded neural populations! In Cathy's case it would be the signals that tell the robotic arm to pick up her coffee.

     Just look at how happy Cathy, as well as the researcher sitting behind her are that this motor prosthesis worked. It is heartwarming to see this awe-inspiring feat of neuroengineering tangibly help her.

 To recap: any hardware or software created to collect, compile, or decipher information from the brain would be a neuroengineering device that fits into this category. And there are a growing number of companies, some of which are legit and some that are just hype, that exist in this space of neuroengineering.

 Separating out companies based on the rigor of their work will be the topic of future blog posts, but to tease what is to come, see the figure below. Not all the endeavors at these companies is legit and I have by no means included all companies in this space, so keep those two things in mind! Picture of the neuropixel probe that is inserted in the brain. The probe is common for many neuroengineering studies.

 An important note: some devices that neither read nor write information out of the brain, like deep brain stimulators or transcranial magnetic stimulators, are sometimes referred to as neural prostheses. They are more appropriately characterized as neuromodulatory devices. Regardless, they can still be incredibly effective at treating some diseases like Parkinson's or Depression!

 Controlling neurons to investigate neural circuits largely deals with the phenomenon of optogenetics. Optogenetics is a method to induce or suppress neural activity by making neurons responsive to specific wavelengths of light. In other words, we can shine a red light on a neuron and make it fire an action potential or stop firing an action potential. So how does it work?

 It all started with Chlamydomonas reinhardtii (pronounced Clammy-dough-moan-us Re-in-harty), a single celled algae that can sense light through its eyespot with light sensitive proteins called opsins. We are going to call them Clammy for short. Picture of Chlamydomonas reinhardtii. Whose opsins were heavily used in optogenetics

 Channelrhodopsin-2 (ChR2), one of Clammy’s opsins, is activated in the presence of blue light and when activated allows positive ions to flow through it. When ChR2 is expressed in a neuron, and you shine blue light on that neuron, the ChR2 channel opens and allows positive ions to exchange across the neuron’s membrane. The most common ions to flow across a neuronal membrane are Na+, K+, and Ca2+. Na+ and Ca2+flow into the neuron, K+ flows out of the neuron. This ion flow causes the neuron to fire an action potential from light alone!

 Since Clammy made their debut, many more opsins have been discovered that both promote and suppress neural firing. Some common opto-inhibitors are archeorhosopsins, bacteriophodopsins, and proteorhodopsins. These opsins pump H+ out of the neuron, thereby hyperpolarizing the neuron. Hyperpolarization makes the neuron's membrane potential more negative and decreases the chance that another action potential will fire.

 Now that we know about both classes of opsins, the excitatory and inhibitory, let's look at an example of one in action.

 In the video above, halorhodopsin is expressed in the C. elegans' motor neurons. Halorhodopsin is activated by yellow light and inhibits the neurons by allowing Cl- into the cell. Therefore, when the yellow light turns on, indicated by the yellow circle in the bottom left of the screen, the halorhodopsin is activated, the motor neurons are inhibited, and the C. elegan cannot move.

 It is amazing that there are so many inhibitory and excitatory opsins that control neurons through so many ions! But, how do we get a neuron to express an opsin?

 The most common methods to introduce opsins into neurons are to use viral vectors or to create transgenic animals. I won’t dive into those details in this blog post, because for now it is more important to spend time on why optogentics is revolutionary.

 This feat of neuroengineering allows researchers to literally control which neurons fire, when those neurons fire, and for how long those neurons fire. Considering the fact that “neurons that fire together wire together”, optogenetics gives neuroscientists a tool to directly and fairly specifically control our neural circuits.

 For example, imagine that every time you ate broccoli you stimulated your neural circuit for pleasure and felt the most elated sensation that you have ever felt in your life. You would probably eat a lot more broccoli! But all jokes aside, it is a little scary that there are a limitless number of behaviors that could be controlled with optogentic or similar methods. We will get into the similar methods in a few paragraphs.

 But how would this work in a human? Because unlike C. elegans, we are not see through. The mouse below has an optical fiber implanted in its head that can trigger ChR2 expressing neurons, and is the poster mouse of the optogenetics research center. Meaning this mouse is likely one of the most famouse mice in the world.
 Optogenetics, has been a revolutionary tool for the field of neuroengineering and neuroscience because it allows you to perturb neural systems in more causal manner than was ever possible before. In addition, it ushered in experimentation for other neuronal control mechanisms like sonogenetics, controlling neurons with sound, and magnetogenetics, controlling neurons with magnets.

 Some argue that sonogenetics, magnetogenetics, and the many other spin-offs are more desirable for humans than optogenetic approaches. Can you guess why?

 Well I don't know about you, but I don't really want an optic fiber inserted into my head even if it could control my neurons. This complaint hits at one of the largest issues, it is hard to expose deep brain areas to light non-invasively. It is probably easier to target those deep brain areas with ultrasound or magnetic fields.

 Much like our other sub-fields of neuroengineering there are budding companies and efforts in this space. For now, I will refrain from commenting on the legitimacy of the technologies at each company and save it for future blog posts! However, some players are Circuit Therapeutics, Bruker, Inscopix, etc.

 Using the brain to inform the algorithms and architectures used in hardware or software designs is another field of neuroengineering. In this field, engineers aim to create more capable and efficient hardware and software by deriving inspiration from neuroscience.

 Let's start with neuroengineering in hardware! One of the most prevalent instantiations of neuroengineering in hardware is neuromorphic computing. Neuromorphic chips are hardware that utilize both analog and digital components to more realistically simulate the nervous system. Pictured below is Neurogrid, a type of neuromorphic chip.
 You might ask yourself, why do we need both analog and digital components on the chip? The main reason is that our CNS and PNS operate through both continuous (analog) and discrete (digital) communications. Therefore, to capture the manner in which the brain communicates, we must include both components in our hardware.

 A digital form of biological communication might seem difficult to achieve, but it does exist. Action potentials are “all or nothing” events and are a form of the nervous system’s discrete communication. Below is a cartoon mock up of an action potential!


 On the other hand, an analog method of communication in the nervous system is the release of neurotransmitters into the synaptic cleft. In reality neurotransmitters are discrete molecules, but at our level of resolution, they can be accurately approximated to continuous values. This is incredibly similar to the analog signals in electrical engineering, because any analog signal can be thought of as a summation of infinitely narrow discrete signals.

 A tricky neural property to implement that is neither truly discrete nor analog, is the connections within and between brain regions. Although these connections either exist or do not, their connectivity strength can vary dramatically. There is no straightforward answer to fully encapsulate this property in a hardware system.

 It will be helpful to ground this explanation in an example, so we are going to explore the implementation of Neurogrid. Neurogrid’s aim is to accurately simulate the dynamics of neural populations in hardware. To capture the properties of a neuron, Neurogrid had to recreate a neuron’s axonal arbors, synapses, and dendritic trees.

 Since hardware has limitations, and we want to ideally simulate hundreds of thousands if not millions of neurons, energy efficiency and the amount of noise inherent to the circuit has to be taken into account. Given these constraints, the axonal arbor was implemented digitally, and the synapse and dendritic tree were implemented with analog components.

To connect neurons to one another, Neurogrid implemented:

  • A spike-routing tree network that has low latency. This means that information packets can flow quickly from neuron to neuron.

  • An architecture with low bandwidth, meaning that not many information packets can flow. This is not ideal, but we will get to why it is acceptable soon.

  • A setup containing deadlock-free communication, which allows you to send one information packet to many destinations.

 Although having low bandwidth is not ideal, having the ability to send one information packet to many destinations is essential for modeling a neuron! It is a necessity because, in real life neurons can send their signals to many other neurons via their axonal arbors.

 These design decisions enabled Neurogrid to model one million neurons with billions of synaptic connections in real time! That is pretty impressive for silicon, even if it still is inferior to carbon.

 Now what about neuroengineering in software? I’m sure you have heard about artificial intelligence, machine learning, or deep learning by now! Well within deep learning, some classes of algorithms such as neural networks, have heavily borrowed principles from neuroscience. A visualization of a neural network is shown below! Does it maybe look like a bunch of interconnected neurons?

 Neural networks are models that perform layer based computations. And there are many kinds of neural networks such as feedforward, convolutional, recurrent, and the list goes on! In this article we are going to focus on the biological analogs within neural networks rather than exploring exactly how they work.

 Each circle in the image above can be thought of as a neuron, in the loose sense that each circle performs a computation. The lines between each "neuron" can be thought of as axons and dendrites because they propagate weighted signals between the computational units. Within a neural network, the weights change during training to optimize the prediction on a specific task. The analog here is that the dendritic and axonal weights change in the brain based on our experiences.

 Those are a few examples of how neural networks loosely resemble the nervous system. And interestingly enough, the principles borrowed have led to dramatically improved performance on tasks such as the classic video game, Atari! The sharing between the two field has even been dubbed a virtuous circle. Thus far, however, information has flown mostly from neuroscience to software.

 Exceptions to our categories. Within this article I have alluded to the presence of other devices and fields within neuroengineering, like neuromodulatory devices. By no means have we explored everything created or existing in the field of neuroengineering. However, the technologies listed here are some of most promising and rigorous in the field.

Final Thoughts

 Neuroengineering is a dynamic and highly interdisciplinary field that aims to understand, mimic, control, and ultimately master the brain. These lofty goals are transpiring quickly making it a more exciting time than ever for the space.

  In future blog posts we will dive further into the details behind each of these sub-fields and the main players in them. We have barely scratched the surface!

  In this article I referenced: [1] Neural prosthetic work from Brown University, [2] Neural prosthetic work from Stanford University, [3] Neural interface work from CTRL -labs, [4] Neuroprosthetic work for the neuropixel probe, [5] Optogenetics work from Stanford University, [6] Neuromorphic computing work from Stanford University, [7] Work from 3blue1brown, and [8] work from DeepMind.


If you like my content let me know, by tweeting or subscribing below!


Get Blog Posts Directly to Your Email!