Teaching a network to fill in missing signals

Paper

Weighted Gate Layer Autoencoders

Venue

IEEE Transactions on Cybernetics

Year

2021

DOI

10.1109/TCYB.2021.3049583

Code & data

IEEE Code Ocean capsule

The problem

When a channel fails mid-recording

Many important measurements are recorded across many channels at once. An EEG headset records dozens of electrodes; an aircraft, a production line, or a robot combines readings from banks of sensors. In all of these settings, a common problem occurs: partway through a recording, a channel stops working. An electrode loses contact, a cable is knocked, or a sensor drifts out of range.

When this happens, the usual options are unsatisfying. Discarding the whole recording wastes hours of collected data. Filling in the missing channel with a fixed interpolation formula ignores what the data itself could tell us about how that particular channel behaves.

The idea

Training the network to expect missing inputs

An autoencoder is a neural network that learns to reproduce its own input. It passes the data through a narrow internal layer and reconstructs it, which forces it to learn the structure that makes the data predictable. We add a gate layer in front of this network. During training, the gates close at random and switch off individual input channels, but the network is still required to reconstruct every channel at the output, including the ones it did not receive.

One gate is closed: the network receives four of the five inputs but must still produce all five outputs. To do this, it has to learn how the channels relate to one another.

A weighting mechanism makes this harder in a useful way: errors on the switched-off channels count for more, so the model focuses on recovering the missing values rather than copying the ones it can already see. By the time training finishes, the model has practised working with missing channels many times over.

What we found

Degrading gradually instead of failing

The benefit appears at test time. When a channel drops out, the gated model reconstructs it from the remaining channels, and the reconstruction follows the shape of the true signal rather than a smoothed average. As more channels are removed, the gated model’s reconstruction error rises only slightly, while a standard autoencoder’s error rises steeply. With ten channels missing at once, the standard model has largely stopped working, while the gated model still produces useful output.

Left: a channel fails partway through a recording. Right: the gated autoencoder reconstructs it from the other channels; the inset shows the error rising only slightly as up to ten channels are removed.

Who should care

Where this is useful

We developed this method with EEG in mind, because brain-computer interfaces often lose electrodes during a session, but the idea is not specific to EEG. Wearable devices, industrial monitoring, and autonomous systems all combine multiple sensors and face the same failure, and the same approach applies. The gate layer is also useful for analysis: because the model learns which channels can substitute for which, it shows the dependency structure within the data.

This work is part of a longer line of research. The original gate-layer idea was published at IJCNN in 2019. A later paper in Sensors (2025), led by Kangjing Li, a postdoctoral researcher who worked on my project under my supervision, studies how the order in which channels are gated can follow the brain’s electrode layout.

Cite & explore

The formal version

H. El-Fiqi, M. Wang, K. Kasmarik, A. Bezerianos, K. C. Tan and H. A. Abbass, “Weighted Gate Layer Autoencoders,” IEEE Transactions on Cybernetics, vol. 52, no. 8, pp. 7242–7253, 2022. doi:10.1109/TCYB.2021.3049583

Run the code and data on Code Ocean →
Back to the paper gallery →