Challenge

In a nutshell

Train an assigned, unmutable, model to classify land patches on-board the European Space Agency's OPS-SAT. Use only a few labelled images in a larger dataset and leverage quantization-aware training methods. Win and run your model on-board ESA's spacecraft OPS-SAT.

Background

OPS-SAT is a European Space Agency satellite devoted to demonstrating drastically improved mission control capabilities that will arise when satellites can fly more powerful on-board computers. The satellite is only 30cm high, but it contains an experimental computer ten times more powerful than any current ESA spacecraft at the time of its launch in December 2019.

The heart of the OPS-SAT satellite payload is the processing platform, which is responsible for providing a reconfigurable environment able to fulfil the objectives of each experiment. The processing platform runs a Linux-based operating system which consists of a flexible and reconfigurable framework, featuring sophisticated processing capabilities, interfaces, memory integrity and reconfigurable logic.

The platform consists of an Altera Cyclone V SoC with an ARM dual-core Cortex-A9 MPCore and a Cyclone V FPGA. 

A data-centric Space ML competition 

“Most competitions hold the Data fixed and let teams work on the Code. Thinking of organizing something where we hold the Code fixed and ask teams to work on the Data. Hoping this will more closely reflect ML application practice and also spur innovative research on data-centric AI development.”

- Dr. Andrew Ng, head of Google Brain, May 24^th 2021

The above quote from one of the most accredited ML scientists is particularly interesting for on the edge Space ML efforts, since the possibility to leave the on-board software untouched to only upload / learn new model parameters has several advantages. To start with, all constraints coming from the satellite hardware platform, such as power or memory consumption are satisfied and need no further efforts. Such a setup is thus potentially reflecting future advanced ML practices on the edge of space systems.

In "the OPS-SAT case" competition we thus decided to try this innovative angle to organize the first data-centric Kelvins competition. Taking into account the OPS-SAT satellite platform constraints, we have chosen a ML learning model able to classify image tiles coming from the on-board sensor. The model was chosen so that inferences can be made with the requested speed, using the available memory, power and computational environment. We created a dataset from the on-board camera sensor, consisting of several raw and unlabeled images. We cropped those images in patches/tiles and classified each of them to one out of eight classes. To enforce the few-shot and semi-supervised learning paradigm, only 10 labelled examples for each of the classes are provided to you. The restrictions on the hardware and software in space are definitely tough and no longer under your control. All you have is the data. Is this enough for you to solve the OPS-SAT case?

Goal

Using the few-shots available, (i.e. labelled images) and the provided dataset, train the weights of an EfficientNet-Lite0 image classifier so that its float16 quantized version reaches the highest possible classification accuracy (as measured by the Cohen’s Kappa Coefficient).

The most succesful models will be run during a dedicated flight campaign, taking place after the competition end, on board OPS-SAT and tested to provide real-time, coarse segmentation by classification of 200x200 pixel tiles from the on-board sensor.

References

[1] Derksen, D., Meoni, G., Lecuyer, G., Mergy, A., Märtens, M. and Izzo, D. Few-Shot Image Classification Challenge On-Board. NEURIPS2021 [pdf]

[2] Evans, David J. "Ops-sat: Operational concept for esa’s first mission dedicated to operational technology." 14th International Conference on Space Operations. 2016. [pdf]

[3] Labrèche, G., Evans, D., Marszk, D., Mladenov, T., Shiradhonkar, V., Soto, T., & Zelenevskiy, V. (2022). OPS-SAT Spacecraft Autonomy with TensorFlow Lite, Unsupervised Learning, and Online Machine Learning. 2022 IEEE Aerospace Conference. [pdf]

[4] Evans, D., Labrèche, G., Mladenov, T., Marszk, D., Shiradhonkar, V., & Zelenevskiy, V. (2022). Agile Development and Rapid Prototyping in a Flying Mission with Open-Source Software Reuse On-Board the OPS-SAT Spacecraft. AIAA SciTech Forum 2022. [pdf]