AI/ML related applications emerging within the space industry
Rapidly growing interest in the use of artificial intelligence (AI) in space exploration and commercialisation is being fuelled by the promise that it can improve the robustness and cut the cost of missions. A menagerie of AI algorithms, some based upon animal behaviours, is being pressed into service to help with tasks as diverse as satellite telemetry analysis, the interpretation of remote sensing data, vehicle autonomy, and even space systems design. In the future, there are opportunities for it to serve more sophisticated purposes, such as optimal constellation organisation and collision avoidance.
AI for data reduction
One common challenge in designing satellites is managing the flood of data that they are constantly producing - coming from diagnostic systems, experiments being undertaken, remote sensing arrays, etc. Since satellites are constrained in mass, volume, power consumption, bandwidth, and computing capacity, an important design goal is to extract as much information from this flood as possible without overwhelming their resources. In the case of monitoring and diagnostic systems, it is also important to do this in a timely manner. One critical design decision therefore becomes whether to process data on the satellite, or to send it back to a ground station. This decision turns on issues such as the available communications bandwidth and the energy needed to transmit data, the trade-off between the complexity of onboard data analysis and compute requirements, and the acceptable latency for any results. AI techniques, such as machine learning (ML), are being used to intelligently reduce data volumes, as well as to speed up data interpretation.
Telemetry analysis
Telemetry data, which describes the condition of a satellite and its subsystems, is usually made up of thousands of sensor data streams. These are expressed in many different units and output formats. The inherent heterogeneity of this data makes it difficult to combine values in a formula that meaningfully represents the satellite’s health.
One early response to the issue of satellite health monitoring, therefore, was to fix upper and lower limits for each sensor variable and set alarms for when they were breached. However, the complexity of modern satellites has made this an onerous task. The next step was to create adaptive limits, adjusted by an AI algorithm that predicted the upper and lower limits of each sensor measurement. A third approach was to use regression and classification techniques to predict the range of each variable, working from past data sets. This approach has been tried using real telemetry data from the Japanese Aerospace Exploration Agency, and the experiment showed that this could successfully detect different types of thruster failure.
Another way to find issues within telemetry data is to use ‘expert systems’, rule- and knowledge-based systems that encapsulate human understanding of how satellites work. There have been some successes, but expert systems cannot deal with anomalies that they have not been ‘taught’ about. A further approach is model-based diagnosis, in which a satellite is modelled in a computer. Live data from a real satellite is then compared with the simulation to reveal any divergences. The challenge here is building a complete-enough model of the satellite in the first place for it to be diagnostically useful in practice.
ML, an AI technique that ‘learns’ about an issue by being exposed to large training datasets, is also being applied to fault detection and isolation in space systems. ML is used to predict outputs given inputs, to classify input data, to associate one bit of data with another even if it has errors, and to conceptualise relationships between data. All these capabilities have been applied to telemetry datasets with varying levels of success. Deep learning techniques, which use complex neural networks, have also been applied to telemetry data in order to extract knowledge. Perhaps the simplest of these approaches has been to gather all the log files for a complex space system and then apply pattern extraction and clustering techniques to that text, then try to reveal insights.
Some researchers are exploiting ML’s ability to discover patterns within large datasets to infer failures in one part of a satellite’s systems from unrelated data generated elsewhere in the craft. If this is possible, the argument goes, perhaps some sensors are redundant: after all, why install a sensor to monitor a condition that you can infer, via ML, from other data? The issue with this approach is knowing which sensors are not needed before you have a dataset that can be analysed to reveal the hidden dependencies that make them redundant.
Analysis of remote sensing data
ML is a rich and rapidly developing discipline, with wide applicability and extensive research frontiers. These include learning transfer, which is already being applied to challenges such as the analysis of large amounts of remote sensing data. In research reported in 2019, a ’fuzzy C-means network’ was used to find and map urban slums pictured in high-resolution optical imagery from the QuickBird satellite. The resultant trained model was then applied to a less accurate dataset from a satellite called Sentinel-2 and was shown to improve the accuracy with which slums were recognised within its lower-resolution imagery. This ’teacher/pupil’ relationship shows great promise for reducing the operating cost of ML analytics - with expensive satellite imaging data being used to train a neural network and then apply the resultant trained network to less costly imagery.
Remote sensing data from satellites is already helping make Earthbound decisions, and again, the amount of data involved demands use of ML techniques to make it manageable. For example, in 2020, the European Space Agency (ESA) and the European Commission (EC) got together to create a ‘Rapid Action Coronavirus Earth Observation’ dashboard, using satellite data to track the impact of the COVID-19 virus. The dashboard integrates Earth observation data from the EU’s Copernicus Sentinel satellites and at least 30 other sources to monitor environmental issues - such as air and water quality, plus economic and human activities (including industry, shipping, construction, traffic, and agricultural productivity). The image below shows the impact of COVID-19 on nitrogen dioxide concentrations in southern Europe between 2019 and 2020.
Drawing together this kind of first-order data from many sources is useful in itself and provides a platform for employing AI techniques to extract deeper insights. Two examples were given at the launch of the dashboard. In one, AI techniques and commercial satellite data were combined to track production volumes at a car factory in Germany. In the second, similar techniques were utilised for tracking aircraft traffic in Barcelona airport. Both could help build up a picture of how economic activity is changing during the pandemic. The site now hosts a wide range of other live indicators and examples - from the number of trucks on Europe’s roads and how full the carpark is at Disneyland Paris, to the quality of the water in Venice’s lagoon.
Tracking nitrogen dioxide emissions as a proxy for economic activity during the pandemic (Source: ESA)
AI in space exploration
What is the role of AI in space exploration? It turns out that humans’ desire to explore beyond Earth is being enabled by AI algorithms that are based, in part, upon the behaviours of animals on the move. Techniques from this menagerie of swarming and evolutionary algorithms are being applied so as to overcome optimisation challenges in space exploration, especially those in which a task has multiple, possibly conflicting objectives.
In a paper submitted to IEEE Access, Paul Oche et al of the National Space Research and Development Agency of Nigeria listed some of these algorithms and applications. For example, ‘artificial immune systems’ can mimic natural behaviours such as:
- Clonal selection - A theory of how immune systems adapt to pathogens
- Negative selection - Using positive and negative selection to mature some types of immune cell.
- Dendritic cell algorithms - Which work at multiple scales at once.
- Immune network algorithms - Based on a theory of immune system self-regulation.
The resultant algorithms can be applied to fault diagnosis, clustering and classification, as well as robotics.
Bio-mimicking AI algorithms
Here are some of the algorithms that have emerged, and how they may be employed within a space context:
- ‘Genetic bee’ algorithms - A recent approach to optimisation that hybridises genetic algorithms with artificial bee colony approaches. These algorithms can tackle multi-objective layout optimisation and network optimisation tasks.
- ‘Chicken swarm’ algorithms These will, according to Oche et al, be of value in re-entry trajectory optimisation. They are modelled on the way that chickens self-organise into hierarchical groups, each with roosters, hens and chicks. The resultant swarm then optimises access to food for each group, and for the groups as a whole (displaying a form of collective intelligence that offers promise as an optimisation technique in other contexts).
- ‘Grasshopper optimisation’ algorithms - These are based on the theory that grasshoppers optimise the way they move in a swarm. This is done by both repelling each other (so they can explore the space they are operating in efficiently) and attracting each other (so that they can exploit promising regions). An algorithm based on this theory, which also steadily reduces the ‘comfort zone’ of the grasshoppers, is shown to be effective at quickly exploring problem spaces and converging towards effective solutions in some types of optimisation problem.
Other researchers have explored hybridising these kinds of bio-inspired algorithms with concepts from quantum computing and even chaos theory, to better address specific issues in space exploration.
Bringing autonomy to space-deployed hardware
Being able to solve complex optimisation problems is a key application of AI in robotics for space exploration. Surface-bound rovers need to be given long-term autonomy, so that they can manage their exploration tasks when beyond the reach of Earth-based controllers. There are two fundamental challenges in implementing this. The first is to give a robotic exploration vehicle the skills it needs to explore autonomously, such as planning, perception, navigation/mapping, interaction, and learning. The second is to make those skills robust enough to cope with evolving conditions and environments, as well as ageing issues in the craft itself.
Probably the best example of this kind of adaptability to date has been NASA’s Opportunity rover. This operated for more than 15 years and travelled more than 28 miles on the surface of Mars in 2018, before it stopped communicating with Earth after a dust storm. Its autonomous features included a mixed-initiative task planner known as MAPGEN, and an autonomous navigation system. Opportunity used the planner to set its own daily mission schedule, which was then reviewed and refined by terrestrial scientists.
The first drive of the Perseverance rover on Mars (Source: NASA)
The use of AI in NASA’s Mars mission has become increasingly sophisticated in later missions. The Mars 2020 mission, which delivered the Perseverance rover and Ingenuity helicopter to the planet’s surface, used a technique called terrain relative navigation to improve the chances of a successful landing. In previous landings, the rover estimated where it was relative to the ground based on radiometric data provided by the Deep Space Network. This approach gave a positional error on landing of one or two kilometers. For Perseverance, the team was able to create a map of the preferred landing site and store it in the rover’s computer. As the craft was descending by parachute, it took photos of the approaching landscape and compared its landmarks to those in the stored map. The descent vehicle then used this information to work out whether its current trajectory would take it to a known safe landing spot, or whether it should target another site for a safer landing. Positional estimation accuracy with this approach was to within 40m, and Perseverance was successfully delivered to Mars’ surface.
Future potential of AI in space industry
There are many other areas where the use of AI is starting to help address complex issues that would otherwise hold back space-related activities. For example, swarming algorithms, like the one described earlier, are being used to ensure the satellites in constellations keep their distance for each other, and act together in an orderly manner.
Conversely, multi-objective optimisation algorithms are being assessed for supporting space junk collection schemes, The idea is that the algorithms could be used to plan a path through a satellite constellation so that a single, autonomous debris-collection robot could gather up as much space junk as possible before pushing (itself and the junk) out of orbit.
Design prospects
AI techniques are also being applied to the design of space missions, using expert systems to act as ’design engineering assistants’. These will help generate initial design inputs, provide easy access to previous design decisions and promote the exploration of new design options. Such assistants, as envisaged by a team led by Audrey Berquand at the University of Strathclyde, could eventually take a role in tracking design iterations and flagging inconsistencies in any models.
The approach taken involves extracting implicit knowledge about relevant space system design, based on past experiences and insights, and storing it alongside explicit knowledge, such as past reports, publications, and data sheets, in a structured form known as a knowledge graph. This knowledge graph can then be interrogated by an inferencing engine - to provide reasoning and deductions about the knowledge it contains and the relationships between different items. A user interface allows designers to interrogate the knowledge base and review the results of their queries as interpreted by the inferencing engine.
A separate project by Hyunseung Band et al of Cornell University is developing Daphne, an intelligent assistant for designing the architecture of Earth-observing satellite systems. The team argues that designing systems such as these is difficult because of the amount of complexity and ambiguity involved and the need to marshal all this uncertainty and produce a creative solution to it. The user can ask Daphne questions about a design problem or request feedback on a specific design. Communication with the assistant is conducted via a web interface or in natural language.
Conclusion
There are many other examples of ways in which aspects of AI, developed in different contexts over the past 60 or 70 years, are now being adapted to accelerate AI uptake in space technology. What is important for electronics designers to understand now is that implementing AI techniques will likely involve new approaches to the partitioning of systems. It will also require the making of new trade-offs between where computation is carried out, the computing power needed, the available communications bandwidth, ongoing power budgets, and many other factors. Perhaps someone will develop an AI assistant to help manage this sort of complexity.
Follow Avnet Silica on LinkedIn
About Author
Paul Leys
Paul Leys is the Market Segment Manager for the Aerospace and Commercial Avionics business at Avnet ...
silica content library/articles/applications-of-artificial-intelligence-in-the-space-industry
Applications of artificial intelligence in the space industry | Avnet Silica
