How streaming HD video from space enables future missions
As the launch of the James Webb Space Telescope proved, streaming HD video from space is going to be hugely popular with the public and important for engineers responsible for future missions. (Image source: ESA)
The world watched on Christmas Day 2021 as the National Aeronautics and Space Administration (NASA) in partnership with the European Space Agency (ESA) and the Canadian Space Agency (CSA), launched the James Webb Space Telescope (JWST) from Europe’s space port in French Guiana.
The telescope, reported to cost $10 billion, is the most ambitious space telescope mission to date.
And in another world first, the whole thing was captured in high-definition and streamed live to the control center as it happened. It was achieved using a vision system developed by Réaltra. The specialist contractor was tasked, late on in the project, with building what could possibly be the world’s most expensive disposable camera.
“We were the first company ever to do a full HD transmission over an RF link in space,” said Michael Martin, engineering manager at Réaltra Space Systems Engineering.
Building the world’s most expensive disposable camera
A division of Realtime Technologies Ltd., Réaltra is based in Dublin, Ireland. Réaltra is the Irish word for galaxy, and Realtime Technologies has manufactured, integrated and tested the electronic boards for the data acquisition systems for a wide range of past and future space missions since 2006. Its mission log includes SpaceX Falcon 9, Vega-C and Ariane 5. It is also contracted to provide a vision system for Ariane 6.
In fact, the system used to capture the deployment of the JWST was initially intended for the Ariane 6. Due to the increased interest in space coming from the public, ESA spotted an opportunity and value in filming the deployment as it happened. ESA asked Réaltra to step up its development program and provide an imaging system for the JWST launch vehicle. That gave the team at Réaltra a small number of months to build, test and install the system.
The solution Réaltra developed is called the RLT-VIKI-x, or VIKI for short. This is a space-qualified independent video telemetry kit for launchers and has been designed to operate as either an integrated part of the vehicle or, importantly, as a totally autonomous modular system. The autonomy was important for the JWST mission because it wasn’t possible to integrate the vision kit into the launch rocket’s systems. The entire system had to function as a standalone solution.
VIKI includes everything needed to capture, process and transmit video, including its own battery. And although it wasn’t integrated into the rocket’s mission systems, it still had to be qualified for space. That meant making sure it could withstand the enormous physical strain of takeoff and designing it to be radiation tolerant. As well as the short timeline, Réaltra also had to contend with developing and testing the system under the restrictions imposed by a global pandemic.
ESA was keen to capture as much of the mission as possible, not just the deployment of the JWST. The value in the data generated extends beyond public interest. The scientific value of the footage captured will far surpass the initial 15 minutes of fame.
The RLT-VIKI vision system developed to capture the space launch of the James Webb Space Telescope was the first vision system to stream HD video live from space. (Image source: Realtra)
According to Réaltra’s Michael Martin, the request was to provide two cameras. Each camera is based on a 2.1 meg pixel image sensor able to deliver 1080p/30fps or 720p/60fps. The pixels have a 12-bit depth and the CMOS 1/3in sensor has a rolling shutter and a wide dynamic range.
VIKI can support a total of six cameras, each individually controlled over an IP link. For the future Ariane 6 mission, Réaltra will put three cameras on the outside of the rocket to view the separation stages, one camera trained on the Vinci rocket (which will be the first mission for the new rocket) and three cameras in the payload.
Each camera has on-board processing and an H.265 compression engine. The cameras are connected to the control unit, which integrates an Ethernet switch for selecting and controlling each camera. The control unit also provides the telemetry, combining the feeds from all cameras and transmitting it as a single, compressed low bit-rate data stream. According to Martin, that could be as low as 125 kbit/s, and the high-definition (HD) video feed from the JWST was around 500 kbit/s.
Martin explained that Réaltra is the only company now using H.265 compression in space. “H.264 is the one normally used, but with H.265 we get 40% more compression,” Martin added. Because the system is built using commercial off-the-shelf (COTS) modules, Martin’s team was able to choose a solution that integrated the most aggressive compression technology available.
The image data is arranged into frames, conforming to the Consultative Committee for Space Data Systems (CCSDS) format. The CCSDS is a service-oriented architecture designed to make it easier to share data across mission systems. Like the use of COTS (see below), CCSDS is a further effort to minimize the cost of space missions. The entire vision system is designed to operate autonomously but can be controlled remotely using simple commands such as turn on/off and start/stop sending.
The modular system is managed by a power distribution unit that includes over/under voltage/current monitoring and protection, as well as a reset function to combat latch-up. This would allow the system to be rebooted if an event caused one or more modules to lock up.
(Trans)mission accomplished
Réaltra Chief Commercial Officer Danny Gleeson explained the importance of seeing the JWST separate in high definition to the people closest to the mission. Those in the control hub were almost moved to tears as they watched the live stream of JWST leaving the Ariane 5. But, as Gleeson said, the importance reaches all stakeholders and the wider community.
“Everyone was reminded that the visual record is really important, from a communications point of view and for their customers,” said Gleeson. “And for ESA it’s very important for public outreach and getting the public onboard. ESA survives on public money from all the member states that fund ESA.”
ESA has various receiving stations around the globe. As the rocket moves across its flight path, it disconnects and reconnects with the receiving stations on that path. “It’s like the way your mobile phone hops between base stations,” Gleeson explained. The received signal was fed back over a ground-based link to the main control station in Kourou. That signal was corrupted in the live feed, but the actual video signal received from the rocket was uninterrupted.
“Because it was happening in real time and everyone wanted to see the images, they just showed what they got,” Gleeson said. “That included the outages because of the ground transmission, but the actual space-to-ground transmission was perfect.”
COTS for radiation-tolerant systems
Commercial off-the-shelf (COTS) is now a major part of aerospace in general and Réaltra in particular. Commercial components have been used for space for decades, but the COTS philosophy relates more specifically to equipment. Using commercial equipment, modified for space, is the way to keep development costs under control. It is important to understand how this term has changed within the space industry over the years, as Gleeson explained.
“In the space sector and particularly in agencies like ESA, they would have regarded COTS in the past as a piece of space equipment that had been designed for one mission but is then available for a subsequent mission. It was on the shelf, but it was already designed for space. Commercial space companies have realized there’s a lot of technology available, designed for other industries such as automotive, medical and so on, that could potentially be used in space.”
Réaltra has a four-step process for using COTS in space missions, as it relates to equipment, not components. That means boards and sub-assemblies that were originally designed for a terrestrial application. The process starts with evaluation. The equipment is put into a space environment, such as a vacuum and environmental chamber, as well as exposing it to the kind of radiation it will encounter in space. That allows it to be characterized under space conditions.
The second step is to do something called mission mapping. Every mission will have different characteristics, based on where in space it will be operating. It may be low earth orbit, a lunar mission, deep space or a launcher. Those conditions are documented and can be applied to the equipment to determine if, as a baseline, it will be able to survive.
Once the basic criteria have been checked, the equipment will still need to be adapted to the mission’s specific requirements. That may involve changing the housing, connectors or mounting. The final step is qualification. Once all the above is known and planned for, qualification should hold no surprises, but testing is perhaps the most important and crucial phase of the process.
Using COTS equipment lowers the overall mission cost. It is usual for COTS to reduce the cost by a factor of two, but in some cases, it could be a factor of 10. Cutting out the non-recurring (design) costs for equipment can be a big part of that cost reduction. The recurring costs can also be lower, based on the experience gained from going through the process for different pieces of equipment. This is the process Réaltra followed, using COTS image sensor modules to create the VIKI system.
Realtra’s engineers were unable to travel to French Guiana to install the image system, so they had to remotely train other engineers onsite to fit it themselves. (Image source: Realtra)
Martin, whose background is in deep space, explained that it is relatively more difficult to use COTS equipment in deep space missions, because the radiation levels are so much higher. However, COTS is entirely feasible for low earth orbit or launch missions, as Réaltra proved.
Making COTS equipment radiation tolerant typically requires shielding. The trade-off here is that shielding adds mass to the payload. The cost of putting anything into space is largely related to its mass. If it weighs more, it costs more to launch. The amount of mass that can be dedicated to shielding will be the trade-off. At some point it will be cheaper to design the equipment using radiation-hardened components and reduce the amount of shielding needed.
Another important design technique is to protect the COTS equipment from radiation by monitoring radiation levels present and shutting the equipment down when the levels are too high. This active protection would not be needed in a benign environment and is an example of how engineers designing for space need to think around the problem.
As Martin explained: “We tried to bury the sensors as far inside the housing as possible. We also did a lot of calculations on the radiation, so we had the environment very well characterized from an analysis point of view.” This allowed the team to estimate the chance of a radiation “hit.”
A lot of data is publicly available that describes the kind of radiation levels to expect in any given mission. This is based on the orbit and position, but also the prevailing conditions around the solar system. This provides crucial information about how many radioactive particles might hit your object at any point in the mission. Testing against these models is how companies like Réaltra can develop radiation-tolerant systems based on COTS equipment. Probability and quantifiable risk provide the basis for the design decisions made.
High-speed decompression, outgassing and testing
Operating in a vacuum is another design consideration. One of the main challenges here is outgassing, even on a relatively short mission such as launching the JWST. In this case, it is the very rapid change from atmosphere to vacuum that needs to be considered.
Perhaps the most important part of the entire design process is testing. Martin’s comment was that the qualification process should be a formality. What he means by that is, if everything is tested thoroughly throughout the process, formal qualification testing should simply be a validation and not present any problems.
Due to the pandemic, the team had to train other engineers to do the integration in French Guiana. Environmental testing was also a challenge, because the engineers couldn’t take the system to their usual test locations in Germany or the U.S. The workaround for that was to use local test facilities. The problem was, there were no space test facilities in Ireland.
In an incredible collaborative effort, a small ecosystem of companies in Ireland worked together to create the test facilities needed. They had to meet the levels demanded by NASA, ESA and other partners and it has created a legacy of test capabilities in Ireland that simply didn’t exist before this project. If there are any positive outcomes of the pandemic, that surely must be one of them.
Using AI in space
Soon, vision systems will not just be used to passively relay information about things that are happening. They will become an active part of the control system, using AI to understand what is happening as it happens, and triggering actions based on that information. The same thing is happening here on Earth in the Industrial IoT, the automotive industry and elsewhere. But the need for autonomous operation is, arguably, greater when the equipment is physically so remote.
One example that Gleeson gave was the use of autonomous space robots, tasked with seeking out and capturing space debris. Réaltra is currently working on a module that integrates AI for use in its products designed for space.
Martin also explained how machine learning could be used in the days before a launch, after the cameras have been fitted, to learn and understand what the payload looks like and how it is expected to behave. It would then use AI during the mission to identify changes and what they may mean.
Where no camera has gone before
There is no doubt that streaming HD video from space missions is going to be hugely popular with the public and even more important for the engineers responsible for future missions. The information gleaned from the JWST launch is only just being revealed, but ESA and Arianespace can expect to get even more useful data from the Ariane 6 mission when it launches, powered by the brand-new Vinci rocket.
Using a COTS approach to designing for space is only going to get better. The systems now used in the industrial, medical and automotive sectors are evolving rapidly. The use of AI in vision systems is well established and that will inevitably transfer to space missions. This exponential improvement in the capability of space mission equipment will deliver huge insights for all humankind.
Follow Avnet Silica on LinkedIn
About Author
Philip Ling
Philip Ling is a senior technology writer with Avnet. He holds a post-graduate diploma in Advanced M...
silica content library/articles/how-streaming-hd-video-from-space-enables-future-missions
How streaming HD video from space enables future missions | Avnet Silica
