Ghosts in the machine

Accident investigation has embraced the digital age with cameras, phones and tracking systems as witnesses

A seaplane carrying tourists dives into a narrow bay, a helicopter disappears in a storm, an aerial firefighting tanker flies into the ground. None of these aircraft carried or were required to carry working flight data recorders (FDR) or cockpit voice recorders (CVR). But among the debris of these crashes—or recording them from the ground—were electronic devices that are allowing investigators to solve the mystery of their crashes and prevent similar accidents in future.

On a drizzly Canberra winter day, the Australian Transport Safety Bureau’s (ATSB) headquarters in the city centre can seem a particularly sombre place. In what resembles a bank building are technical facilities where investigators analyse smashed, burned or twisted items that used to be airframes, engines and instruments.

The recorders room, with its anechoic padding on the walls, has the sacred, solemn and claustrophobic atmosphere of a cathedral crypt. This is where high-fidelity audio speakers reveal the testimony of cockpit voice recordings in multichannel sound. Audio analysis can identify the sounds of alarms and switches being moved and allow engine and propeller speeds to be calculated from their audio signatures. Data analysis can identify the location, speed and attitude of an aircraft during an unusual event. These are the traditional techniques of accident analysis, refined to high levels of accuracy and insight, but powerless if the aircraft does not carry an FDR or CVR.

Technology bears witness to disaster more accurately than even the most expert human observation ever could.

New investigation techniques are developing as quickly as the technologies that make them possible. The toughness of silicon chips and their prevalence in modern navigation and communication equipment, whether handheld or panel mounted, and their capacity to retain data allow records to be extracted from wreckage. And the three-part combination of high-resolution imaging, pervasive use of dashcams and camera phones, and computing power first developed for the entertainment industry, means technology bears witness to disaster more accurately than even the most expert human observation ever could.

Recovery of microelectronics from aircraft wreckage became a common technique in the second decade of this century. One of the earliest cases was the investigation into the fatal crash of a CT-4 aircraft of the Royal New Zealand Air Force in 2010. As a military aircraft, the CT-4 did not have an FDR or CVR, but it did have an airframe fatigue monitoring recorder, which was not crash or fire protected to the same extent, and had been damaged by the impact and fire. The ATSB extracted data from this device to assist the RNZAF inquiry. It revealed the aircraft had departed from controlled flight during a ‘fishtail pass’ involving large rudder inputs, which had been flown at low airspeed and height. The process pioneered what has become a widespread technique in digital crash investigation—removing the computer chip from the damaged device and installing it in either a similar device or a universal data reading electronic chassis.

Three recent tragedies have shown how important digital reconstruction has become to accident investigation.

de Havilland Canada DHC-2 Beaver, Jerusalem Bay, NSW, 31 December 2017

After lunch at a waterside restaurant on a remote bay, tourists departed Cottage Point on a charter flight to Rose Bay in Sydney.

The standard flight path was to climb to the north then turn right along Cowan Creek toward the main channel of the Hawkesbury River, until the seaplane gained sufficient altitude to fly above terrain and return to Rose Bay. The aircraft was seen instead to enter Jerusalem Bay below the height of the surrounding terrain and left of its intended flight path. Shortly after entering the bay, witnesses reported seeing the aircraft suddenly enter a steep right turn and dive into the water.

Bell UH-1H, near Anna Bay, NSW, 6 September 2019

A privately operated ex-military Bell ‘Huey’ helicopter disappeared on a flight from Archerfield, Qld, to Bankstown, NSW. The aircraft was last seen on radar making a turn out to sea over Anna Bay, near Newcastle, NSW. Weather at the time was cloudy, windy and rainy. Two attempts by the Williamtown Approach controller to contact the pilot were unsuccessful. Five people died in the crash, which was not seen by anyone else.

Lockheed EC130Q, near Peak View, NSW, 23 January 2020

An aerial firefighting tanker was dropping retardant on a bushfire near Cooma, NSW. After the retardant drop, firefighters on the ground saw the aircraft bank left, before becoming obscured by smoke after a few seconds. Soon after this, the aircraft was seen flying at a very low height above the ground, left wing down. A few seconds later, the aircraft collided with terrain in a fiery crash. Its three crew were killed.

When the chips are down: flightpaths from photographs

When the Beaver crashed in northern Sydney on the last day of 2017, it seemed there would be few data clues available to the ATSB. The aircraft carried no CVR or FDR, its 1940s Pratt & Whitney Wasp Junior radial engine was made long before the invention of full authority digital engine control (FADEC) and its instruments were all mechanical gauges. But it had been a tourist flight. ATSB investigator Duncan Bosworth remembers how the investigation quickly focused on the passengers’ cameras and phones.

The ATSB’s analysis was able to narrow the initial search area from 20 by 20 nautical miles to just one square kilometre.

‘We knew the passenger in the front seat had “a big camera”, which ended up being a digital SLR,’ Bosworth says. The camera, immersed in mud and water, was retrieved from the wreckage and its compact flash (CF) memory card became the starting point of an investigation within an investigation. ‘It was a pretty old CF card, about three years, not one you could still purchase,’ Bosworth says.

The card had evidence of water damage to the circuit board and corrosion on its components. Investigators cleaned it, placed it in an oven for several days to dry and stored it in argon gas to prevent further deterioration. They X-rayed the card to discern how damaged it was and how best to recover its data. The card manufacturer was unwilling to help, so the investigators went hunting for similar cards to use as a donor motherboard for the damaged card’s silicon chips.

By chance following a call-out for assistance, a suitable donor card—similar but not quite identical—was found in a desk drawer at another government agency. Investigators removed the chips and began the painstaking process of transferring the five chips from the damaged card to the donor card. This involved using a microscope to help place 144 solder attachment points on an area considerably smaller than a 10-cent coin. ‘You can’t have too much coffee that morning,’ Bosworth jokes, with more than a hint of truth.

Heating and cooling of the chips during removal and donation had to be carefully controlled to ensure no additional damage to its encoded memories. After five chips were successfully attached, the card was placed in a read-only forensic device and 412 images (including deleted images) emerged, 20 of them from the accident flight. The pictures were time-synced with images from a mobile phone recovered from the wreckage. There was no other recorded information on the aircraft.

NSW Police recreated the camera’s photos from a Polair helicopter, using the same type of camera and lens settings. This gave approximate three-dimensional coordinates of where they were taken. More recently the ATSB used SynthEyes, a technology originally developed for movie special effects, that precisely calculates the position of the camera and objects in the image (see breakout below). ‘The maths behind it works well for investigations,’ ATSB investigator Paul Ballard says. ‘If you can scale properly, based on identifiable ground features, you can get an accurate path of objects in the frame. The more shots the more accurately it can calculate.’

There were several witnesses to the accident but their testimony, like all human memory, was variable, contradictory and inconsistent. But witnesses who had taken pictures brought a useful gift to the investigation.

‘We had witnesses who recalled the pilot took off from Cottage Point and then turned left, instead of right—but how could a very experienced pilot who knew the area so well do that?’ Bosworth says.

The release of the investigation’s preliminary report brought to light someone else on the river who had photos of the aircraft in a right-hand turn. It seems the aircraft did not turn left but continued to the right before turning into Jerusalem Bay. (Update: the final report was issued in January 2021 and was covered by flightsafetyaustralia.com)

Paul Ballard was able to use the witness and passenger images to construct a credible flight path. ‘I could use SynthEyes to track ground features in the images that could be referenced to Google Earth, to work out where the aircraft was.’

When no digital devices are onboard an aircraft involved in a crash, investigators can still find clues from ground-based imagery. For instance, the Lockheed Hercules that crashed in January while fighting Australia’s disastrous bushfires did have a cockpit voice recorder, but when ATSB investigators played it in the vault, they heard voices from a routine flight months before the crash. However, for this accident, the ATSB is again using witness video footage and SynthEyes to more accurately derive the aircraft’s flight path.

A digital consensus: radar, devices and tracking

Had it happened last century, the night-time disappearance of a Bell UH-1 helicopter into the sea off Newcastle would probably have become one of Australian aviation’s enduring mysteries. But the combination of ADS-B and data from devices on the aircraft meant investigators knew the aircraft’s flight path with reasonable accuracy within days.

ATSB investigator Alex Talberg says, ‘We had the ADS-B data from Airservices, we had ADS-B data which we initially accessed using [aircraft tracking app] Flightradar24, and we also had an iPhone and an iPad running OzRunways’.

This meant investigators had several independent means of reconstructing the helicopter’s flight path. But there were also complexities.

‘OzRunways data uses GPS on your phone, sending a packet via the 3G network,’ Talberg says. ‘It’s only as accurate as your phone and network. That’s pretty accurate most of the time but does not handle high-G situations. It’s also predictive to some degree, averaging and filtering as it goes.’

The investigation created a composite picture—where holes existed in one flightpath data source, another source was often able to patch them. There was, for example, a second source of ADS-B data available to investigators.

‘We reached out to Aireon, a space-based ADS-B company,’ Talberg says. ‘Aireon’s advantage is no gaps in coverage. A lot of aircraft have terrestrial ADS-B that beams downwards but, if they are over water, that provides a pretty good reflective surface and Aireon was able to provide satellite ADS-B data.’

Each set of data showed the aircraft turning as it descended to the sea. ‘All the data sources were telling us with some consistency where we should start looking,’ Talberg says.

The ATSB’s analysis was able to narrow the initial search area from 20 by 20 nautical miles to just one square kilometre.

‘They found it within a day, about 100 metres from where we said to start,’ Talberg says. Discovery of the wreckage helped the investigation and recovery of remains provided a small solace to their next of kin.

Flights, cameras, action

The new technology that makes it easier to determine the factors involved with aircraft accidents and incidents has extended the range and scope of ATSB investigations. Its position is aircraft owners should fit and maintain these systems. And you don’t have to have an accident to benefit from them.

‘Aircraft data recording systems can be a readily accessible tool for both flying training, maintenance and safety investigation,’ the ATSB said in its report onto the crash of a Bristell light sport aircraft in 2018. ‘Aircraft owners should make themselves aware of the data recording capability of their aircraft and ensure that the systems are fully functioning and backing up information.’

‘Aircraft data recording systems can be a readily accessible tool for both flying training, maintenance and safety investigation’

Investigators have been frustrated in the past to find functional devices in crashed aircraft that had nothing to reveal. ‘We see it less now, but 10 years ago we’d investigate aircraft with GPS, only to find the recording feature had been turned off,’ Bosworth says.

Redundancy, always a good principle in aviation safety, is both appropriate and achievable for microelectronic devices, ATSB investigators say. With recording technology small, light and relatively cheap, why stop at one?

The investigation of a Cirrus SR22 that crashed and caught fire at Orange, NSW, in May 2018 makes precisely this point. Cirrus aircraft have a semi-crash-protected recoverable data module (RDM) in the fin spar. In most crashes this would be a well-protected location but the RDM was damaged in the post-crash fire. However, the ATSB was able to extract identical data from an SD card in the aircraft’s Garmin G1000 electronic cockpit display.

Portable devices sometimes escape post-crash fire by being thrown clear. This happened in the crash of a television reporting helicopter at Lake Eyre in 2011. The ATSB was able to derive flight path information from a GPS unit.

Memory cards can retain information after exposure to temperatures of up to 500 degrees. ‘We know they are not going to be as crash protected as a flight recorder but we’ve discovered how we can get information from them, even when they are damaged,’ Ballard says. ‘So long as heat hasn’t physically damaged the chip, you have a chance of recovering data.’

There is no requirement to install a flight recorder on general aviation aircraft but several makers offer lightweight voice, data and video recorders for owners and operators who can see the advantages of recording and analysing flight information. Devices are available from makers including L3Harris, Appareo and New Zealand company Eye in the Sky, which makes a lightweight cockpit camera and recorder suitable for piston helicopters and aeroplanes.

*CASA has recently approved the installation of portable ADS-B devices in VFR aircraft. The first to be accepted as meeting technical standards is the uAvionix SkyEcho 2 ADS-B transceiver.