S&T is guiding responders through darkness with real-time indoor visualization technology.
Trying to find your way through an unfamiliar building filled with dark, dense smoke while lives are on the line is not an ideal work environment, but for countless firefighters, it’s the reality. These types of low-visibility situations are unavoidable—and can be extremely hazardous. Responders can become disoriented during emergency operations when every second counts. Rescuing trapped or incapacitated civilians, keeping tabs on fellow responders, communicating with the command center, and locating exit routes—all while navigating through flames and debris—is an enormously daunting task. The Science and Technology Directorate (S&T) has been working with an innovative private industry partner and first responder evaluators to design a technology that will guide firefighters through low visibility environments and get them safely back home to their families.
Seeing Through the Smoke
C-THRU is a real-time, indoor visualization system currently in development with San Francisco-based startup Qwake Technologies, Inc. S&T is supporting this research and development effort with funding as well as expert oversight every step of the way. “The main objectives of the project are to provide effective navigation in obscured environments, enable safe and coordinated situational awareness in chaotic conditions, and support rapid decision-making in high stress situations,” said S&T Program Manager Maua Karen Johnson. “The bottom line is C-THRU will save lives.”
The C-THRU system will deliver a hands-free, augmented reality visual overlay, powered by a micro-computer safely housed inside an easily wearable device that attaches to a standard firefighter helmet. C-THRU consists of two components: the Navigator (worn on the helmet) and the Visual Command (a tablet used by incident commanders to see what their crew members are seeing). The display is meant to minimize confusion and disorientation, using advanced thermal imaging to identify surfaces and objects in the vicinity, such as stairs and people.
According to Qwake CEO Sam Cossman, “One of the most important features of C-THRU is to take that raw thermal image … and kind of draw lines around all the objects in the frame. An outline around a human body or the geometry of a building is rapidly shown through the heads-up display. So, you can very quickly say, ‘Am I in a small closet or a big warehouse? Is there a victim on the ground? Is there a staircase to an egress?’ This capability is referred to as edge detection and it leverages artificial intelligence and machine learning to help firefighters rapidly make sense of their environment.
In addition to the edge extraction capability Cossman described, Qwake is working on developing a backtracking function that provides turn-by-turn guidance, allowing a responder to follow the path they took into the building back out to safety. It is all too easy to get turned around in an unfamiliar, low visibility environment. The ability to automatically locate a safe exit route addresses the very real concern of firefighters losing their way during a response because they are naturally focused on other issues. C-THRU eliminates the need to form a mental map of one’s surroundings, allowing them to focus their attention to the mission at hand.
The personal navigation is further enhanced with a simple, nonverbal two-way communication capability. Incident commanders can request personnel accountability reports through the Visual Command tablet to ensure that crew members are not in distress. Responders wearing C-THRU can send a “thumbs up” reply, confirming their status or, if needed, issue a mayday to request help.
There is also the opportunity to gain post-operation insight via the Visual Command tablet to support continuous improvement.
Innovations upon Innovations
The heart of the C-THRU device is the Navigator heads-up display. A small piece of abrasion-resistant glass flips down within the user’s field of vision and images are projected onto this piece of glass. A reflective coating known as a dichroic filter splits the beam of light that’s projecting the images, bouncing certain wavelengths of light back to the viewer’s eye while also allowing them to still see through the glass. The glass features advanced hydrophilic coating materials designed to help reduce fogging and power consumption.
C-THRU is designed to be attached to a firefighter’s existing helmet during smoke investigations, structural firefighting, and general rescue operations. Three interlinked modules spread functionality and weight across the helmet, while reducing impact risk and snag hazard. The center of mass is strategically located towards the back of the head rather than the front to reduce forward tipping and improve comfort. The adjustable mechanical hinge boasts a breakaway safety feature, should the device become caught on something during a response.
The technology was also designed to be able to take the heat. A major obstacle for any firefighters’ gear, extreme temperatures were addressed up front with conscientious enclosure design and physical packaging. C-THRU has a rubberized surface with hard plastic beneath that protects the printed circuit boards and enables the software to properly function despite extreme temperature. Thermal survivability stress testing isn’t limited to what the system—or the user—will encounter during a firefighting response, it also involves the function of the system itself. Developers needed to determine how much power the system uses with all peripherals connected and ensure the central processing unit and graphics processing unit could remain operational without overheating. S&T and Qwake created a full spectrum of simulated user stress levels to assess performance and optimize the C-THRU device for use in any situation a firefighter may face.
S&T Prioritized Ease of Use
Every aspect of the product has been designed with user experience in mind, including ergonomic, well-placed buttons. In fact, a whole lot of thought has gone into refining the placement and function of the buttons to provide an intuitive tactile experience. Rather than simply pressing down from above onto a flat surface, like a computer keyboard, the architecture of C-THRU’s buttons have a tilted angle and are each divided by elevated ridges so the user will be able to comfortably align their fingers and recognize which buttons are activated without seeing them. The development of the button geometry and configuration has incorporated invaluable feedback from firefighters in S&T’s First Responder Resource Group who shared the need for large buttons that can accommodate their large gloves.
“I've been involved for quite a few years now, so I've seen the prototype evolve to where we're at right now … it's come a very, very long way. And I'm impressed,” said Walter Vidosh, Menlo Park Fire Protection District Captain.
The control menu is similarly intuitive and well-tested. C-THRU’s three features of navigation, vision, and communication are accessed via an interactive flow that employs user experience/user interface best practices. The system is designed to minimize the number of selections necessary to arrive at a desired result and never leave the user in a digital dead end. Key communications features and functions include critical alerts such as emergency evacuation and mayday distress signal, incident notifications related to various stages of a response, and system status information, for example, battery charge levels.
Putting the Tech to the Test
S&T officials, representatives from Qwake, and firefighter evaluators recently convened at the San Diego Fire-Rescue Training Facility in California at the end of August for an operational field assessment to test the capabilities of this innovative equipment enhancement. The assessment included both the Navigator helmet accessory and Visual Command tablet. S&T’s National Urban Security Technology Laboratory (NUSTL) oversaw the event, designing three different scenarios that subjected this technology to the punishing realities of firefighting.
Kris Dooley, NUSTL Program Analyst, explained, “NUSTL will consolidate the data received from the responders, develop a report that will be published for the responder community, and that information goes to Qwake for potential inclusion, to enhance the products prior to commercialization.”
Firefighters were outfitted with C-THRU Navigator prior to entering a heated, dark, smoke-filled space. The scenarios allowed them to assess operational aspects such as comfort, usability and compatibility with personal protective equipment as well as the edge detection thermal imagining, and incident command communication functions. The feedback from the responders will help further refine the product for successful commercialization in 2024.
As Captain Justin Quarisa, of the Cosumnes Fire Department in Elk Grove, California, put it, “The wow factor is just the incredible technology. Obviously, we have thermal imagers, but putting that on your helmet with a heads-up display and then the firefighter tracking is something that everybody's been trying to solve. So just having those capabilities and then the AI component where it enhances what you're seeing with the green image or you can, you know, take it off if you don't like it, but it really just gives you a lot better picture of a room, almost like you're watching a video game.”
You can learn more about C-THRU by listening to our Tech Speak minisode of the Technologically Speaking podcast, “Almost Like You're Watching a Video Game” on Apple or Google podcasts or on our website.
Contact STMedia@hq.dhs.gov with related media inquiries.