WEBVTT - https://subtitletools.com 00:00:00.000 --> 00:00:05.110 Good afternoon. Welcome to the Department of Homeland Security, Science and Technology Directorates 00:00:05.110 --> 00:00:10.140 webinar on AUDREY and artificial intelligence for 00:00:10.140 --> 00:00:15.180 first response. I would like to start by saying if you are on 00:00:15.180 --> 00:00:20.210 the phone please be sure to connect to our Adobe Connect session and that is where you may also submit 00:00:20.210 --> 00:00:25.240 your questions via the chat feature. For those who are still 00:00:25.240 --> 00:00:30.290 getting to know us. The Science and Technology Directorate is the R&D arm for the Department 00:00:30.290 --> 00:00:35.340 of Homeland Security, which is helping to support the mission to protect the nation. 00:00:35.340 --> 00:00:40.390 DHS S&T are operational components 00:00:40.390 --> 00:00:45.450 Coast Guard, FEMA, Critical Infrastructure and the First Responder community 00:00:45.450 --> 00:00:50.520 that's our ecosystem of entities that we are supporting 00:00:50.520 --> 00:00:55.530 in our day to day operations. Today we will be talking about the role of AUDREY 00:00:55.530 --> 00:01:00.610 and artificial intelligence and how it can and will play a role 00:01:00.610 --> 00:01:05.690 in supporting our First Responders. With us for this discussion is John Merrill 00:01:05.690 --> 00:01:10.780 in the Science and Technology Directorate. Good afternoon. [John Merrill] Good afternoon, I'd like to make one 00:01:10.780 --> 00:01:15.860 comment to make sure that everybody understands what we mean when we talk about AUDREY. 00:01:15.860 --> 00:01:20.980 AUDREY is actually a acronym that was coined by NASA JPL who is our primary performer on 00:01:20.980 --> 00:01:26.090 this project. It actually means Assistant for Understanding Data through Reasoning, Extraction 00:01:26.090 --> 00:01:31.220 and Synthesis. Yes, that tends to be a little bit of a tongue twister but AUDREY is a little bit 00:01:31.220 --> 00:01:36.260 easy to pronounce so we've stuck with that name and over the past several years 00:01:36.260 --> 00:01:41.300 in many publications that we put out, NASA JPL has done a pretty decent job of 00:01:41.300 --> 00:01:46.320 pushing the information out and then to this AI capability. 00:01:46.320 --> 00:01:51.370 A while back, about four and half years ago, NASA JPL had approached us and asking 00:01:51.370 --> 00:01:56.420 us if we had any need for a artificial intelligence capability. 00:01:56.420 --> 00:02:01.460 At that time we did not have any requirements from the First Responder community, but over a period of about 00:02:01.460 --> 00:02:06.530 six months from the time of our initial engagement, we had many First Responders that came up and 00:02:06.530 --> 00:02:11.600 to us, including the fire department, EMS and law enforcement stating that they receive 00:02:11.600 --> 00:02:16.660 literally petabytes and zettabytes of data which they have absolutely no access to 00:02:16.660 --> 00:02:21.730 and they wanted to know is there a possibility that we could possibly use AI to 00:02:21.730 --> 00:02:26.810 decipher that and provide them with actual information. So throughout the course of this 00:02:26.810 --> 00:02:31.900 broadcast we are going to ask several questions related to that and hopefully 00:02:31.900 --> 00:02:36.010 give you some answers in regards to how AUDREY is currently being used in the Respondent 00:02:36.010 --> 00:02:40.110 community and hopefully maybe used in other areas as well. 00:02:40.110 --> 00:02:44.220 [Host] Great that is so helpful. Well let's get going 00:02:44.220 --> 00:02:48.240 then and tackle some of the questions that have been provided by iCERT. 00:02:48.240 --> 00:02:52.270 Which is the Industry Council for Emergency response Technologies, who like S&T 00:02:52.270 --> 00:02:56.310 is focused on technology innovation to improve First 00:02:56.310 --> 00:03:00.350 Responders safety and effectiveness in the field. But before we do that 00:03:00.350 --> 00:03:04.390 again please submit your questions via the chat feature on this webinar. 00:03:04.390 --> 00:03:08.480 Anything we can't answer today due to time, you can put 00:03:08.480 --> 00:03:14.560 in the chat feature or you can also email us at first.responder@hq.dhs.gov. 00:03:14.560 --> 00:03:18.630 Alright so, is the goal of AUDREY to 00:03:18.630 --> 00:03:23.700 simply analyze the data coming to provide a clearer 00:03:23.700 --> 00:03:28.770 picture of the situation at hand to humans making decision or is the end goal 00:03:28.770 --> 00:03:33.850 to augment humans in making command control decisions, within prescribed guidelines? 00:03:33.850 --> 00:03:38.940 [John] One thing that I think to be very clear on is that when we started the 00:03:38.940 --> 00:03:44.120 development of AUDREY, AUDREY was a digital assistant. It does not 00:03:44.120 --> 00:03:49.240 take the place of a human in the decision chain. So it does provide the clearer picture 00:03:49.240 --> 00:03:54.290 based on the amount of data that is produced and bring it in provides the First Responders with 00:03:54.290 --> 00:03:59.300 actionable information. Think of it this way, from the firefighter that is 00:03:59.300 --> 00:04:04.320 responding to a major incident. There is literally hundreds of data points in which they can access to 00:04:04.320 --> 00:04:09.360 Building management, IOT information, lighting, wind 00:04:09.360 --> 00:04:14.390 weather conditions in which the firefighter as they are approaching could use that information 00:04:14.390 --> 00:04:19.440 and preparation to actually putting out a fire. So our hope is that AUDREY can be used. 00:04:19.440 --> 00:04:24.470 In a manner that provides to the full situational awareness and again it does not take the human out of the decision 00:04:24.470 --> 00:04:29.500 chain. It's a digital assistant for the human. [Host] Alright, great. 00:04:29.500 --> 00:04:34.520 So have there been any use cases or actual testing for 00:04:34.520 --> 00:04:39.560 AUDREY or AI in general? Supporting efforts for 00:04:39.560 --> 00:04:44.650 initial emergency call taking and categorization. [John] Yes, we did. 00:04:44.650 --> 00:04:49.720 I want to say about 18 months to 2 years ago, we conducted a high level pro type test with 00:04:49.720 --> 00:04:54.780 with Grant County Washington. For those of you who don't know that area. If you know where the cascades are 00:04:54.780 --> 00:04:59.860 It's located in Washington State, if you go east about 100 miles that's where Grant County is. Grant County 00:04:59.860 --> 00:05:04.940 is approximately 22 hundred square miles and patrolled by about 55 deputies. 00:05:04.940 --> 00:05:10.040 They had approached us saying that hey we have an incident here, we do not have 00:05:10.040 --> 00:05:15.230 complete situational awareness, however we have the data we do not know how to push it to our respondent 00:05:15.230 --> 00:05:20.350 community. The same thing applied to the firefighters as well. So talking to them 00:05:20.350 --> 00:05:25.470 closer along with discussions with their 00:05:25.470 --> 00:05:30.540 MACC, Multi Agency Communications Center, for everybody else it will be known as a PSAP. 00:05:30.540 --> 00:05:35.590 But they called it MAC. We found out that we could possibly work with them through the 00:05:35.590 --> 00:05:40.640 PSAP or the MAC by going directly and working only with the fire department or 00:05:40.640 --> 00:05:45.690 the law enforcement community. So working with them we decide okay let's take a look at the data that's coming 00:05:45.690 --> 00:05:50.760 in. What can we use or what could they use from the mission perspective? So we 00:05:50.760 --> 00:05:55.820 were able to tap in and provide them with sensor points on body worn cameras. 00:05:55.820 --> 00:06:00.880 Physiological capabilities and be able to inject it through using Wi-Fi and the 00:06:00.880 --> 00:06:05.960 TO1 service provider which was, I want to say there were Verizon or AT&T in that area 00:06:05.960 --> 00:06:11.130 and we were able to push it directly into the PSAP. What was also pretty unique 00:06:11.130 --> 00:06:16.240 is that while we were absorbing the events that were occurring for a scenario that we [unintelligible] 00:06:16.240 --> 00:06:21.350 We had folks here at DHS from [unintelligible] from approximately 2200 miles away 00:06:21.350 --> 00:06:26.470 who were actually seeing that live as well. For all the feed that was coming into the PSAP 00:06:26.470 --> 00:06:31.510 and the PSAP was also pushing the data directly back out in real-time to the First Responders. 00:06:31.510 --> 00:06:36.550 [Host] Wow, that's impressive. Well then how would AUDREY interface 00:06:36.550 --> 00:06:41.590 with 911 and CAD CVE solutions to support the 00:06:41.590 --> 00:06:46.620 collection and analyses of the multimedia that you were kind of talking about earlier and other data elements? 00:06:46.620 --> 00:06:51.670 [John] One of the things and again I need to emphasize that when you're working in a PSAP 00:06:51.670 --> 00:06:56.710 environment or a 911 center is that we're there to take and help the human. 00:06:56.710 --> 00:07:01.770 What we're thinking of doing with respect to AUDREY again that digital assistant that would 00:07:01.770 --> 00:07:06.830 help that dispatcher, the call taker to be able to bring that information in and 00:07:06.830 --> 00:07:11.890 synthesize it and she'll give them actual information. Under no circumstances at this point and time 00:07:11.890 --> 00:07:16.940 or do we have the ability to physically take the information or 00:07:16.940 --> 00:07:22.010 rather I'm sorry, to actually physically take the call automatically and then patch through the call take orders 00:07:22.010 --> 00:07:27.080 etc. We don't want to do that. At this point what we are interested in is getting the call in 00:07:27.080 --> 00:07:32.110 and have AUDREY in the background so there if there is additional background noise that AUDREY needs to 00:07:32.110 --> 00:07:37.200 or if let's say it's a law enforcement or a 00:07:37.200 --> 00:07:42.300 other type of incident or active shooter, you have to listen to the background to see if there are other 00:07:42.300 --> 00:07:47.410 environmental issues that the responding agencies need 00:07:47.410 --> 00:07:52.520 to be aware of. I would use an example that we had with one of our colleagues in Orange County, Florida. 00:07:52.520 --> 00:07:57.540 They told us that he was responding to a domestic dispute 00:07:57.540 --> 00:08:02.560 case. As he was responding and about to walk up and knock on the door, the dispatcher basically called them to 00:08:02.560 --> 00:08:07.600 stop, do not approach, return to your squad car. He came back, called dispatch and asked what's the problem? 00:08:07.600 --> 00:08:12.640 What the dispatcher noticed is she heard a chamber being round into a shot gun. So it essentially saved his life. 00:08:12.640 --> 00:08:17.670 So think about the perspective of AUDREY sitting in the background being able to 00:08:17.670 --> 00:08:22.700 take that information in and provide that PSAP to dispatcher 00:08:22.700 --> 00:08:27.740 or the call taker, that additional information to be able to push it through the responder as they are approaching the 00:08:27.740 --> 00:08:32.790 incident vice actually being on the incident and be receiving at a later time. 00:08:32.790 --> 00:08:37.840 [Host] [Nods her head] Wow. So then 00:08:37.840 --> 00:08:42.890 how could, well I think we've talked a little bit about this next question but how could AUDREY support the 00:08:42.890 --> 00:08:47.950 initial 911 call triage to prioritize the calls? 00:08:47.950 --> 00:08:53.010 Or decrease call taker workload or can it? [John] No, at this point and time 00:08:53.010 --> 00:08:58.090 it's not mature enough to do that. We are still in the early 00:08:58.090 --> 00:09:03.170 phases of the AUDREY development. One thing that I do want to make clear is that 00:09:03.170 --> 00:09:08.280 that I want to make a decision between AI and what we call AGI, Artificial General Intelligence. 00:09:08.280 --> 00:09:13.300 When originally we started it was called artificial intelligence. We were just using generic terms 00:09:13.300 --> 00:09:18.510 to generic software applications that actually push the data out. We're to the point now that we can we can 00:09:18.510 --> 00:09:23.520 actually adjust data and teach AUDREY to learn, to understand circumstances. 00:09:23.520 --> 00:09:28.640 To understand what a dispatcher will do. So going back to the PSAP issue if there 00:09:28.640 --> 00:09:33.660 something that is going on that AUDREY could potentially assist with and would 00:09:33.660 --> 00:09:38.700 and the other is to be able to bring in other forms of data in which AUDREY would be able to synthesize 00:09:38.700 --> 00:09:43.740 and push only the information that's relevant to that situation. 00:09:43.740 --> 00:09:48.780 [Host] So then does AUDREY or how would 00:09:48.780 --> 00:09:53.860 AUDREY determine the level of confidence in that output or expose that 00:09:53.860 --> 00:09:58.950 to the end user consuming the output? [John] Very good question. One of the things 00:09:58.950 --> 00:10:04.010 that we run into when you're talking about teaching a computerized capability is 00:10:04.010 --> 00:10:09.150 that you do have to have very good data to baseline. If the baseline 00:10:09.150 --> 00:10:14.230 data that you have is faulty, it's not gonna learn 00:10:14.230 --> 00:10:19.250 it doesn't know the good from the right. So what we have 00:10:19.250 --> 00:10:24.340 to do is to make sure that we take a look at that data set that seems to be pushed into 00:10:24.340 --> 00:10:29.430 AUDREY, it is clean, it has been validated and verified 00:10:29.430 --> 00:10:34.530 and is good in terms of up in the confidence level. So that when an incident occurs 00:10:34.530 --> 00:10:39.650 AUDREY knows how to match it against that database. 00:10:39.650 --> 00:10:44.780 When a call comes in hearing that background noise, we need to teach AUDREY what that background noise is 00:10:44.780 --> 00:10:49.810 supposed to be. If you don't it doesn't know what to look for. So again it's a learning 00:10:49.810 --> 00:10:54.850 phase and it takes many many years to get that right data in 00:10:54.850 --> 00:10:59.880 and unlike a human, actually it's like a human. Where you learn over a period number of years 00:10:59.880 --> 00:11:04.910 get your knowledge built up. We are doing the exact same thing with AUDREY. [Host] So then when you're 00:11:04.910 --> 00:11:09.960 doing that are you tailoring that baseline data to a particular 00:11:09.960 --> 00:11:15.000 user bases? You know fire, law enforcement and so forth or is it generalized? [John] No 00:11:15.000 --> 00:11:20.040 You are absolutely correct. We are looking at from the various disciplines. We're looking at from the firefighters 00:11:20.040 --> 00:11:25.100 from what they need and we also have the law enforcement and we also have the paramedics. 00:11:25.100 --> 00:11:30.160 Case in point, we just did a short test with the paramedics 00:11:30.160 --> 00:11:35.240 and which we looked at the physiological monitoring capabilities, pushed the information in 00:11:35.240 --> 00:11:40.320 and AUDREY would actually push that data out to the emergency room to actually work 00:11:40.320 --> 00:11:45.420 directly with the paramedic in the field. [Host] Looks like we have a couple of questions from the chat. 00:11:45.420 --> 00:11:50.500 The first one is "How local will AUDREY be? That is 00:11:50.500 --> 00:11:55.620 will there be only one national AUDREY system or could 00:11:55.620 --> 00:12:00.730 a local jurisdiction have its own version, a localized version of AUDREY?" [John] Yes, we're looking at 00:12:00.730 --> 00:12:05.870 three different instances. At the national level and by the way it is a 00:12:05.870 --> 00:12:10.900 distributed network and what we are envisioning is that the distributing network would be networked together 00:12:10.900 --> 00:12:15.940 so that you've got multiple locations for that knowledge base. So let's start 00:12:15.940 --> 00:12:20.970 at the tactical level in which you would have an instance for the firefighter, the EMS or the law enforcement officer. 00:12:20.970 --> 00:12:26.010 They would have a minimum computing capabilities, but actually push back into 00:12:26.010 --> 00:12:31.060 our connection directly into the main database. However if they were to lose that connectivity 00:12:31.060 --> 00:12:36.100 it would still have some smaller computing capabilities to actually do their job. 00:12:36.100 --> 00:12:41.170 At the next level, it would be more at the, like a laptop level 00:12:41.170 --> 00:12:46.230 or command center. I'm sorry not the command center, but the [unintelligible] commander, the battalion chief 00:12:46.230 --> 00:12:51.290 would have a little bit more computing power in which they can also reach back into the main 00:12:51.290 --> 00:12:56.380 network for AUDREY. Now at the highest level which 00:12:56.380 --> 00:13:01.450 would have the most computing power would be at more at the PSAP level or throughout the country that is 00:13:01.450 --> 00:13:06.540 physical network. So if an incident is going on at a local jurisdiction AUDREY can reach back, 00:13:06.540 --> 00:13:11.640 scower the network to find out has this occurred before and it can bring that information back in. 00:13:11.640 --> 00:13:16.740 [Host] [Nods] And then the other questions was "if you 00:13:16.740 --> 00:13:21.850 could restate the term AGI and the elements of AGI?" [John] Yes. 00:13:21.850 --> 00:13:26.870 When we started the AUDREY program we were physically looking at only the 00:13:26.870 --> 00:13:31.890 artificial intelligence piece. As we moved forward and that 00:13:31.890 --> 00:13:36.920 AI started to mature, we needed to teach and let AUDREY learn on its own. 00:13:36.920 --> 00:13:41.960 So the piece on the general is it is that it is Artificial General Intelligence capability. 00:13:41.960 --> 00:13:47.000 It's basically modeled after human neurological network where you learn. 00:13:47.000 --> 00:13:52.030 Same thing with a computer. You teach the computer to learn and understand varied instances. And again it goes 00:13:52.030 --> 00:13:57.060 back to the verification and validation of the data that you feed it and make sure that data is 00:13:57.060 --> 00:14:02.110 absolutely correct, so that AUDREY would know the difference between right and wrong. 00:14:02.110 --> 00:14:07.150 [Host] Alright. Once of thee additional questions that 00:14:07.150 --> 00:14:12.250 we have was "What benchmarking methods are used to verify the accuracy of 00:14:12.250 --> 00:14:17.310 the output?" [John] Again this goes to the confidence level of the data 00:14:17.310 --> 00:14:22.360 we receive. There is really no standard or bench mark that has been established 00:14:22.360 --> 00:14:27.440 that we know of yet. We are working with NASA JPL who is our primary performer at this time. 00:14:27.440 --> 00:14:32.510 Our expectation is that by working directly with the responder community and 00:14:32.510 --> 00:14:37.600 and what their expectations are. I'll use our Hastings County experiment as an example. What their expectations 00:14:37.600 --> 00:14:42.690 are as an example. They gave us a book of various terminology and in terms of 00:14:42.690 --> 00:14:47.790 their medical language that they use. I was astounded that their medical terminology 00:14:47.790 --> 00:14:52.900 and how they pronounce it was a little bit different than the U.S. side. [Host] Alright, so 00:14:52.900 --> 00:14:58.020 Hastings is in, Ontario. [John] Yes, Hastings Ontario. We had partnered with 00:14:58.020 --> 00:15:03.060 defense research and develop center in Canada and they had a vested interest in this capability 00:15:03.060 --> 00:15:08.090 as well. So throughout the test what we are looking for is 00:15:08.090 --> 00:15:13.120 when somebody says a certain word, does it understand it 00:15:13.120 --> 00:15:18.170 correctly. It has to understand especially when it comes to medical prescriptions and medications to be able 00:15:18.170 --> 00:15:23.220 to actually transport that information up to the medical doctor that can prescribe back to the 00:15:23.220 --> 00:15:28.280 paramedic. [Host] Looks like we have another question from chat which is great. 00:15:28.280 --> 00:15:33.340 "Does AUDREY have video analogy capabilities? For example analyses of 00:15:33.340 --> 00:15:38.400 a picture or a person identity verification for example?" [John] We are not doing 00:15:38.400 --> 00:15:43.460 persons. We are only doing inanimate objects. We have 00:15:43.460 --> 00:15:48.540 done some very high level simple tests. We download a active shooting 00:15:48.540 --> 00:15:53.710 incident which was posted to a law enforcement website. In which a shooter came in with a 00:15:53.710 --> 00:15:58.820 small sidearm. And what we are trying to teach AUDREY was okay do you understand what this situation 00:15:58.820 --> 00:16:03.920 looks like and can you physically identify a small sidearm? 00:16:03.920 --> 00:16:09.040 And based on the data that we were able to post within 00:16:09.040 --> 00:16:14.090 AUDREY to teach it to understand it. It was able to pick up that small sidearm in a matter of about 00:16:14.090 --> 00:16:19.130 10-15 seconds. Our ultimate goal is to be able to do that object recognition match to the 00:16:19.130 --> 00:16:24.170 FBI database. So the FBI's small arms database. So AUDREY would know exactly what that 00:16:24.170 --> 00:16:29.210 would look like. Again from the facial recognition, know only that it would look at 00:16:29.210 --> 00:16:34.260 and say yes it's a person that's it. Not from the physical identification of who that person is. 00:16:34.260 --> 00:16:39.280 We ae not doing that. [Host] Okay. And you 00:16:39.280 --> 00:16:44.330 actually answered one of our next questions, which was 00:16:44.330 --> 00:16:49.370 "Does it analyze video and images?" And it sounds like it does but it has a limited capability at this 00:16:49.370 --> 00:16:54.430 point. [John] Yes, it does. And we are also looking into working with some of the 00:16:54.430 --> 00:16:59.490 requirements that came in from the U.S. Coast Guard for search and rescue capabilities. It is very difficult to see 00:16:59.490 --> 00:17:04.570 a person in the water and the coast guard research and development center conducted their own 00:17:04.570 --> 00:17:09.640 high level tests and gave us some video footage and we were able to feed that into AUDREY 00:17:09.640 --> 00:17:14.740 and then in another pilot test, we fed the video imagery 00:17:14.740 --> 00:17:19.740 that came in to see if they could see, to physically see the person in the water and AUDREY was able to find 00:17:19.740 --> 00:17:24.830 that person in the water. And again this was in calm flat glass water 00:17:24.830 --> 00:17:29.940 but many many times, I would say about 99% of the time when the U.S. Coast Guard does 00:17:29.940 --> 00:17:35.060 search and rescue it is not calm water. It is very rough seas and it is very difficult to see a person in the waters. 00:17:35.060 --> 00:17:40.190 [Host] Sure, sure. SO how does AUDREY compare 00:17:40.190 --> 00:17:45.220 to AWS and IBM's AI engines? 00:17:45.220 --> 00:17:50.260 Alexa and Watson? [John] AUDREY was specifically built 00:17:50.260 --> 00:17:55.290 for the First Responder community. Earlier I had mentioned about the Firefighter, the EMS 00:17:55.290 --> 00:18:00.340 and Law Enforcement. Our primary focus is there. They are the ones who drive and tell us 00:18:00.340 --> 00:18:05.430 what their looking for in terms of how it's being used. If you ware to do a comparison between 00:18:05.430 --> 00:18:10.490 AUDREY, Alexa or Watson and ask for some medical terms from when we started 00:18:10.490 --> 00:18:15.580 this process about three years ago. You would probably get some gobbled response 00:18:15.580 --> 00:18:20.620 from Alexa and Watson, but over the period of the last couple of years the advancements 00:18:20.620 --> 00:18:25.710 that have been made in the natural language processing has been advanced 00:18:25.710 --> 00:18:30.740 significantly. So I think they are pretty par at this point and 00:18:30.740 --> 00:18:35.830 time from the point of AUDREY it is specifically for and will be license for the First Responder community. 00:18:35.830 --> 00:18:40.910 [Host] So 00:18:40.910 --> 00:18:46.020 the next question was "What are some specific use cases this AI engine can handle?" 00:18:46.020 --> 00:18:51.130 And so it sounds like First Responder community is the primary focus but are there other use cases that you are 00:18:51.130 --> 00:18:56.260 looking at? [John] [Pause] Right not our focus has been 00:18:56.260 --> 00:19:01.290 the respondent community. Within the last couple of months we did get a new requirement that came in 00:19:01.290 --> 00:19:06.330 from the law enforcement community to assist the driver 00:19:06.330 --> 00:19:11.360 to keep their eyes on the road to reduce their accidents, but 00:19:11.360 --> 00:19:16.400 that's just at the beginning phase, on collecting our requirements find out what that actual gap is. 00:19:16.400 --> 00:19:21.450 Our expectation is that we hope to kick that program off in about another six months to a year. 00:19:21.450 --> 00:19:26.500 [Host] Okay and the next 00:19:26.500 --> 00:19:31.570 question came from iCERT members, "Will we be able to participate in the 00:19:31.570 --> 00:19:36.630 build out from a feature and a functionality perspective?" How can they help is what I take this to mean. 00:19:36.630 --> 00:19:41.630 [John] Yes, the United States government when we take on a contract to do 00:19:41.630 --> 00:19:46.690 R&D our goal is to produce a product that's going to commercialized and transitioned out into the field. 00:19:46.690 --> 00:19:51.770 We are not in the business to compete with a private sector. 00:19:51.770 --> 00:19:56.840 What we try to do as we move forward is to identify a potential partner that 00:19:56.840 --> 00:20:01.930 can take over the AUDREY program at some future date. When it is mature enough that we can transition this out. 00:20:01.930 --> 00:20:07.010 We don't expect that AUDREY will be transitioned out for probably another six months to a year. 00:20:07.010 --> 00:20:12.120 At which point it will still need to be taught that yes, it will still need to be taught how to do specific actions for the 00:20:12.120 --> 00:20:17.240 law enforcement community, the fire community and the paramedic community. 00:20:17.240 --> 00:20:22.360 The challenge is going to be is to find out who that potential partner is. 00:20:22.360 --> 00:20:27.390 AI GI as a whole has exploded in the last six months to a year and it's very difficult to find 00:20:27.390 --> 00:20:32.430 that potential partner and we do have to work with NASA 00:20:32.430 --> 00:20:37.480 JPL because they are the ones that actually wrote the code 00:20:37.480 --> 00:20:42.530 for us and we will be working with them to figure out okay, who can we work with? 00:20:42.530 --> 00:20:47.580 [Host] Right, right. Got it. So this one, the next question is fairly 00:20:47.580 --> 00:20:52.650 general but I think it's really important to the First Responder community. "It's how can 00:20:52.650 --> 00:20:57.710 artificial intelligence help protect firefighters and police?" 00:20:57.710 --> 00:21:02.770 [John] That is the primary reason we actually developed AUDREY for full situational awareness. 00:21:02.770 --> 00:21:07.850 Getting that information when they need it the most and again I'll use the firefighter as 00:21:07.850 --> 00:21:12.880 an example again. From the perspective of a tactical edge where the firefighters physically go into 00:21:12.880 --> 00:21:17.980 the building. How nice would it be to physically track them into AUDREY and if there is any sensors 00:21:17.980 --> 00:21:23.070 that are within the building to push it into AUDREY and if it senses any type of chemical or biological agents 00:21:23.070 --> 00:21:28.170 that are possibly burning in that area, being able to push that information in, also push it into the database. 00:21:28.170 --> 00:21:33.290 So if a similar incident were to occur somewhere else within the United States 00:21:33.290 --> 00:21:38.400 then you can push that information and basically say, "Hey this occurred, you might want to 00:21:38.400 --> 00:21:43.470 take a look at this. It might be relevant to your current mission. [Host] So we have two more 00:21:43.470 --> 00:21:48.510 questions from chat. The first is “What types of IOT equipment will AUDREY receive 00:21:48.510 --> 00:21:53.530 inputs from? For example, types of sensors or equipment today or 00:21:53.530 --> 00:21:58.580 anything planned in the future?" [John] IOT is a very big 00:21:58.580 --> 00:22:03.600 word and a very broad term. I will start at the tactical level of what we are currently 00:22:03.600 --> 00:22:08.650 using. I had mentioned about the physiological monitoring capabilities to be able to 00:22:08.650 --> 00:22:13.690 check the health status of whether your law enforcement, paramedic or firefighters. 00:22:13.690 --> 00:22:18.750 Being able to track them and to baseline their health data to help them out 00:22:18.750 --> 00:22:23.820 and let's go a little bit further up the chain at the IOT level 00:22:23.820 --> 00:22:28.890 from the sensors that are built within a city. There are a multitude of sensors that are currently available 00:22:28.890 --> 00:22:33.970 whether it's private or public. We are working with a couple of small businesses in which 00:22:33.970 --> 00:22:39.060 they are able to take public facing data and also proprietary facing data 00:22:39.060 --> 00:22:44.150 to bring it in. Ingest it. Catalog that information to 00:22:44.150 --> 00:22:49.260 the cloud and if you are a properly authenticated and authorized user of that data you can use AUDREY 00:22:49.260 --> 00:22:54.380 to go into that database and bring that information down to you. That was done 00:22:54.380 --> 00:22:59.510 more of a table top exercise. We have not done it as a full-blown exercise for a 00:22:59.510 --> 00:23:04.540 firefighter or law enforcement vehicle going down the road. I would suspect that could be 00:23:04.540 --> 00:23:09.580 possibly accomplished within the next year or two, but from other since of perspective 00:23:09.580 --> 00:23:14.610 there's literally hundreds and millions of sensors out there that we can tap into but our primary goal is 00:23:14.610 --> 00:23:19.660 to meet the missions specifically related to the respondent community and make sure that they have 00:23:19.660 --> 00:23:24.710 everything they need for full situational awareness. 00:23:24.710 --> 00:23:29.750 [Host] And then our net question was "Are you running AUDREY in the commercial cloud or in a DHS 00:23:29.750 --> 00:23:34.820 data center?" [John] Neither at this point. It is right now with the NASA JPL within 00:23:34.820 --> 00:23:39.890 the FedRamp certified cloud service. Our ultimate goal is to move out into a distributed network 00:23:39.890 --> 00:23:44.940 AWS is one that has been considered that we could potentially use however 00:23:44.940 --> 00:23:50.020 because of the data that we are looking for it is going to be highly protected within each 00:23:50.020 --> 00:23:55.090 jurisdiction. There had to be policy related guidance from each of the jurisdiction 00:23:55.090 --> 00:24:00.190 on how they are gonna share that data. That is being addressed but it's a long ways off from 00:24:00.190 --> 00:24:05.310 actually having a distributed network throughout the country. Our expectation is that probably 00:24:05.310 --> 00:24:10.410 gonna start at the local level and then bring it up into more of a national level cloud based service. 00:24:10.410 --> 00:24:15.540 [Host] Got it. The next question we have is 00:24:15.540 --> 00:24:20.570 how can AI decrease response times for you know 911, First Responder response times?" 00:24:20.570 --> 00:24:25.610 [John] I'll go back about 3-3 1/2 years when we kicked off AUDREY. 00:24:25.610 --> 00:24:30.620 I asked AUDREY or I'm sorry, I asked NASA JPL 00:24:30.620 --> 00:24:35.660 could you use this for doing some deep learning analytics 00:24:35.660 --> 00:24:40.690 for past data from 00:24:40.690 --> 00:24:45.730 a local jurisdiction. NASA JPL is located right there 00:24:45.730 --> 00:24:50.780 next to L.A., so we went and partnered with L.A. county, oh I'm sorry the city of L.A. 00:24:50.780 --> 00:24:55.820 They gave us one half year's worth of data in which the PSAPs were receiving the data. 00:24:55.820 --> 00:25:00.870 And we did some background MLAs on it and bottom line was we were able to help them 00:25:00.870 --> 00:25:05.930 preposition some of their resources to reduce the response time. 00:25:05.930 --> 00:25:10.990 Our ultimate goal is to look at it from that perspective, but ultimately we are also looking at it from full situational 00:25:10.990 --> 00:25:16.080 awareness and safety of the First Responders and our hope is as they are getting this information 00:25:16.080 --> 00:25:21.160 it will reduce their time in terms of actually physically having to look it up themselves. So they can 00:25:21.160 --> 00:25:26.250 concentrate on their mission and going to where they are needed the most. 00:25:26.250 --> 00:25:31.330 [Host] Again another question from chat was "Can we please clarify the difference 00:25:31.330 --> 00:25:36.430 between First Responders and Law Enforcement in the context of what you described as doing something 00:25:36.430 --> 00:25:41.550 to help the driver. Which driver?" [ John] Thank you. I should 00:25:41.550 --> 00:25:46.680 have clarified. When I talk about First Responders, I'm talking about all the disciplines. From the Law 00:25:46.680 --> 00:25:51.740 Enforcement perspective about what we are trying to do here in the next 6 months to a year. We're talking about 00:25:51.740 --> 00:25:56.770 the law enforcement officers driving a quad car. If you take a look at a fully decked out vehicle there's a 00:25:56.770 --> 00:26:01.820 multitude of different instruments, radios and displays that 00:26:01.820 --> 00:26:06.860 he or she has to focus on. Our ultimate goal is can an AI capability assist them 00:26:06.860 --> 00:26:11.910 somehow? We don't know. We're in the process of researching that and seeing if there's the capability 00:26:11.910 --> 00:26:16.970 is already out there. Our expectation is that there are already other companies that have already 00:26:16.970 --> 00:26:22.040 done this and if it has we may partner with them and say hey can you focus this 00:26:22.040 --> 00:26:27.090 and can you look at it from a law enforcement perspective. [Host] So then 00:26:27.090 --> 00:26:32.170 when do you feel automated, artificial intelligence or even 00:26:32.170 --> 00:26:37.240 AGI will be ready for early adopter commercial field testing? 00:26:37.240 --> 00:26:42.260 [John] That's a question I would say it depends, because 00:26:42.260 --> 00:26:47.350 we are building something that is for all three disciplines. We are putting a little bit further ahead 00:26:47.350 --> 00:26:52.460 with the firefighting than with the law enforcement and not so much 00:26:52.460 --> 00:26:57.550 with the paramedic. Our ultimate goal is that probably in 00:26:57.550 --> 00:27:02.670 about I would say another year to 18 months we may have a low scale prototype of that we might be able to 00:27:02.670 --> 00:27:07.690 deploy. And again it will depend on the receptive nature 00:27:07.690 --> 00:27:12.700 of the organization we are going to work with and we do have a lot of first responders that are willing 00:27:12.700 --> 00:27:17.740 to participate in our prototype testing. So there is no shortage of volunteers there. 00:27:17.740 --> 00:27:22.780 Ultimately I would like to see it probably prototyped inside of a PSAP 00:27:22.780 --> 00:27:27.780 because having that in a PSAP you are already addressing 00:27:27.780 --> 00:27:32.820 it from the firefighters, law enforcement and paramedic perspective. [Host] I'm curious 00:27:32.820 --> 00:27:37.860 how or why firefighters, while your making more progress 00:27:37.860 --> 00:27:42.890 you're a little further along with the firefighting community or the firefighters themselves? 00:27:42.890 --> 00:27:47.940 [John] I think because when we initiated this program 00:27:47.940 --> 00:27:52.990 we had a couple of firefighters that were participating originally in this discussion. 00:27:52.990 --> 00:27:58.030 And we made just the decision at the point and time, why don't we just address it. 00:27:58.030 --> 00:28:03.090 There was no particular rhyme or reason as to why we 00:28:03.090 --> 00:28:08.160 selected them and t was just that the input we got from them on what they 00:28:08.160 --> 00:28:13.250 would like to see thought it could potentially be an easy win. Although there is no such thing as 00:28:13.250 --> 00:28:18.330 what I consider working with artificial intelligence or artificial general intelligence. 00:28:18.330 --> 00:28:23.420 It's going to be a learning process and our hope is again working with all the disciplines 00:28:23.420 --> 00:28:28.510 and hopefully in about another year to 18 months we will have something out there. 00:28:28.510 --> 00:28:33.620 [Host] Great, great. Let's see. So 00:28:33.620 --> 00:28:38.750 what well, we somewhat touched on it. What is the timeline for AUDREY to be complete? 00:28:38.750 --> 00:28:43.870 Will to be rolled out in stages? But I think you have some thoughts on AUDREY and then 00:28:43.870 --> 00:28:48.940 the transition to artificial intelligence in general as a tool. [John] Yes, it's gonna be 00:28:48.940 --> 00:28:53.990 like I said it has to be rolled out in small phases. What we don't want to do is I had mentioned the PSAPs 00:28:53.990 --> 00:28:59.010 earlier, when we work with a PSAP it has to be a much smaller PSAP. I don't want to go into 00:28:59.010 --> 00:29:04.070 a large metropolitan area where you literally have hundreds of thousands of calls that are coming in. 00:29:04.070 --> 00:29:09.110 The role out is going to be in phases working with a small metropolitan area or a small 00:29:09.110 --> 00:29:14.170 non metropolitan area, urban areas first. Then roll out to probably to the metropolitan area that they work directly with. 00:29:14.170 --> 00:29:19.230 Having it that way you've got the nice close working relationship in a local jurisdiction. 00:29:19.230 --> 00:29:24.310 [Host] Okay as we kind of close the discussion out 00:29:24.310 --> 00:29:29.380 today. Are there any other thoughts that you want to leave our viewers and listener 00:29:29.380 --> 00:29:34.470 with on this topic. [John] As I have learned over the past 00:29:34.470 --> 00:29:39.570 four, four and half years that we have been working on the AUDREY data, good data. 00:29:39.570 --> 00:29:44.680 Patience. We have probably run into more fire walls 00:29:44.680 --> 00:29:49.800 and failures in terms of trying to get the successes that we 00:29:49.800 --> 00:29:54.920 were looking for but when I say failure I'm not taking about saying okay that's 00:29:54.920 --> 00:29:59.960 far enough we can't do this. I'm talking about from the technical perspective of learning from it. 00:29:59.960 --> 00:30:05.000 going back through the modifications. It's an iterative process and it's going to take a lot of time 00:30:05.000 --> 00:30:10.090 and effort on the part of our research to produce the product we are actually looking for and 00:30:10.090 --> 00:30:15.100 The First Responder community as a whole they're waiting. 00:30:15.100 --> 00:30:20.140 They understand the direction that the AI is going and they 00:30:20.140 --> 00:30:25.180 really cannot wait in terms of how this is moving along 00:30:25.180 --> 00:30:30.240 because it's so slow. They don't understand the slowness. They've seen our promo videos. 00:30:30.240 --> 00:30:35.290 They fully expect that to be the AUDREY of today. Which it is not and our goal 00:30:35.290 --> 00:30:40.340 is ultimately to be in that direction in the next several months. [Host] Okay, alright. 00:30:40.340 --> 00:30:45.420 Well before we wrap up, we'd like to thank iCERT. The Industry council for emergency 00:30:45.420 --> 00:30:50.480 response technologies, who helped get the word out about today's webinar. 00:30:50.480 --> 00:30:55.570 Helped to gather all the questions and ensure that we can have this really robust 00:30:55.570 --> 00:31:00.640 conversation. So for anyone who wasn't able to get 00:31:00.640 --> 00:31:05.730 there questions answered or would like more information. More details on anything we discussed 00:31:05.730 --> 00:31:10.840 today. The videos that John just mentioned. 00:31:10.840 --> 00:31:15.940 Please contact us at first.responder@hq.dhs.gov. 00:31:15.940 --> 00:31:20.960 It's also on the screen so feel free to 00:31:20.960 --> 00:31:25.990 contact us there for more information. We'll be happy to answer your questions. 00:31:25.990 --> 00:31:31.256 So thanks very much. Thanks John. [John] Thank you. [Host] And everyone have a great day.