Why You Shouldn’t Use NIST For UAS Drone Training

PURPOSE AND PROCESS OF THIS EVALUATION

In 2019, several Public Safety Agencies expressed their interest in adopting NIST official standard training methods as their official training standard for their remote pilots. Steel City Drones has spent significant time researching and studying the entire NIST program to determine if the NIST standard is worth pursuing.

As a company that offers comprehensive drone training to clients all across the country, Steel City Drones is always looking at ways to improve our own training processes, thus we gave this standard our due diligence trying to be as methodical and open-minded as possible. Our evaluation below is our professional opinion based on the data that we reviewed.

Evaluation Process

We reviewed all documents that are publicly available:

• NIST’s complete NFPA 2400 version 2018A
• NIST promo video
• Standard Test Methods to Evaluate Homeland Security Applications; Response Robots (E54.09) Small Unmanned Aircraft System Capabilities and Operator Proficiency for Emergency Response Applications PDF

In addition, we also:

• Exchanged emails with several key NIST staff to clarify information about their program. Specifically, we talked to Adam Jacob to clarify specifics about the standard and to answer questions.
• Talked extensively with one Public Safety agency located near Las Vegas that started adopting the NIST standard.

Lack Of Training Standardization Leads To Misinformation And Perception Issues

Before we give our feedback on the NIST standard, its important to address several key issues. The lack of a national UAS training standard puts the burden on key stakeholders to formulate their own training programs.

Historically organizations chose to go down 1 of 3 roads for drone integration.

1. Build their own program through due diligence and trial and error
2. Use an external training resource-vendor and completely rely on that vendor completely as the industry expert.
3. Completely wing it and run their program without funding and or any formal structure of training or certification. Many Public Safety agencies run their UAS flight operations without any formal training, Part 107 certification, or an FAA Certificate of Authorization.

These issues lead Public Safety stakeholders to rely on Industry experts (some are good, some are not) and other neighboring agencies to guide them which often leads to bad advice.

When considering training standardizations its important to understand the limitations of the equipment and how it’s very important to train operators around the limitations. When we engage a client our goal is to give a system overview picture of what drone integration and program development really need to include. We then work our way up from there.

We then develop a drone program that includes all of the necessary elements such as selecting the correct equipment, developing policy and procedures, and comprehensive training.

The FFA’s Role

Unfortunately, the FAA UAS rules and regulations do not help aid in this process at all. The FAA has gone from extremely over-regulated requiring remote pilots to obtain a private pilot license (2014-2016) to extremely under-regulated which requires no flying proficiency test (2016-Present).

In 2015 I was required to get a license to fly an airplane and not only did I have to perform solo flights, but I also had to pass a very difficult check ride that included being able to land an airplane with no engine power.

Currently, a remote pilot needs no formal training or skills proficiency to obtain a license to fly a UAS. This becomes problematic because of the low barrier to entry, the FAA sends mixed messages to organizations as the skill level needed to fly unmanned aircraft. As a result, many organizations see drones as nothing more than toys and completely underestimate what it takes to fly an unmanned aircraft.

KEY REFERENCE POINTS TO CONSIDER

What Is The Definition Of Effective Remote Pilot Proficiency?

Over the past 3 years, we have seen a lot of training programs come and go. Most training programs miss the mark on offering a comprehensive training package that checks all the boxes. Most training programs are developed by non-experienced real-life commercial remote pilots and or from competent experienced trainers.

This again goes back to the fact there are no nationwide standards and just about anyone can claim to be an expert. So the real question is who is qualified to make Industry-standard training guidelines? In our opinion, it has to come from experienced pilots and trainers in the industry.

When evaluating a training program we compare it to the system we currently have in place. It has taken us years of trial and error to develop our standards and that is our benchmark that we go off of. In our professional opinion, the following guidelines should be a part of every UAS training program.

1. The pilot knowing their aircraft and support equipment completely (including apps)
2. Ensuring FAA compliances and knowing what is best for their organization (flying under Part 107 or a COA) or when to use a combination of both.
3.  Flight proficiency (Being as proficient on the remote control as possible) which includes being able to fly “manually” without relying on or using any technology or automation.
4. Proper flight mission planning and Preparation.
   – Properly identifying hazards, risks, and proper risk mitigation strategies.
   – Using all available resources properly (crew resource management)
   – Demonstrating effective “POSITIVE” communication with the entire flight crew.
5. Being able to perform long-distance missions. This includes:
   – Understanding where a realistic line of sight stops and understanding the size of aircraft limitations.
   – The ability to maintain aircraft orientation and the ability to regain orientation as needed.
   – Advanced flying skills such as using the horizon properly to prevent flyaways.
6. Ability to make quick and effective aeronautical decisions based on the latest information.
7.  Being properly trained for emergency situations before they happen so the pilot can react properly.
8. Ability to recognize poor human depth perception that can occur in flight.
9. Ability to multi-task by flying the aircraft and simultaneously communicating with flight operations. This includes safe and effective task management of all flight operations.
10. Proper use of advanced camera sensors with a basic understanding of camera operator functionality.
11. Night flying and learning the effects of flying at night and proper risk mitigation.

NIST TRAINING STANDARD

The NIST UAS training consists of setting up short distance courses that are scalable in size for smaller courses.

They use various apparatus’ with buckets and stickers that the remote pilot performs flying exercises to teach pilot basic maneuvering skills. Each operator will work through exercises of position, transverse, orbit and
spiral.

HOW DOES NIST TRAINING STAND UP?

What Does NIST UAS Training Standard Cover Effectively?

The NIST covers some key elements of remote pilot flight proficiency and there is no disputing that a training program like NIST can make any remote pilot better on the sticks and properly train hand-to-eye coordination for shorter distances. The NIST standard also does a good job teaching many of the basic skills.

The course teaches the following proficiencies by using different levels of metrics on courses that have a wide variety of apparatus.

• How to hover a drone using GPS
• Orbit a point
• Fly straight and level
• Avoiding obstacles
• Landing accurately

This Training Would Be Good For The Following Flight Missions Types

• Short-range flight missions less than 200 feet.
• Swat and indoor situations where FPV technology can be utilized.
  – Learning how to fly in a controlled course where you can go back and forth between line-of-sight and FPV are good skills to acquire.
• Bomb Squad
  – Learning very delicate aircraft movements and hovering skills would be very important for this type of application.
• Quick assessments of a scene where the remote pilot does not have to fly far distances.
  – These are situations that require short distance flying.
• Fire and explosive investigations that require scene support (short range)
  – These situations normally require shorter distance flying.

Training Features

• This course helps identify objects and the use and coordination of cameras and aircraft orientation.
• These types of courses are in isolated and controlled environments, which is great for beginners.
• These courses are designed for different types of aircraft and different sizes.
• This training has very tangible and measurable results that can be tracked for progress.
• Training exercises can be performed at different aircraft speeds.

The NIST UAS Training Standard Falls Short On Many Critical UAS Training Elements

The NIST training is missing a large number of crucial elements outlined above in regard to a training program. We feel the NIST standard is too narrowly focused on flying maneuvers and is often more complicated than it needs to be.

• NIST claims there are 5 elements to remote pilot flight proficiency and using our base model as a reference we strongly disagree with that assertion. NIST does not cover the wide range of proficiency milestones that must be achieved.
• NIST training courses are most effective for short flight training, usually less than 200 feet long.
• Although it is possible to embed apparatus features into different training scenarios that have longer flight missions. In our opinion, this type of training does not prepare remote pilots for a large number of realistic flight conditions. Most commercial flights will be more than 500 feet in distance or further.
• Flying a drone on a course at shorter distances does not give realistic flight sight pictures that a remote pilot experiences in many types of commercial flights.

Longer distance flight missions cover the following type of public safety flight missions:

• Search and Rescue
• Inspections including Bridge, Cell tower, Power plant, and Tall buildings
• Water rescue including rivers, lakes, and dams.
• Large event monitoring
• Wildland/brush fires

• By NIST’s own admission they are not trainers, they are robotics researchers.
• A NIST course can’t teach a drone pilot to properly use ATTI in real life situations because a remote pilot has to develop skills for maintaining slow speeds at far distances. The perception of how fast you are flying at further distances is a key element that requires practice exercises that takes these elements into account. Performing effectively at long distances can’t be simulated with a small course. Learning orientation awareness and depth perception awareness through site pictures such as the horizon for reference are all extremely important elements that must be incorporated.
• The NIST training program also lacks critical ground school to learn how to properly perform flight missions. Effective UAS training goes way beyond stick proficiency and we are not sure how anyone can develop UAS training standards without the required ground school element. Thus supplemental training is required which requires an organization to piece mail a training curriculum together which can be problematic.
• A NIST course can’t prepare a remote pilot on how to properly fly at night or to prepare a remote pilot on how to recognize and mitigate the effects of night flying.
• The NIST course is a bit more complicated than needed for the average remote pilot. Our approach for drone operators is to keep it as simple as possible. The NIST course can be intimidating for many students and in our opinion is a bit over the top for students learning flight maneuvers.
• Access to a course like this for students will be limited. We prefer training exercises that students can recreate in any open area.

CONCLUSION

The NIST training standard has good intentions with some levels of effectiveness but it requires a substantial supplemental training element that must be incorporated carefully if it’s going to be used at all. It is impossible to realistically engineer an “effective” UAS training course that’s small in size due to the inherent characters most commercial flights have.

We feel there are more effective ways to train remote pilots and the importance of keeping training as simple as possible can’t be overlooked. IF these training processes were efficient and realistic, we would absolutely incorporate these courses in our multiple training facilities across the country.

We believe that there should be one master training program for internal drone integration of any organization which includes one master training authority that oversees all aspects of training for that organization. It is very easy to have too many cooks in the kitchen. The training program should be using various levels of training and not relying heavily on one component.

It is the combination of all aspects of training that gives an organization the best training program. If an organization wants to use a system like NIST, the organization must ensure that all aspects of training that NIST does not cover are performed efficiently and no parts of the training program are compromised.

Our Recommendations

Instead of using a course like NIST we highly recommend high-end flight simulators for supplemental training.

The DJI Enterprise simulator is the ideal supplemental training aid that covers all the aspects of the NIST training in a very controlled atmosphere. Operators can train anytime regardless of weather conditions and is a training aid that we really feel works effectively. There are a large number of training modules and exercises that can be performed on the simulator.

For a complete training package we recommend the following elements:

• Personal 1-on-1 instruction
• Flight instruction with a low Instructor to student ratio (no more than 6 per instructor)
• Advanced ground school theory and training – How to do jobs from start to finish
• Practicing in between formal lessons with small teams to acquire 10 flight hours of training
• Use of DJI practice simulator to supplement training
• Situational and team tactical training exercises.
• Night flying instruction and flying

Our Credibility And Background

Our top flight-instructors have collectively logged over 18,000 flights and have flown more than 5500 hours since 2009. We have worked in all types of applications such as inspections, search and rescue, accident reconstruction, project documentation, and mapping.

Our staff did not accumulate this level of experience by flying a drone inside a city park, or on a computer simulator. Our trainers have flown aerial content missions for more than 1000 clients to date.

Steel City Drones has also trained more than 1000 remote pilots since 2016 and has set up drone programs for more than 25 Public Safety agencies. Training this volume of students has given us clear indicators of how remote pilots learn, process information, and develop their skills.