English 
German 
Japanese 
Chinese In the last couple of decades, road vehicles have increasingly been developed to include smart driving assist features to help prevent driver fatigue and help to avoid accidents with other vehicles and static objects – these are known as advanced driver assistance systems (ADAS).
The relative simplicity of ADAS testing
Track testing of these ADAS features has matured and standardised such as with test definitions from Euro NCAP, NHTSA, and others. Using mobile platforms such as the Guided Soft Target (GST), LaunchPad, Soft Pedestrian Target (SPT), or real vehicles driven by AB Dynamics’ driving robots, these tests can be synchronised perfectly around a vehicle under test (VUT) using our patented Synchro software.
These innovations have allowed ADAS tests to be performed precisely, efficiently, and safely. Track testing an ADAS feature using one of our guided platforms has remained manageable since there is only one VUT and one other test object to consider - there are relatively few permutations to deal with in terms of the relative trajectories between the VUT and platform. For example, an automatic emergency braking system only needs to consider the relative speed, distance and time-to-collision of the car directly in front.
To an autonomous track testing future
The ultimate evolution of driving assistance features is to allow a vehicle to autonomously drive itself on public roads whilst not being a hazard to other road users – so-called autonomous vehicles. Additionally, both autonomous and human-driven vehicles will be connected in the next few years using Vehicle-to-Everything (V2X) technology. Vehicles will communicate with each other and to roadside infrastructure, such as traffic lights, to warn of hazards and dangers in the road ahead. These new complex technologies also require testing to ensure safe and reliable operation.
So how do we test these new systems? As we see it, autonomous vehicle testing has so far branched into four main categories, each with benefits and drawbacks:
- Simulation Data Farming is where many computer servers running software such as the rFpro Data Farming solution provide a simulated environment of scenery, moving objects, vehicle dynamics and sensor output to virtually test autonomous vehicle driving software. This is a software-only solution that can run faster or slower than real-time, and where an actual vehicle and sensor hardware are not required. This is used as an algorithm validation tool, but also to train the vehicle’s driving software by highlighting correct and unsafe behaviour, in a process known as supervised learning. This solution can significantly increase the volume of testing by several orders of magnitude but requires an accurately modelled world.
- With Vehicle-in-the-loop (VIL) testing, an autonomous vehicle is configured by engineers to operate in a lab, either when stationary or when driving on a dyno rig. Sensors that are built into vehicle view a simulated world of challenging driving scenarios - a blend and hardware and simulation. For example, the camera system on the autonomous vehicle would view 360° simulated imagery, or where synthetic GPS data is broadcast into the vehicle's antenna. VIL testing adds an element of realism since actual hardware used, whilst also making the test set-up and configuration much easier for the test engineers. But not every sensor hardware type can be used effectively in this way; and it does not provide all of the systemic subtleties of the real world, such as how a vehicle driving over a pothole causes extra motion blur on the on a camera sensor.
- Real-World Fleet Testing involves autonomous vehicles that are permitted to drive in real-world traffic under the supervision of a safety driver who is there prevent accidents if the driving software fails. Mistakes and good driving behaviour data are recorded, which can help to provide long-term statistics about performance, such as the mean time between failure (MTBF) - autonomous vehicle manufactures tend to talk about ‘disengagement rates’ (miles between incidents where human driver is forced to take back control). It also helps to train the vehicle’s driving software by feeding driving data back into the autonomous vehicle software using a supervised learning technique or to allow software engineers to diagnose and fix problems in the software. The randomness of the real world, however, makes it nearly impossible to check if a correction to the autonomous vehicle driving software has worked as intended as that exact situation may never be encountered again.
- With Track Testing, an autonomous vehicle is configured to drive on a test track along with a combination of soft guided robot targets, dynamically controlled roadside infrastructure and static road furniture. The aim is to make the scene as realistic as possible to what an autonomous vehicle will experience in the real-world – yet in a highly controlled and safe environment. This setup provides precise and realistic test scenarios but is time-consuming in comparison to simulation. These tests can be repeated and adjusted by the engineers if necessary to allow fine-tuning of software algorithms or provide validation with confidence.
Track testing is the only reliable way to test the most dangerous scenarios
With track testing of autonomous vehicles, however, the sheer number of permutations, variations in the scenario, and increase in road actors may seem like an impossible task – there are a near-infinite number of possibilities to test!
Whilst a large proportion of the testing can be carried out in simulation, the most hazardous or challenging scenarios will likely require track testing – and even with this constraint, the task remains daunting. Yet, there is no safer and more reliable way to test the complete autonomous vehicle system with close-in dangerous scenarios than by performing track testing with a swarm of robot-driven platforms.
A single scenario with one AV under test could require several platforms like the GST and LaunchPad for close-in manoeuvres, with a further fleet of real vehicles as additional background traffic actors that are either robot or human-driven. This scenario may need to be replayed several times throughout the day to get consistent and valid confirmation that the test has been passed. The scenario may be further complicated by adding other AVs into the mix when testing Vehicle-to-Everything (V2X) technology. There may be tens or hundreds of scenarios to test, so it likely that duplicate AVs are tested simultaneously on segregated parts of the track by different teams.
How to manage this complex new challenge
So how do we cope with this complex new world of track testing? Given the requirements for track-testing autonomous vehicles, our software and hardware solution for safety, logistics, and management of platforms must also evolve to cope with these new challenges.
With our Ground Traffic Control (GTC) system, however, we have not just made an evolutionary step, but a revolutionary step. GTC has been designed from the ground up, for the future of robot track testing, such as for the autonomous vehicle testing scenario described above. GTC is a real-time test management and monitoring solution with many features to facilitate this new complex testing regime.
GTC uses a hub-and-spoke model where the real-time GTC Server will automatically connect to all robot-controlled test vehicles including the VUT. The GTC server continuously monitors the robot systems throughout the day. Any operator at a base station can monitor, command, and control vehicles using the GTC Software. The Windows-based GTC Software provides a smooth and intuitive interface with asset management, live platform state, and a map view that provides a smooth and continuous map from Microsoft Bing.
With so many robot-driven platforms on the track, you might ask, “Surely there are going to be more accidents?”
Well, with GTC, we have solved this problem by including an advanced collision detection and prevention algorithm that will monitor the trajectory of each vehicle to ensure they can pass near each other safely without colliding. The system will allow AB Dynamics’ driving robots and platforms to gently brake to yield, merge and queue along their paths. This innovation will dramatically reduce the risk of vehicle accidents on the test track. It does not require any extra sensor technology such as radar or lidar because we already know the accurate positions using accurate and reliable GPS based navigation systems.
This will help prevent collisions between vehicles, but how can we possibly manage so many objects on the track with a small team? In our experience, a single base station operator could only monitor and manage three to four robot platforms at a time, so how could we possibly manage a large test with 16 platforms?
The GTC system will allow multiple base station operators to connect-in and see the whole picture on the track. They can monitor all vehicle movements or create testing groups where they take control of any number of vehicles to start and stop their tests. This allows multiple, but separate tests to happen on the track whilst giving situational awareness to each operator. For large testing groups, multiple base station operators can team-up and control a more substantial robot testing fleet by sharing the workload.
GTC is not just a boost to autonomous vehicle testing however, it will also revolutionise the way that other track testing activities are performed. GTC can equally increase productivity and safety of other activities such as durability and fatigue testing, dynamics testing and general proving ground management.
Get in touch with AB Dynamics to explore how your track testing could be revolutionised by GTC.
To keep pace with other countries at the forefront of AV development and to fully explore the opportunity that ICVs (Intelligent Connected Vehicles) present for the country’s industry, China has recently created national guidelines for the evaluation and testing of self-driving cars.
Unlike some other countries, China only allows manufacturers to evaluate AVs on the public highway once they have been issued a licence to do so. Permits are granted by third-party authorities that have been appointed by the government to certify testing in designated regions. For Shanghai, the exclusive supplier of licences is Shanghai SH Intelligent Automotive Technology Company Ltd.
SH Intelligent operates the first national ICV pilot zone authorised by the Ministry of Industry and Information Technology (MIIT). Since 2018, SH Intelligent has supported the Shanghai government, determining if a manufacturer can conduct AV testing on Shanghai’s public roads through its own comprehensive, on-track testing programme. Once clients are certified, they can test on 37.2 kilometres of the city’s roads, spanning urban main roads, secondary trunk roads and industrial parks.
Enabling SH Intelligent to conduct its certification process is a suite of products by AB Dynamics.
Dr Lin Li, deputy General Manager, SH Intelligent, has overall responsibility for AV testing and has worked in Active Safety development since 2009. Here, he discusses the challenges and the solutions facing the evaluation of autonomous vehicles in a highly populated environment where the expectation is that autonomous vehicles will very soon be the norm, rather than the exception.
What is driving China’s approach to licencing AV testing before evaluation is permitted on the public highway?
“Mainly, from the government and public’s point of view, it is the ideal way to ensure basic safety in public road tests.”
Why do international manufacturers choose to test in China, where certification is required, rather than in other countries?
“China already has a large amount of policies and measurements in place to facilitate and encourage the testing of autonomous vehicles and there is a network of dedicated, off-road AV testing centres throughout the country. Additionally, the population is known to be accepting of new technology, which is particularly important as China has potentially the largest global market for AV usage. Finally, unlike the US or EU, China has an incredibly complex road network which is highly congested with vehicles and pedestrians, the behaviour of which can be vastly different to what manufacturers may have experienced elsewhere in the world.
What does achieving AV testing certification involve?
“Evaluation takes place at a government-organised testing facility. We are the sole provider of AV testing certification for Shanghai and our testing ground, named ‘A Nice City’, is China’s first and most advanced ICV (Intelligent Connected Vehicles) testing zone. The process involves testing the AV on track in a variety of scenarios, spanning normal driving – for example, following a car – and extreme situations, like cut-ins, where another vehicle changes lane in front of the AV. AVs can be assessed against 17 differing tests, incorporating nearly 100 test scenarios, with manufacturers choosing which ones conform to their requirements. For example, the 17th test certifies V2X (vehicle-to-vehicle / vehicle-to-infrastructure), but a vehicle without this technology may achieve certification if the selected tests are passed.”
How important is accuracy and repeatability to the testing process?
“It’s essential to ensure the test result is objective. For example, we repeat some tests as much as 30 times to ensure the results are consistent, so we can be confident the same result would occur in the real world. Additionally, the high levels of accuracy and repeatability allows us to conduct testing as efficiently as possible, which will prove even more important as AV testing intensifies. This is a notable reason why we use AB Dynamics products.”
What equipment is required to carry out the tests?
“We use a multitude of testing equipment to determine certification and, with what is potentially at stake, it’s imperative that we use the best available. Our preference for AB Dynamics products is influenced by the company’s reputation and the usage of its equipment by the world’s top 25 automotive manufacturers. AB Dynamics products feature when conducting almost all the 17 test categories. Presently, we have the company’s SR60 steering robots, GST (Guided Soft Target) and will shortly be adding LaunchPad.”
Why, specifically, did you choose AB Dynamics equipment?
“Having the steering robot is essential for ICV testing, allowing us to precisely control the vehicle motion as well as stability. The GST is a key piece of equipment too. Its resemblance to a conventional car and ability to move freely up to 360 degrees provides a great deal of realism and flexibility when conducting test scenarios. We considered similar products but found GST to be the most stable. Durability is also impressive – we used the GST for more than 2,000 tests in 2018.”
Does the certification process benefit from testing equipment originating from a single supplier?
“Absolutely. The products are designed to integrate with each other, which in turn makes the proving ground more efficient and reliable. This is particularly significant now that AV testing scenarios are becoming more complex, involving more and more elements to replicate what may occur on the road. For example, with LaunchPad we will able to add a Vulnerable Road User (VRU), making them ‘alive’ and able to move 360 degrees like the GST. In future, we will use the steering robot, GST and LaunchPad together to build complex scenarios.”
How high is the demand to test AVs in Shanghai?
“Shanghai facility has issued the first licences for self-driving cars in China. Since 2016 we have catered for more than 60 clients, spanning automotive, transportation and communication industries. We have issued licences to SAIC Motor, NIO, BMW, Momenta, Tusimple and Autowise.”
As simulation becomes increasingly accepted in terms of testing, do you foresee it playing a role in your certification process?
“Currently, we are focused on actual on-track testing in our certification process, but simulation is something that we may consider, especially in relation to the efficiency gains that it could potentially generate, we employ simulation in our R&D testing. It’s highly unlikely that simulation will replace track testing completely – many of the various laws and guidelines we need to adhere to demand that vehicles are physically tested.”
AV development is growing at a pace, how will this affect your function?
“We can only see demand for certification increasing. As well as enhancing our test equipment, we will be mirroring the increase in demand with the growth of our capacity. The area we are able to issue certification for in Shanghai is where 40 per cent of China’s overall automotive research takes place. We are planning to expand this area so that it covers 60 per cent, which in turn will see us able to cater for AV-testing clients who were previously out of our catchment area. Ultimately, we also wish to offer certification that takes in different types of roads, such as rural as well as urban environments, and increasingly cater for commercial vehicles as well as cars.”
What will be the biggest challenges to your operation in the future?
“Without doubt, it is the creation of highly-complicated testing scenarios. Simply installing and calibrating the testing equipment to command and monitor several elements can be very time-consuming. In terms of this, the compatibility of our AB Dynamics products is a significant bonus.
“As we’ve mentioned, demand is also likely to increase. It’s a challenge we are prepared for and even though certification isn’t required for on-road AV testing in some other countries, international manufacturers still see China as an essential test environment. There’s certainly the belief by some that whoever wins in China, wins on the world stage.”
About Shanghai SH Intelligent
Shanghai SH Intelligent Automotive Technology Company Ltd. operates the first national pilot zone (A Nice City) in China authorised by the Ministry of Industry and Information Technology (MIIT). It is the only third-party that is authorised by the Shanghai government for autonomous driving license plate issuing. It has established a test chain for autonomous driving vehicles from simulation, VeHIL lab, enclosed test track and open track testing. Meanwhile, it leads or contributes to the creation of testing standards by industry bodies or associations, which include 5GAA, ISO and ISMTA, China domestic national standard working groups lead by CATARC, plus alliance standard. Together with the whole industry, SH intelligent is exploring the future of autonomous driving.