DEVELOPER BLOG

Synthetic Data Cloud for real-time Driver Monitoring System (DMS) development and validation of Vision AI models

By SKY ENGINE AI   01 September 2022

Driver Monitoring

|

Generative AI

|

Automotive

|

Synthetic Data

|

Deep Learning


SUBSCRIBE
       

 

Driver monitoring systems (DMS) based on the AI and Computer Vision (CV) are quickly becoming standard on board of vehicles of any type and model. The safety standards push the sensitivity and alerting adequacy to play critical role when assessing the DMS's applicability to broad adoption. The drivers do not need to face additional distraction coming from misleading alerts as their attention should be laser focused on the driving task. Although there are several apprentice DMS systems on the market yet, the gap still exists but the good news is that it can be filled by the deep learning in virtual reality with Synthetic Data where the AI models are tested and trained prior to the real-world deployment leading to improved adherence to the regulations, better interoperability with ADAS systems, and most importantly thousands of fatalities can be prevented.

 

Synthetic Data Cloud for real-time in-cabin monitoring system development and validation of vision AI models

The Driver Monitoring System (DMS) will become a standard safety component throughout the world as early as 2023. In the United States the Department of Transportation will compel to begin developing guidelines and rules to combat distracted and intoxicated driving, as well as to modernize the New Car Assessment Program (NCAP). Initially, European regulations will apply to distracted and sleepy driving. Eventually, Europe will need impairment detection systems to cover alcohol and drugs.

There is always time to consider improvements in safety through this technology. According to the National Highway Traffic Safety Administration, distracted driving was responsible for over 3,000 deaths in 2019, a 10% increase over 2018. Drunk driving has also grown. In 2019, 10,142 people were killed in drunk driving accidents. Early projections for 2020 show a 9% rise in DUI fatalities. Many people anticipated road deaths to decrease during the epidemic, but the reverse happened.

To further boost the reliability up to higher Safety Levels it is critical to develop scalable, complementary and interconnected AI models with high adequacy. This can be mostly ensured by training sensors in a virtual environment prior to the real-world deployment. The SKY ENGINE AI platform enables Data Scientist friendly development of such AI models using platform's deep learning in virtual reality technology for computer vision applications. Driver monitoring systems usually employ RGB or infrared (IR) cameras and computer vision software to guarantee that the driver is awake and aware, and is paying attention to the road. The system may be configured to do a sequence of consecutive actions, beginning with a mild alert or warning and advancing to slowing or halting the vehicle if the driver is no longer able to control it.

Key benefits of the DMS/OMS for manufacturers, fleet owners, insurers, drivers and occupants 

The key benefits of using the driver and occupants monitoring system include:

  • Increased safety of the driver, passengers and others on the road by activities, emotions, attentivness and drowsiness detection
  • Alerting driver before collisions happen – avoid collisions than report on after the fact
  • Compliance to General Safety Regulations (GSR)
  • Improved comfort and experience by driver and occupants identification for settings personalization with i.e. facial recognition
  • Supports autonomous driving L3+
  • EuroNCAP 4-5 star rating relevance
  • Helps drivers improve their skills – especially important for newer drivers
  • Track and confirm drivers following the regulations and procedures in fleet safety policy
  • Highly decrease operational costs, reduce insurance claims and expenditure, reduce collisions
  • Cost-efficient system based on scalable, modular approach and seamless integration with exisiting systems (ADAS)
  • High flexibility of adaptation to the universe of models, markets, and vehicle interiors
  • Driver's position, weight and pose monitoring (3D skeleton) to facilitate optimization of the safety systems

 
SKY ENGINE AI approach to the in-cabin monitoring system development

For the human driver, the scene is too complex, reactions too slow and timeframe too short to provide quick and adequate reactions in any conditions. The combination of the DMS with Advanced Driver-Assistance Systems (ADAS) can use predictive analytics to further boost crash-prevention capabilities. ADAS incorporates technology like as automated emergency braking, blind-spot recognition, and lane departure alerts and its capabilities may be further enhanced when integrated with the DMS. For instance, after detecting drowsiness or high distraction level the DMS can send signal to the ADAS to increase the distance to preceding vehicle first before stopping the car completely.

Such DMS can be developed at a fraction of cost of real data acquisition and labelling providing complete solution for driver monitoring and in-cabin sensing. Also, it is viable to employ low-cost cameras in the car as synthetic data and complex vision AI algorithms can be created in the SKY ENGINE AI platform to allow reliable driver monitoring.

The computer vision AI models for the Driver Monitoring System (DMS) that can be developed and effortlessly retrained (when necessary) in the SKY ENGINE AI Synthetic Data Cloud:

  • Identification of the driver in order to allow the vehicle to automatically restore its preferences and settings
  • Monitor driver fatigue to alert him when potential drowsiness situation is detected
  • Monitor driver attentiveness by ensuring he’s keeping his eyes on the road, hand on the wheel and that he/she is aware of any dangerous situation
  • Recognize and track any additional objects and elements floating inside the car
  • Monitor actions and activities of children, babies, toddlers in a child seat
  • Detect and monitor activities/actions by pets
  • Pilot a user interface using gestures or gaze by automatically selecting HMI areas, highlight important features on the road, etc.

In addition, training the AI models for the DMS in the SKY ENGINE AI platform preserves the privacy as these models are trained in virtual reality with synthetic data: humans, cars, objects, etc. Virtual vehicle interiors serve the purpose of training, testing and validation of the AI models before deployment in a real car saving hundreds of hours and highly reducing the overall cost of such DMS.


Driver and occupant state (i.e. emotions, drowsiness, distraction)

In-cabin monitoring systems capable of detecting driver and occupant state, i.e. drowsiness, emotions or distraction can have their vision AI models trained on synthetic data with dense 3D key points and rich ground truth. Synthetic, but yet very realistic humans have been created, for the job of simulating the driver and the occupant behavior in the car, performing several activities that may also be prohibited by law (depending on the country and regulatory). These should be accurately detected by any modern monitoring system like the DMS. The synthetic humans were created along with the entire car interior and context under changing outside environment conditions to preserve the impact they have on the trained vision AI models.

SKY ENGINE AI in-cabin monitoring occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
Violent, angry, expressive, tension
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Distracted, phone calling, hands off the wheel
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Drowsy, tired, lack of focus
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Yelling, angry
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
Drowsy, tired, female driver in face mask
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS with 2D/3D bounding boxes
Distracted female driver, using mobile phone
 

SKY ENGINE AI in-cabin monitoring occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM distraction drowsiness


Lighting conditions – internal and external

In the SKY ENGINE AI, we have developed a complete toolchain of components for generating synthetic environments such as cars of different models/makes, objects and external environments, and any genuine interior may be replicated to help our customers creating the AI models faster. We employ procedural object geometries and materials arrangement with various randomized factors, such as camera angles, lighting, daytime and external surroundings, and introduction of customised items into the scene, to obtain highly balanced datasets and to ensure a high degree of diversity in the data itself.

SKY ENGINE AI in-cabin monitoring occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
Car interior light on, delicate external light spills out in the car
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Car interior light on, extra light in the interior from passing cars
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Car interior light on, external road lamps lighting effect on interior
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Car interior light off, reaction to approaching car dazzling lights
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
Driver close up, dazzling light from passing cars, road lamps light spills out in the car, seatbelts off
 
SKY ENGINE AI synthetic data cloud for AI models training, testing and validation for driver monitoring system - DMS ADAS for car OEM
Driver close up, varying light intensity in the car, on the driver and objects
 

Illumination variation introduced to the synthetic data generated/simulated in the SKY ENGINE AI cloud can also include:

  • Light sources
    Point light and spotlight light sources with energy dissipation modelling are supported.
  • Environment lights
    Both image and HDR environment lights are supported.
  • Emissive objects
    Light sources with custom geometry (defined in mesh) and surface luminance (defined in material) could be used.

 

SKY ENGINE AI in-cabin monitoring occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM


Gaze estimation package and Facial landmarks

SKY ENGINE AI Synthetic Data Cloud supports multiple ground truth types including 3D facial landmarks (key points) according to a given specification. Also, gaze vectors can be obtained for drowsiness, distraction estimation or personalization. These ground truths are available for imagery data generated in different modalities (i.e. RGB, NIR, etc.).

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS with 2D/3D bounding boxes


Driver and occupants’ activities simulation for action recognition

Several, described earlier, synthetic data simulation functionalities are required and the vision AI models tasked to perform detection and recognition of multiple actions taken by the driver and occupants with unprecedented accuracy, including capturing changing situations in the interior of the car, have to be rigorously trained with myriads of samples of well-balanced and diverse data reflecting these activities. Eating different types of food, drinking, smoking (electronic devices, cigarettes, etc.), calling, using phone/laptop, carrying different objects, and many more. Driver and occupant 3D pose estimation setup with 3D skeletons and configured key points model. In addition, there is head pose estimation setup with centroid position and Euler angles of head.

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM


Driver/occupants objects handling. Varying weather conditions, time of the day and exterior environments

Driver and occupants can handle hundreds of different objects in the simulations, including but not limited to cups, bags, smoking devices, food, keys, mobile phones, tablets, cigarettes, drinks, books, etc. Any larger objects that can be placed on the car seats can be included, such as handbags, boxes, grocery bags, laptops, sport equipment, children toys, blankets, and more.

Changing weather conditions, time of the day can be introduced into the simulated scene including common variations like cloud cover, rain, haze, snow, dust or sunny day for any exterior environments i.e. urban or countryside. In addition, light intensity and spill out in the car originating from these weather conditions can also be parametrized and strong sun effects on car cabin, objects, child seats, main seats and occupants are possible.

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
Snow outside, urban environment driver and occupants, smoking activities
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Summertime, sunny weather, countryside, driver handling coffee cup
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Driver and occupant carrying phones, floating objects on the front seat, time of the day – dusk
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
Driver eating, urban environment, cloudy weather, in-cabin objects reflecting in the side and roof windows
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
Driver in glasses carrying drink, occupant making phone call, floating objects on the front and rear seats, rainy weather
 
SKY ENGINE AI synthetic data cloud for AI models training, testing and validation for driver monitoring system - DMS ADAS for car OEM
Driver looking back, floating objects on the front seat (bag), child seat in rear, time of the day – evening, urban scene
 

SKY ENGINE AI Synthetic Data Cloud can generalize to any lighting, background, time of the day or weather conditions reflecting also the impact of that evolving light in the scene on the car’s interior and all the humans, pets and objects present.


Multi-modality support, custom camera/sensor characteristics and pixel-perfect and diverse ground truth

SKY ENGINE AI Synthetic Data Cloud is offering physically-based simulation environment enabling production of data for numerous actions and activities of a driver and the occupants in the in-cabin context. Such datasets can be simulated and generated in different modalities including Visible light (RGB sensors); Near infrared (NIR); Radars; Lidars and more (i.e. X-rays, UWB, etc.). SKY ENGINE AI is sensors agnostic providing tools that make it possible to re-create custom real camera/sensor’s characteristics including focal length, field of view (FOV) and relationship between them, quantity of random noise, modulation transfer function (MTF) of the lens setup, perspective, aspect ratio, available contrast and more.

The platform allows generating pixel-perfect ground truth for several in-cabin objects which can be expressly simulated with depth maps, normal vector maps, instance masks, bounding boxes and 3D key points ready for AI training. Moreover, there are render passes dedicated to deep learning, animation and motion capture systems support and determinism with advanced machinery for randomization strategies of a scene parameters for active learning approach.

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM

Synthetic images above: Car interior, driver, two occupants on a rear seat, objects and ground truths simulated and generated in the SKY ENGINE AI Synthetic Data Cloud, 6 images: (Left) 3D key points on humans; Depth map; Semantic masks on objects, humans and separately on multiple components of these objects and humans. (Right) Smoking action and objects in the car (NIR); 2D bounding boxes and masks (NIR); Normal vectors map.

SKY ENGINE AI in-cabin monitoring driver and occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM


Keep your seat belts fastened

SKY ENGINE AI provides simulated seat belts and their status (on, off, incorrectly fastened) for driver and occupants including children in child seats with separate harness. Available in numerous realizations, adaptive to human’s geometry using generative simulation for perfect body fit including driver/occupant activities.

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM


Gestures recognition

Gestures simulation for i.e. hands on wheel monitoring, indications, pose estimation, objects handling, touching controls and panels, etc. Includes detailed 3D skeleton of hand, palm and fingers. Can serve hand/palm pose estimation models, or spatiotemporal model operating on extracted position of joints.

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM

 

Driver gesture recognition AI models training with synthetic data in the SKY ENGINE AI Synthetic Data Cloud (video). 3D pose estimation (keypoints and skeleton). (Left) Infrared synthetic data, (Right) Visible light data.


Baby, toddler, younger child on board

Children at different age and with variety of settings can be simulated in child seats of numerous models/makes and carrying out several activities like playing toys, using phone or tablet, eating, reading, drinking including gestures and more. Simulated synthetic data can serve 3D pose estimation with dense child skeleton annotations, activity tracking with spatiotemporal action classification models operating on extracted position of joints. Seat belts in child seats can be separately simulated with their annotations.

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM


Pets and accessories in the cabin

Pets of different kind and breed can be included in the scene (i.e. dogs, cats, etc.) with accessories i.e. animal carriers. Also, interactions of a driver and occupants with pets and/or accessories can be simulated.

SKY ENGINE AI in-cabin monitoring driver occupant monitoring DMS synthetic data cloud for vision AI models training and validation for cars, trucks, trains in automotive car OEM

 

SKY ENGINE AI synthetic data cloud for AI models training and validation for driver monitoring system - DMS ADAS car OEM
Cat sitting on a front row seat, dog and cage on 2nd row (RGB)
 
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
Animal carrier on a front row seat, cat and dog on 2nd row (RGB)
 
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
Cat in the animal carrier on a front row seat (RGB)
 
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
Semantic masks on cat, dog and animal carrier (RGB)
 
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
Animals in the cabin (NIR)
 
SKY ENGINE AI synthetic data cloud for AI models training for driver monitoring system - DMS ADAS car OEM
Dog and animal carrier on the 2nd row seat (NIR)
 

Meet our Synthetic Drivers and Occupants in Context

The SKY ENGINE AI Synthetic Data Cloud enables creation of very realistic humans and objects to propel development of next generation in-cabin monitoring systems. Meet Tina and Mark, who have been created, along with other hundred drivers, for the job of simulating the driver and the occupant behaviour in the car, performing several activities that may also be prohibited by law (depending on the country and regulatory), but should be accurately detected by any modern monitoring system like the DMS. These synthetic humans were created along with the entire car interior and context under changing outside environment conditions to preserve the impact they have on the trained vision AI models.

SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with semantic masks and 3D key points
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS with 2D/3D bounding boxes
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS – Complex ground truth system with depth maps
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS with instance masks

Simulated RGB synthetic data for in-cabin monitoring with highly complex pixel-perfect ground truths generated in the SKY ENGINE AI cloud, 8 images: (Left, top-bottom) Meet Tina – Driver and Occupant; Driver and car interior – camera in the center stack rear view mirror; SKY ENGINE AI complex ground truth system – semantic segmentation masks for separated body parts and 3D key points including invisible face landmarks; Ground truth – depth map; (Right, top-bottom) Meet Mark – Driver; Driver and car interior – camera in the console; Complex ground truth system – 2D bounding boxes; Semantic segmentation masks.

SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS
SKY ENGINE AI synthetic data for AI models training for driver monitoring system - DMS ADAS

Synthetic data of car interior, driver and objects (i.e. bag, keys, mobile phone, etc.) and ground truths simulated and generated in the SKY ENGINE AI platform, 6 images: (Left) Drowsy male driver (RGB), Drowsy male driver (IR), Semantic segmentation, (Right) Keypoints and gaze estimation, Distracted female driver, Depth map.

Driver-monitoring systems capable of detecting drowsiness or distraction are only the beginning. As these technologies develop, they will become a part of a larger interior sensing platform that offers personalization, improved safety, infotainment, and even communication with smart home systems. The DMS can recognize the driver and enable personalisation to change the seat, temperature, side mirrors, and so on to the driver's preferences. The technologies will be able to determine if the driver is intoxicated or experiencing a medical emergency. Drivers will be able to operate some functions in the car by using their eyes or gestures and  the camera can watch both the driver and the cabin, allowing it to recognize whether a kid has been left in a car seat, assess whether an essential object has been forgotten, or assist in personalizing infotainment, HVAC, or other in-cabin capabilities.

Synthetic data simulation & ground truth generation for the DMS

SKY ENGINE AI Synthetic Data Cloud offers efficient solution for simulating data in several modalities including ground truth generation of any kind as yet outlined, so that the computer vision developers can quickly build their data stack and seamlessly train the AI models covering numerous situations and corner cases in the dataset. The parameters that can be modified within the scene created in the SKY ENGINE AI cloud include following:

  • Rendered image resolution and ray tracing quality.
  • Scene textures resolution (full, reduced for space and time optimization).
  • Scene textures parameters (car interior materials, patterns, colors).
  • Outside environmental maps type.
  • Environmental light intensity.
  • IR point lights strength, angle and direction.
  • Lens parameters: type (pinhole, fisheye, anamorphic), focal lengths, principal point, distortion coefficients (radial, tangential).
  • Modality selector (visible light, near infrared).
  • Camera position and orientation (like rear view mirror, console, etc.).
  • Camera randomization ranges (X, Y, Z, roll, pitch, yaw).
  • Shaders’ parameters (clearcoat level, sheen level, subsurface scattering level, light iterations, antialiasing level, and more).
  • Post Processing (tone mapping, AI denoising, blur / motion blur, light glow).
  • Occupancy probabilities on each seat (adults, children on child seats, empty child seats, big items, piles of items, empty seats). For items, also between seats.
  • Driver and occupant action probabilities (drinking, eating, smoking, driving, idle, looking around, grabbing an object from another seat etc.).
  • Seatbelt status (fastened/unfastened) probabilities.

Note: All parameters can be defined as the rules for randomization (range, predefined distribution, custom probabilistic distribution).

Facial expressions simulations: drowsiness, yawning, smiling, blinking, reaction to dazzling light, looking around, checking mirrors – in-cabin driver monitoring scene at night, car interior lighting on (video).

 

Universe of applications: insurance, cab, lorry, train, airplane fleet

The SKY ENGINE AI platform can effectively be used to simulate Physically-accurate, photo realistic synthetic data in the RGB and IR modalities enabling accelerated training of the accurate AI models in the driver monitoring applications. These computer vision models, when incorporated in the DMS/OMS system, enable real-time assistance in identifying in-car activities and help recognizing event risks, such as their probability, direct cause, likelihood of potential injuries and severity accumulated over pre-defined period of time. Accidents can be predicted, managed, and, at best, largely prevented. Insurers may be advised of the most likely events, given on-the-road and in-cabin circumstances, with a high level of accuracy. This can assist in discussing policies and applying discounts to the drivers or fleet owners reinforcing strict safety measures, noticing less issues, and recommending additional training assistance. Such system can apply driver scoring for post-accident analysis of the behaviour to further optimize insurance policy and its cost, which can be especially interesting for fleet owners. In the premium offering, drivers can get a monetary value in the form of monthly cash-backs, based on their road safety performance measured by accurate evidence-based risk indicators for insurance companies.

The DMS is expected to keep all the data aboard the vehicle to avoid triggering privacy concerns. However, to dispute any insurance claims it could be employed to work with recording mode on. Fleet owners or insurance companies can get insights on the detailed status of the driver and can alert a dispatch center or the driver to deliver real-time life-saving alerts. Insurers can offer Pay-How-You-Drive (PHYD) policy models to enable accurate evidence-based risk indicators including drivers scoring. Next, the drivers can get a monetary value in the form of monthly cash-backs or policy discounts, based on their road safety performance.

Let us know about your cases and get access to the SKY ENGINE AI platform or get tailored AI models or synthetic datasets for your driver and occupant monitoring applications. As we support much more industries than just an automotive a broad range of data customization is available even for specific sensors and environments.