Safety Validation Solutions for Automotive Perception Functions

aidkit is a ML Quality Ops platform built for ML Engineers and designed for their workflow.

Its features allow users to systematically ensure every safety-relevant aspect is covered, tested, and evidenced prior to deployment of a ML-augmented perception component.

Have a systematic method able to produce comparable and understandable statistical evidence

Discover all relevant ODDs

Rapidly yet thoroughly test perception components against all common and corner cases/scenes

Efficiently test your perception component in numerous ODDs

Manage an efficient data campaign that acquires and collects the right data

Get a Demo

Safety Validation Solutions for Automotive Perception Functions

aidkit is a ML Quality Ops platform built for ML Engineers and designed for their workflow.

aidkit’s features allow users to systematically ensure every safety-relevant aspect is covered, tested, and evidenced prior to deployment of a ML-augmented perception component.

Get a Demo

Safety assurance with aidkit

Validate Perception Function Safety across ODDs and Scenes before Deployment

green checkmark for correct results
red x for incorrect results
button to swipe left
button to swipe right

Model is detecting or classifying correctly

Model is failing to detect or classifying incorrectly

Training Data vs. Real World

Unexpected attributes in the real-world environment can lead a model to malfunction. Testing models in challenging scenarios before they occur in production prevents severe consequences.

Advanced Feature-set

aidkit ’til You Make It

Extract risk scores for safety argumentation

Provide explainable, evidence-based risk scores containing impact and probability of occurrence for every tested ODD scenario

Provide examples that challenge your perception component

Generate a wide range of corrupted images for average- and worst-case analysis (i.e., perturbation-based image operations)

Discover and explore new ODD scenarios

Combine existent ODD attributes to automatically identify ODD extensions, interaction effects, and correlations

Tailor your Data Campaign and augment training & test data

Know exactly what data to collect and label to meet your performance shortfalls and leverage the corrupted images your model could not handle for re-training

Track data lineage and fulfill audit requirements

Maintain an overview of existing models, model versions, data sets/splits ,and evaluations plus monitor interim pipeline results

Automate and accelerate perception component testing

Connect aidkit to your existing MLOps workflow via our Python-Client and scale your computing to execute testing faster

Your aidkit Benefits

aidkit Covers every Aspect for Testing your Perception System

Identify and test all relevant scenarios, whether known or unknown

aidkit offers easy testing of diverse scenarios, as well as the identification of new safe and unsafe areas via the combination of existing ODD attributes

Discover what data you need to improve and augment your data with artifacts

aidkit identifies the edge and corner cases where you need more data so your data collection and labelling is targeted, or import artifacts directly for retraining

Evidence for Safety

aidkit generates artifacts backed by empirical evidence and risk scores for each scenario allowing easy comparison and support for safety claims

Up-to-date on the latest standards and the means to test them

aidkit updates benefit from neurocat’s active engagement in developing the latest industry standards (e.g. ISO PAS 8800) which we integrate as state-of-the -art testing methods

Track, automate, scale

aidkit facilitates easy tracking of data lineage and metrics, monitoring of pipelines including interim results, automate iterative testing and retraining, and horizontal scaling

One Platform with two flavours for all your testing needs

Input- and framework-agnostic tooling facilitates testing all your use-cases in a single platform available as a REST-API you can integrate right in your workflow and a Web interface for configuration and tracking

Customized corruptions

neurocat will work with you on the approximation of your ODD catalogue with customized augmentation

Support for every iteration

neurocat is there to support you with the tailored solution projects you need, from aidkit integration to technical challenges of safety validation to results interpretation