How can AI-based inspection systems be improved and validated before industrial deployment without disrupting production lines or investing in costly new experiments?
In this user story, we show how New Infrared Technologies (NIT) used AI-MATTERS to test and validate the performance of an AI-based infrared inspection solution, improving automatic defect detection accuracy and increasing industrial deployment readiness with reduced upfront risk or any production disruption at its clients facilities. AI-MATTERS provided access to both technology and domain expertise, data enrichment and benchmarking services to validate the solution under realistic industrial conditions.
New Infrared Technologies (NIT) is a technology provider developing AI-enabled infrared autonomous inspection systems for manufacturing. As a tech provider, NIT needed to technically validate and mature its AI-based defect detection solution, moving from a promising prototype towards deployable performance levels while minimizing development cost, risk and dependency on end-user production facilities.
The Challenge
The AI‑based infrared inspection solution for laser welding quality control suffered from limited and unbalanced datasets, particularly for defective cases. As a result, detection accuracy remained insufficient to support industrial deployment decisions.
NIT lacked access to large volumes of representative defect data and could not easily perform new experimental campaigns, which would have required repeated access to end‑user production facilities and could impact ongoing manufacturing processes. This uncertainty prevented objective performance validation and hindered further development and market uptake of the solution.
The Approach
Through AI‑MATTERS, NIT gained access to a Testing and Experimentation Facility (TEF) providing:
- Expert support for AI algorithm testing and validation
- Data enrichment through AI‑based synthetic defect generation
- Controlled benchmarking and performance evaluation at semi‑industrial scale
The collaboration focused on testing the AI solution accuracy through the progressive inclusion of newly generated synthetic defect data and validating performance through comparative benchmarking.
What has been tested?
The Technology
AI-based anomaly detection enhanced through synthetic data generation (GAN-based approach), applied to high-speed infrared imaging for laser welding automatic quality control.
The Use case
Validation of an AI defect detection solution by enriching limited real datasets with realistic synthetically generated images, improving the robustness and accuracy of welding defect detection based on a more complete and representative datasets for model training (specifically augmenting defects casuistic).
Experiment setup
Generative networks were trained to learn the distribution of real infrared welding data and generate new, realistic synthetic defect samples. The experimentation focused on the progressive inclusion of synthetic data into the training process, allowing systematic evaluation of its impact on model performance.
The original anomaly-detection algorithm and an improved version re-trained with enriched datasets (real data combined with AI-generated synthetic data) were validated through comparative benchmarking using the same fixed reference test dataset. This ensured an objective and reproducible assessment of performance improvements under controlled conditions prior to industrial deployment.
The Impact
The experiment resulted in:
- Improved defect detection accuracy, increasing F1-score from ~92% to ~97% (customers’ expectation to make the decision investment is to be above 95% accuracy levels)
- Technical validation of performance improvement through controlled benchmarking
- Avoidance of additional experimental campaigns, reducing time, cost and dependency on access to production lines
NIT received an enriched dataset, a re‑trained AI model and a benchmarking report comparing the original and improved algorithms.
Key Insights
- Synthetic data generation is an effective strategy to address data scarcity and imbalance in industrial AI applications.
- Controlled benchmarking is critical to objectively validate AI solutions performance and as selling argument for potential customers.
- Achieving near‑industrial performance levels is essential to engage end users and reduce adoption risk.
What’s next?
- The performance improvement achieved and validated during the experiment has generated renewed interest from NIT’s end users.
- Future testing activities may explore hybrid approaches combining synthetic data generation with targeted experimental data acquisition, supported by the capabilities and infrastructure provided by AIMEN through AI‑MATTERS.
- In addition, the TEF infrastructure could be used to demonstrate solution performance under industry‑relevant conditions, emulating specific end‑user remote welding scenarios.
Why AI-MATTERS?
AI‑MATTERS enables companies to:
- Test and validate AI solutions before industrial deployment
- Access data, domain and technology expertise, and benchmarking infrastructure
- Reduce technical and operational risk before investing
Through collaborative experimentation and validation, companies can move from AI prototypes to technically validated solutions with increased confidence.
FAQs
Two services were provided for this use case:
The TEF addresses data scarcity challenges through a combined approach:
- Real industrial data generation through controlled experiments,
- Synthetic data creation and augmentation, and
- AI model training, validation and re‑training.
These activities can be executed through interoperable digital pipelines with access to HPC resources (e.g. large or complex models), enabling companies to generate representative datasets and validate AI models under industry-relevant conditions before deployment.
For more information and a specific dive into your use case, we welcome you to contact us for a non-binding conversation.
Synthetic data does not replace real industrial data, but it can significantly complement it. In AI-MATTERS, synthetic data is used to enrich limited real datasets, especially for rare or hard-to-capture scenarios, improving model robustness while reducing the need for costly experimental campaigns. Hybrid approaches between synthetic and real captured data are the recommended strategy to limit the investment while guaranteeing the representativeness of datasets, and AI-MATTERS can support both.
The AI‑MATTERS Manufacturing TEF enables companies to test and validate AI solutions without requiring continuous access to customer production lines. Through industry‑relevant testing and experimentation environments, the TEF provides access to physical and digital infrastructures where AI models can be validated using real and synthetic data, controlled experiments and benchmarking. This allows technology providers and manufacturers to validate AI performance under realistic industrial conditions while avoiding disruptions to ongoing production.
About the Author
Project Manager at AIMEN
Ángel Pérez Mariño is a chemist with a PhD in Materials Science and an innovation project manager highly active in bridging R&D and implementation of AI and Industry 5.0‑aligned technologies. Through initiatives such as AI‑MATTERS, SURE5.0, AIRISE and BRIDGESMEs, he has contributed to the development of AI‑based solutions and technology roadmaps, supporting European SMEs in testing, validating and implementing advanced technologies for the industry.
Would you like to know how our project managers can help your organisation?
Contact us for a non-binding conversation: https://ai-matters.eu/contact/