Success Stories

Success Stories

Robotic Arm based Test Automation of Car Infotainment Screen

Robotic Arm based Test Automation of Car Infotainment Screen

The Customer was looking for automating the testing of their infotainment screens, as system testing and regression testing cycles were long as well expensive with sub-optimal level of test coverage. 

Solution Description

  • Designed computer vision software for Enabling Robot based testing for infotainment systems. 
  • The vision software combines real-time image recognition and text recognition which enables the robot to identify the on-screen elements and navigate through the menu

Outcome and Benefits Delivered

  • The pre-trained convolutional neural network model has given high accuracy results for screen/icon recognition than traditional methods.
  • Recognition and localization of elements on the screen.
  • Image recognition can be done in real-time from a live video stream.
  • Text localization and recognition.
  • LSTM and attention-based neural network for text detection
  • Real-time on-screen slider movement tracking using image processing.

The Solution / System Description


  • 4 axis DOF robot with customised gripper.
    • Payload 500g 
    • Max. Reach 320mm 
    • Position Repeatability(Control) 0.2 mm 
    • Communication USB / WIFI / Bluetooth


  • Industrial grade camera for image acquisition - 5MP, CMOS, 15.0 fps, 2592 x 1944, 1/4", Rolling Shutter
  • Neural network based image recognition for identifying icons and screens
  • Custom algorithms for measurements and position detection using OpenCV
  • Hand eye coordination of robotic arm with machine vision was implemented
  • Trained a custom neural network OCR on Google Cloud for high accuracy text detection for special fonts.
  • Automated tool for creating dataset for training OCR


  • Automated robotic testing of infotainment systems.
  • Machine vision in manufacturing.
  • Better quality control of touch screen related products.

Figure 1

figure 2

Download this Success Story
You may also like