A Deep Learning-Based System for Product Recognition in Intelligent Retail Environment

This work proposes a pipeline that aims to recognize the products in a shelf, at the level of the single SKU (Stock Keeping Unit), starting from a photo of that shelf. It is composed of a first neural network that detects the individual products on the shelf and has been trained with the SKU110K dataset and a second network, designed and built within this work that associates to the single image created by the first network, an embedding vector, which describes its distinctive features. By obtaining this vector of the input image, it is possible to measure the similarity, by means of the cosine similarity, between this vector and all the embedding vectors in the comparison dataset. The vector with the highest cosine similarity is associated to an image labeled with the EAN (European Article Number) code and, therefore, this EAN will be that of the input image. Given the particular task, there are not currently any dataset able to meet our requirements as they have not such a granular level of detail (EAN labeled), so a new properly designed dataset is created to solve this task.