Wearable Systems for Improving Museum Experience

In this chapter we present original approaches for the development of a smart audio guide that adapts to the actions and interests of visitors of cultural heritage sites and exhibitions either in indoor or outdoor scenarios. The guide is capable of perceiving the context. It understands what the user is looking at, if he is moving or is inattentive (e.g. talking with someone), in order to provide relevant information at the appropriate timing. Automatic recognition of artworks is performed with different approaches depending on the scenario, i.e. indoor and outdoor. These approaches are respectively based on Convolutional Neural Network (CNN) and SIFT descriptors, performing, when appropriated, object localization and classification. The com- puter vision system works in real-time on the mobile device, exploiting also a fusion of audio and motion sensors. Configurable interfaces to ease interaction and fruition of multimedia insights are provided for both scenarios. The audio-guide has been deployed on a NVIDIA Jetson TX1 and a NVIDIA Shield Tablet K1, tested in a real world environment (Bargello Museum of Florence and the historical city center of Florence), and evaluated with regard to system usability.