CSI - Context Sensitive Intelligence

Today’s retail stores are facing new challenges. While a few years ago competitive advantages could be achieved by well-stocked product portfolios and bargain prices both domains have been lost to internet shops. To preserve and improve customer retention also in stationary retail environments a sophisticated customer relationship approach by customized onsite services is desirable. Therefore, comprehensive knowledge of the customer base is necessary. To gain knowledge about customers without using vague customer surveys or short-time observations an automated sensor-based solution is conceivable.
Because image sensors do not require any interaction with the customer they are predestinated for customer behavior analysis. The project “context sensitive intelligence” applies image sensors for recording raw data about customers within retail environments. The gathered data is used as input for computer vision algorithms as well as procedures from the field of data mining. Thereby, three dimensions of customer insight are differentiated: customer movements within the retail environment, customer activities at the point of sale and socio-demographic factors in terms of age and gender.
For tracking customer movements within a retail environment, raw image data from sets of single lateral or ceiling-mounted image sensors are used. By applying algorithms from the field of computer vision to the raw data coordinates and timestamps of customers are extracted. For instance, these datasets are used to extract further information such as speed, changes in direction or proximity to other persons and objects. This information is processed to reveal typical movement behavior as well as persons belonging to each other.
Customer activity recognition is accomplished by utilizing raw image data captured by a combination of stereo cameras and infrared sensors. Therefore, persons and their limbs’ positions are detected by the cameras along with a set of computer vision algorithms. The three-dimensional coordinates of the limbs and the relations between them enables gesture recognition algorithms to detect specified gestures. Sets of gestures are summarized to activities such as taking a product out of a shelf. The approach enables to gather more detailed information on the actions of customers at the point of sale.
For age and gender estimation raw image data from a lateral point of view are used as input data for face, gender and age recognition algorithms. Therefore, only one image of the considered customer is necessary to identify his or her gender and an approximate age interval based on large sets of training data.
The information of these three dimensions is linked in one database containing anonymized persons, their gender and age, their movements at the point of sale as well as their activities at specific positions within the retail environment. This database is used to discover typical customer behavior and therefore customer typologies. On the one hand, existing typologies are applied to cluster customers according to typical behavior patterns being extracted previously. On the other hand, approaches like artificial neuronal networks or hidden Markov models are applied to create own typologies based on revealed patterns. This enables a classification of customers based on a subset of data.
Knowledge derived from these analyzes is used for management information systems. For instance, different behavior of customers during different periods of time, hotspots within the retail environment or typical paths of customers can easily be monitored by retail managers. Besides, this knowledge is used to support sales personnel within the retail environment. For example, systems using this knowledge can alert sales personnel when customers with specific behavior patterns occur or crowded places arise. Eventually, by combining information about the movements, activities and socio-demographic factors valuable customer insights are gained.