We address the problem of registration, comparison and averaging of temporal paths on nonlinear Riemannian manifolds such as spheres and shape manifolds. Past methods for registration of trajectories on manifolds, such as those used in temporal alignment of human activity data, used quantities that are not proper distances (they are not even symmetric). Without a proper distance, it is difficult to define average paths (or templates) or setup a classification solution. An important property needed here is that the chosen quantity should be a distance and it should be invariant to identical time-wrappings (or re-parameterizations) of the paths.

Figure 1. Given two paths (in red and blue), we obtain an average without temporal alignment (left, in black) and with temporal alignment (right, in black).

Project Description

The need for warping/alignment while analyzing or averaging trajectories can be motivated with a simple example. Consider two paths, drawn in red and blue, on a unit sphere as shown in Figure 1. Though these two paths have the same structure, i.e. two bumps each, any current measure of similarity that does not account for temporal warping will perform poorly in comparing them. Moreover, if we are looking for some statistical description like an average path, ignoring the temporal warping could lead to structural inconsistencies like the presence of three peaks, shown by the black path on the left. If we solve for the optimal temporal alignment, then such inconsistencies are avoided and the distance between the two paths is correctly found to be small. The black path in the right shows an average path obtained using the method proposed in this paper, which accounts for the time-warping variability. In the problem of activity recognition, ignoring temporal variation also leads to structural inconsistencies, which can result in poor recognition performance. The sequences of shapes shown in the first two rows of Figure 2 correspond to two different instances of the same activity. There is an obvious temporal misalignment between the two sequences. If this variability is ignored, the linear correspondence bet between these two activities is incorrect leading to a large distance. The average sequence after accounting for time-warping (fourth row) looks more natural and is a better representative of the two given sequences than the simple average (third row).

This work is under review for 2012 CVPR. We will add more details later.

Figure 2. Rows 1 and 2: Two instances of the same activity. Row 3: Average sequence without temporal alignment. Row 4: Average sequence after temporal alignment.

Related Publications

J. Su, Z. Zhu, A. Srivastava and F. Huffer. Detection of shapes in 2D point clouds generated from images. ICPR, pp. 2640-2643, August 2010.