The problem of semantic indexing of audio-visual documents is of great interest due to the wide diffusion of large audio-video databases. In this paper, we propose a semantic indexing algorithm based on the controlled Markov chain modelling framework. Controlled Markov chain models are used to describe the temporal evolution of low-level descriptors extracted from the MPEG compressed bit-stream. The proposed algorithm has been conceived for soccer game video sequences, and seems promising based on the simulation results obtained in this preliminary study.