The MPEG-4 video standard is nowadays more and more used for video compression, and for applications such as video editing, internet or wireless video communications. However the manipulation of video objects, which is one of the most interesting functionality of MPEG-4, makes many watermarking methods inefficient, especially methods which embed the digital signature in the full spatial domain. This paper presents a new video watermarking technique resisting to MPEG-4 video object manipulation. This so-called scrambling technique allows to adapt any classical spread spectrum watermarking scheme operating in the spatial domain to the MPEG-4 requirements concerning VO manipulation. Thanks to the usage of scrambling and multi-layer embedding, an efficient algorithm for video watermarking is then proposed. Our results show that this algorithm is robust to Video Object manipulations and lossy compression.