When the results for video are discussed in connection with FIGS. As seen in FIG. Information about what type of filterbank was used is sent to the bitstream multiplexor The coefficients themselves are passed to the scale factor module and to the perceptual model module For both audio and video data, the quantized coefficients from Quantizer are noiseless coded and then sent to the bitstream multiplexor Note that only one Huffman table is used in video.
The specific method by which SFB are chosen for marking is not specified; however, for audio, SFB encoded with the null Huffman table H 0 should probably not be marked. For video, zero coefficients should remain zero and not be modified. Hence, the marking set will be dynamic. One embodiment of the method for electronic watermarking in the compressed domain utilizing perceptual coding is illustrated by the flow diagram in FIG. As illustrated in FIG.
Finally, the integrally watermarked encoded source is output from the perceptual coder of FIG. A feedback mechanism similar to the one described by Hartung and Girod can be used to prevent modification of scale factors that would increase the bit rate significantly. It should be noted that if the attacker can identify the frame and SFB containing the mark data, then that data can easily be removed. A possible attack on this method would be to run a perceptual model on the decompressed output. While it is unlikely that the perceptual model could indicate unambiguously every marked location, it seems likely that many could be identified.
An alternate embodiment, applicable only to audio data, is illustrated by the flow diagram in FIG. In this embodiment, the watermark data is represented via two particular characteristics of the bitstream data. The value of the watermark data bit one bit per SFB can be indicated in many ways; for example, if the SFB index is even, the value is 0, otherwise 1. A Huffman table is then selected for encoding SFB that can still encode all the coefficients with the required dynamic range. The watermark data bit is set according to any desired scheme, and the quantized coefficients are derived using the alternate non-usual Huffman table.
That is, sectioning itself can erase the mark data indication. Also, this marking is particularly easy to identify, since an attacker looking at the bitstream can observe that the codebook used to encode the coefficients in the SFB is not the minimum codebook required. However, by a sensible choice of SFB, it is possible to insert mark data in a way that will not be modified by sectioning but rather mimics the action of sectioning and therefore is somewhat less obvious to an attacker. The methods of FIGS.
Another embodiment of the invention, illustrated by the flow diagram in FIG. As in the embodiment of FIG. The watermark data bit s are set before a quantization step. If this change were introduced after quantization, the change in some quantized values would be greater than the perceptual noise floor.
Equivalently, an attacker who modifies the quantized values to eradicate or modify the mark will be introducing energy changes that exceed the noise floor. Because the changes in step-sizes will be small, because not all coefficients will change, and because the attacker will not have access to the uncompressed cleartext source material, the attacker will generally not be able to identify those SFB which are used for marking.
Further, the change in bit rate associated with marking should be small. In this third embodiment method, the value of the watermark bit can be indicated in a variety of ways, e. For both audio and video, the increase in bit count incurred by this method must be monitored. As illustrated in the flow diagram in FIG.
Then, the scale factors are established from perceptual thresholds at step With the watermark bit s set and the scale factors established, the next step is to establish a plurality of scale factor bands, M, in which to locate the set watermark bit s at step Generally watermark sequences are inserted a few bits per frame. The data to be carried by the stream is typically mapped into a marking sequence prior to embedding, where the characteristics of the mapping function depend on the type of attack expected.
Indeed, since there may be a wide range of attacks, the data may be redundantly mapped in different ways in the hope that at least one mapping will survive all attacks. This leads to the issue of recognizing where a marking sequence begins. One approach is to use synchronizing codes. However the attacker may be able to identify these codes, and if the attacker can eliminate or damage the codes, recovery of mark data may not be possible. In the system of the present invention, synchronization is tied to frame boundaries.
The scale factors included at the beginning of the frame are modified by modifying the LSBs so that they represent a sequence which contains one or more synchronization codes. Specifically, when a frame is selected for synchronization insertion, and a scale factor LSB does not match e. Although the synchronization code can be damaged, random flipping of scale factor LSB by an attacker will introduce artifacts. To recover the watermark, a synchronization code is sought and the data is recovered a manner appropriate to the watermarking method. To evaluate the audio watermarking system of FIG.
Watermark synchronization is indicated by the sequence comprising the LSB of the first 44 decoded scale factors in a long block. When the value of the LSB of a scale factor does not match the corresponding bit in the synchronization code then the scale factor is decremented and the spectral coefficients adjusted accordingly, resulting in perceptually irrelevant overcoding of the associated spectral data. The table of FIG. Cost is expressed as increase in bits per frame As can be seen in FIG. The increase in rate was 0. An important issue for any watermarking algorithm is the quality of the reconstructed signal following an attack which erases the watermark.
A naive attack on this marking algorithm has been simulated by zeroing all scale factor LSB. This attack results in unacceptable distortion in the reconstructed audio signal. The baseline system for video compression uses a rudimentary perceptual model.
I frames are generated every half second; all other frames are P frames. Watermark data was inserted into both I and P frames, and the results were taken from an average over two different 10 second sequences. The first 44 macroblocks of a frame are used for synchronization. The next several macroblocks or in the Table, out of of a frame carry mark bits using the embodiment of FIG.
However, a dead-zone is applied to the original Q p to ensure that zero coefficients remain zero. A table showing the results of this simulation is shown in FIG. The corresponding increases in rate were 0. Simulation of a naive attack on this algorithm by zeroing all scale factor LSBs demonstrates that this attack results in a perceptible 1. What has been described is merely illustrative of the application of the principles of the present invention.
Other arrangements, methods, modifications and substitutions by one of ordinary skill in the art are also considered to be within the scope of the present invention, which is not to be limited except by the claims which follow. Year of fee payment : 4. Year of fee payment : 8. Effective date : Year of fee payment : A method and apparatus are described for inserting a watermark in the compressed domain. The watermark inserted does not require a reference. An overall watermarking system incorporating the invention combines cleartext, bitstream, and integrated watermarking.
In a perceptual coder, the data enters a filterbank, where it is processed into multiple separate coefficients. The invention supports three embodiments for inserting a mark into the bitstream imperceptibly.
It is assumed that some set of scale factor bands have been selected, into which mark data will be inserted. In an alternate embodiment, watermark data is represented via two characteristics of the bitstream data. A Huffinan table is selected for encoding the Scale Factor Band receiving watermark data which is not the table that would normally be used. The watermark data bit is set according to any desired scheme, and the quantized coefficients are derived using the alternate Huffinan table.
In another embodiment, watermarking is integrated with quantization. What is claimed is: 1. A perceptual coder for encoding at least one compressed audio or video signal to include hidden data, comprising, in combination:. The coder of claim 1 , further including means for selecting a specific Huffman Table for encoding said at least one compressed signal.
The coder of claim 2 , wherein said selected Huffman Table is not the Huffman Table that would normally be selected for encoding said at least one compressed signal. The coder of claim 1 , wherein said hidden data has a watermarking function. The coder of claim 4 , wherein said hidden data has the form of at least one watermark data bit. The coder of claim 1 , further including:. The coder of claim 6 , wherein each member of said set of multipliers is close to unity. The coder of claim 7 , wherein said compressed signal has at least one associated scale factor and said means for modifying comprises, in combination:.
The coder of claim 6 , wherein said compressed signal has at least one associated scale factor and said means for modifying comprises, in combination:. The coder of claim 1 wherein said means for adding hidden data is responsive to said means for quantizing said at least one compressed signal. A perceptual coder for encoding at least one of compressed audio or video signal to include hidden data, comprising, in combination:. The coder of claim 11 , wherein said hidden data has a watermarking function. The coder of claim 12 , wherein said hidden data is added to at least one non-zero one of said integer values.
The coder of claim 11 , further including:. The coder of claim 14 , wherein said compressed signal has at least one associated scale factor and said means for modifying comprises, in combination:. The coder of claim 14 wherein each member, x, of said set of multipliers is equal to 2 Ni , where N is a positive integer and i represents a set of indices associated with said compressed signal. The coder of claim 16 , wherein said compressed signal has at least one associated scale factor and said means for modifying comprises, in combination:.
A method of perceptually encoding at least one compressed audio or video signals to include hidden data comprising, in combination, the steps of:.
A Watermarking Scheme for High Efficiency Video Coding (HEVC)
The method of claim 18 , further including the step of selecting a specific Huffman Table for encoding said at least one compressed signal. The method of claim 19 , wherein said selected Huffman Table is not the Huffman Table that would normally be selected for encoding said at least one compressed signal. The method of claim 18 , wherein said hidden data has a watermarking function.
The method of claim 21 , wherein said hidden data has the form of at least one watermark data bit. The method of claim 18 , further including:. The method of claim 23 , wherein each member of said set of multipliers is close to unity. The method of claim 24 , wherein said compressed signal has at least one associated scale factor and step of modifying comprises the steps, in combination, of:. The method of claim 23 , wherein said compressed signal has at least one associated scale factor and said step of modifying comprises the steps, in combination, of:.
The method of claim 18 , further including the step of marking said at least one audio or video signal with hidden data before compression. The method of claim 18 , further including the step of bitstream marking said quantized compressed signal with hidden data. The method of claim 28 , further including the step of marking said at least one audio or video signal with hidden data before compression. The method of claim 18 wherein said step of adding hidden data occurs simultaneously with said step of quantizing said at least one compressed signal.
The method of claim 18 wherein said step of adding hidden data occurs after said step of quantizing said at least one compressed signal. A method of perceptually encoding at least one compressed audio or video signal to include hidden data comprising, in combination, the steps of:. The method of claim 32 , wherein said hidden data has a watermarking function. The method of claim 33 , wherein said hidden data is added to at least one non-zero one of said integer values. The method of claim 32 , further including:. The coder of claim 35 , wherein said compressed signal has at least one associated scale factor and said step of modifying comprises, in combination:.
The method of claim 35 , wherein each member, x, of said set of multipliers is equal to 2 Ni , where N is a positive integer and i represents a set of indices associated with said compressed signal.
The method of claim 37 , wherein said compressed signal has at least one associated scale factor and said step of modifying comprises, in combination:. The method of claim 32 , further including the step of marking said at least one audio or video signal with hidden data before compression.
- Take Me Out To The Ballgame.
- Design of Digital Video Coding Systems: A Complete Compressed Domain Approach;
- Design of Digital Video Coding Systems.
The method of claim 32 , further including the step of bitstream marking said quantized compressed signal with hidden data. The method of claim 40 , further including the step of marking said at least one audio or video signal with hidden data before compression. A method of perceptually coding as signal to add watermark data using a codebook characterized by the steps of:. USP true USB1 en. WOA1 en. Device, method and storage medium for superimposing first and second watermarking information on an audio signal based on psychological auditory sense analysis. USB2 en. Optimization methods for the insertion, protection, and detection of digital watermarks in digital data.
Methods and apparatus for embedding and recovering an image for use with video content. System and methods for permitting open access to data objects and for securing data within the data objects. Optimization methods for the insertion, protection, and detection of digital watermarks in digitized data. Methods, systems and devices for packet watermarking and efficient provisioning of bandwidth. Associating metadata with media signals, and searching for media signals using metadata.
Electronic papermarking system, electronic information distribution system using the same and storage medium. System, method, and product for distortion-compensated information embedding using an ensemble of non-intersecting embedding generators. System method, and product for information embedding using an ensemble of non-intersecting embedding generators. System for embedding additional information in video data, and embedding method. Information processor capable of making electronic watermark and communication network capable of accepting connection of the information processor.
Electronic watermark embedding apparatus, the output control unit and a computer-readable storage medium. Information recording method, recording device, record reproducing method and record media. Method for simulcrypting scrambled data to a plurality of conditional access devices. Watermark detector and a digital watermark detection method used for the same. Apparatus and method for embedding information, image processor and processing method, content processor and processing method, monitor and monitoring method, and storage medium.
Perceptual modeling of media signals based on local contrast and directional edges. Modulating one or more parameters of an audio or video perceptual coding system in response to supplemental information. Information processing apparatus and its control method, computer program, and storage medium. Apparatus and Method for controlling the copy and play of a digital audio contents using digital watermarking. Using embedded steganographic identifiers in segmented areas of geographic images and characteristics corresponding to imagery data derived from aerial platforms.
Scalable Video Compression based on Wavelets
Real-time, distributed, transactional, hybrid watermarking method to provide trace-ability and copyright protection of digital content in peer-to-peer networks. Method for time aligning audio signals using characterizations based on auditory events. With a face photo application data creation method and its implementation apparatus and the processing program.
Derivation and quantization of robust non-local characteristics for blind watermarking. Data distribution system, terminal apparatus, distribution center apparatus, highefficiency encoding method, high-efficiency encoding apparatus, encoded data decoding method, encoded data decoding apparatus, data transmission method, data transmission apparatus, sub information attaching method, sub information attaching apparatus, and recording medium. Information recording apparatus, information reproducing apparatus, and an information recording method, information reproducing method, and computer program.
The information recording apparatus, a video signal output device, the method stream output, a program, a recording medium, and data structures. Digital watermarking and fingerprinting including symchronization, layering, version control, and compressed embedding. Method and system for watermarking digital content and for introducing failure points into digital content.
Information processor, information processing method and information processing system. DED1 en. EPA3 en. BRA en. Method and apparatus arranged to embed a watermark in an information signal, information signal with a watermark, recording carrier, method and apparatus for detecting a watermark in an information signal, the computer program, and method of making affordably for loading a computer program.
FRB1 en. Identification mark generating method and identification mark generation program. Considering that HEVC may will be used in a variety of applications in the future, the proposed algorithm has a high potential of utilization in applications involving broadcast and hiding of metadata. Later, during the decoding process, the embedded message can be detected and extracted completely. The experimental results show that the proposed algorithm does not significantly affect the video quality, nor does it escalate the bitrate.
The main goals of HEVC design include increased video resolutions and the exploitation of parallel processing architectures . HEVC is suited for a variety of applications, such as broadcast of high definition HD TV signals over satellite, terrestrial transmission systems and cables, video content acquisition and editing systems, security applications, camcorders, Blue-ray discs, Internet and mobile network video and real-time conversational applications that include video conferencing, video chat, and tele-presence systems .
One of the downside, of the ever-growing nature of Internet and multimedia technologies, is the high risk associated with the ease of manipulation, tampering and illegal copying of the digital contents, especially the multimedia. The security of digital contents, therefore, constitutes a quintessential aspect of copyright protection in today's multimedia related industries.
For this very reason, the integrity, verification and authentication of digital videos form an active research area today . Digital watermarking of HEVC encoded videos may be a difficult task, because the codec eliminates most of the redundancy that the watermarking process may exploit. Casual embedding of a watermark, thus, may escalate the final video file size or otherwise affect the quality of the video; a carefully conceived embedding strategy is thus needed.
Keeping these in view, we intend to propose an HEVC watermarking scheme that would have negligible effect on both the video quality and the final file size. Our strategy is to embed a watermark, during the encoding process, that can be completely extractable during the decoding process. Normally, a watermark may either be embedded in the spatial domain or the frequency domain.
With spatial domain video watermarking, the hidden data may be lost during the quantization step of the underlying video codec. One solution to this problem is to embed the watermark in such a way that it survives the quantization loss. But this may come at the cost of lower imperceptibility. A better solution is to go for the frequency domain and better embed the watermark after the quantization step, i. In our approach, we adopt this later approach and embed the watermark message in the selected non-zero QTCs of all the frames of the video.
The rest of the paper is organized as follows. For a better comprehension of this article, the first part of Section outlines a brief overview of the state of the art HEVC standard. The second part of the same section provides a brief literature review regarding the watermarking techniques proposed for various video coding standards, in vogue.
The proposed watermarking algorithm is outlined in Section 0. Section 0. It is an effort to improve upon the existing tools used in H. Besides, many new coding tools have been introduced in the HEVC; the most important change being its frame partitioning. Figure 1 illustrates a block diagram of the HEVC encoding process. Following are the salient features of HEVC:. The size of luma, L, may refer to 64, 32 or 16 samples.
PU is created when a prediction method is chosen. The PU can be sub divided into 2 rectangular or 4 square partitions, in the inter-prediction. For motion compensation, the PU division may be unidirectional or bi-directional. HEVC uses transform coding of the prediction residual in the similar way as its predecessor H. The residual block is split into smaller square transform blocks TBs.
HEVC also has mode dependent alternative transform. The DST is only applied on luma transform blocks. Transform coefficients in the encoder side are quantized to limit the number of bits. At the start, the quantization level is defined by a quantization parameter QP value that controls the uniform-reconstruction quantization URQ scheme.
To further decrease the bitrate, the QTCs are entropy coded. Only one entropy coding is specified in the HEVC, i. The degradation of frames, which is caused by compression, is restored by applying three kinds of filters namely the sample adaptive offset SAO , de-blocking and adaptive loop filters ALF. To increase the parallel processing capability, HEVC introduces three new features other than the slices, such as tiles, wavefront parallel processing WPP and dependent slices. It is pertinent to note that H.
While being still in its rudimentary phase, efforts regarding the watermarking of HEVC videos are scarce. The literature is, however, replete with algorithms regarding the watermarking of videos based on H. With H. Still, there are many methods that rely on the motion vectors MVs - rather than the DCT coefficients - for embedding in the compressed video domains and are usually classified as MV-based watermarking schemes  ,  — .
Zhang et al. In this scheme a 2D-8 bit watermark message logo is converted into a binary sequence, and then the watermark message is embedded into the middle frequencies, i. In another robust method, by Noorkami et al. This method requires entropy decoding for embedding the watermark. For handling the visual degradation, the method looks into the human visual model. While using a key dependent algorithm, the message is embedded in a selected subset of the coefficients with reasonable visual watermarking capacity. In  , the watermarking involves the nonzero quantized AC residuals in the P-frames.
The authors have shown that the visual quality of video is not compromised even if all the non-zero quantized AC residuals are used to embed the watermark. This scheme may, however, affect the performance of the context adoptive variable length coding CAVLC , which may in turn increase the bit-rate, due to the presence of many non-zero quantized AC residuals with the value of 1; CAVLC encodes the trailing ones T1s separately. The advantage of this technique is that it does not increase the bitrate. The main disadvantage of these schemes is that their payload is very low.
Besides, these are not robust to re-encoding with different parameters. The blind scheme of  embeds the watermark into the syntactic elements of H. The H. In  , a combined watermarking and encryption scheme is presented for H. In this scheme, an end to end commutative security system for video distribution is proposed. The authors have investigated the trade off between robust watermarking, encryption scheme security and transcoding possibilities. The watermark is embedded into the DCT coefficient using the quantization index modulation system.
One such method  embeds the watermark into the DCT coefficients of the compressed video stream, whereas the watermark detection is performed using the uncompressed video. In one blind scheme  , the watermark message is embedded in the bit-stream of MPEG-2 without affecting the bit-rate. In  , the message is embedded into the video by pseudorandomly selecting the macroblocks MBs from every luminance block. This is followed by the use of this mask to weigh the watermark amplitude and then modify the selected middle frequency QTCs to carry the watermark information.
The differential energy watermarking DEW algorithm  is based on the selective discard of high frequency DCT coefficients in the compressed data stream.
3 editions of this work
This real-time method encodes the label bits in the pattern of energy difference between the DCT blocks. In MV based watermarking schemes, the watermark is embedded either directly in the video bitstream  —  ,  or during the video encoding process  ,  , . The watermark is usually extracted from partly decoded video. A method, for H. An adaptive threshold, used to select the required MVs, determines the number of bits to be embedded. In  , the message bits are embedded in the two least significant bit LSBs of the larger component from the MVs of H. The payload of this scheme is very low, however.
The technique of  , for MPEG, hides the copyright information in larger magnitude MVs, especially those with low phase angle change. The scheme is fragile having limited payload. In  , first the luminance component of P frame is divided into low-texture and high-texture area and then MVs are modified according to the texture of the area.
The prediction errors of the matched blocks are calculated again according to the changed MVs. Finally, the new MVs along with new predicted errors are encoded. The modulation is based on the mapping rules between MV resolution and message bits. The water scrambling scheme of  is based on the MPEG compression scheme wherein the MVs are extracted in two ways. It is a stream replacement scheme for video watermarking. All such schemes notwithstanding, references regarding the HEVC watermarking are almost non-existent in the literature, mainly due to its early stages of development.
The proposed algorithm targets mainly the imperceptibility of the cover and it can be employed in applications where robustness is of secondary importance, e. The watermark is embedded in the coefficients whose values agree to a certain predefined threshold.
Low-Power Architectures for Compressed Domain Video Coding Co-Processor
The value of the threshold is selected on the basis of the size of the watermark, in bits. Figure 2 outlines the proposed watermarking scheme for HEVC. To avoid any significant escalation in the compression efficiency, only the non-zero QTCs are being considered for embedding - otherwise, many zero magnitude coefficient may become non-zero in the embedding process, thereby affecting the zero runlengths.