Last update : August 31, 2013

To edit a video, you need to cut & join numerous clips. This process is called smart editing and is particularly difficult if the video is encoded with H264.

H.264/MPEG-4 Part 10 or AVC (Advanced Video Coding) is is a block-oriented motion-compensation-based codec standard developed by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG).

H.264 is used in Blu-ray Discs, cable television services, real-time videoconferencing, mobile devices, … The standard defines 17 sets of capabilities, which are referred to as profiles, targeting specific classes of applications.

The highly compressed video file formats like MPEG-2 or MPEG-4 were exclusively designed for playback or distribution… not for editing. The term “compressing”  is misleading, because these video file formats are made in a similar fashion and achieve their very high compression rates by throwing away information. They rely on a system of deleting data that is unnecessarily repeated in frame after frame of the videos. The data is replaced by a reference to an earlier or later frame.

In MPEG-2 streams there are three types of pictures :

• I-Pictures (Intra-coded picture) : these are easiest to think of as a complete picture and are slightly compressed like a JPEG photo file is compressed.
• P-Pictures (Predicted picture) : these are incomplete pictures and only contain the infomation that has changed since the last I-Picture or the last P-Picture.
• B-Pictures (Bi-predictive picture) : these ones are the most highly compressed because they can use information from previous I- or P-Pictures and forward I- or P-Pictures for reference in playback.

The following picture (courtesy Wikipedia) shows the relations between  I, P and B frames.

The I, P, and B pictures are arranged in groups of pictures (GOP) in a way so that the video file can be played back by a video device or software. There are generally two types of GOP’s : short GOP’s and long GOP’s. The sequence of the transmitted frames is not linear; P-frames are send before the related B-frames.

The following picture shows a typical sequence of a short GOP (I =blue, P = red, B  = yellow). The term VOP (video object plane) is used in relation with the video codecs (IVOP, PVOB, BVOP).

You easily understand that cutting or joining such video sequences without disturbing the sequence of frames or the synchronization with the sound can be very complex.

In the case of  MPEG-4 streams it’s even more difficult.

The granularity of the establishment of prediction types in MPEG-4 is brought down to a lower level called the slice level of the representation. A slice is a spatially distinct region of a picture that is encoded separately from any other region in the same picture. In that standard, instead of I pictures, P pictures, and B pictures, there are I slices, P slices, and B slices. Motion estimation provides for the searching of sub-macro blocks of variable size, from 16×16 down to 4×4 blocks. Motion vectors allow up to quarter pixel accuracy for luminance, and up to 1/8th pixel for chrominance. MPEG-4 carries out intra-prediction for intra coded blocks before the transform, performed on either 4×4 or 16×16 blocks and allowing up to 9 directional modes for direction dependent prediction. Residual data transforms are executed on 4×4 blocks with modified integer discrete cosine transform (DCT) which avoids rounding errors. The employment of an adaptive in-loop filter increases subjective quality of video. The standard provides two alternative and more efficient processes of entropy coding. Context-adaptive variable length coding (CAVLC) utilizes multiple variable length codeword tables for transform coefficient encoding considering spatial neighborhood of the coded block. Context-adaptive binary arithmetic coding (CABAC) in addition provides highly efficient automatic adjustment for underlying probability model of encoded data. Long GOP’S are usual in MPEG-4.

Cutting and joining MPEG-4 videoclips without re-encoding (lossless) to keep a high quality and without creating visual or audial drops at the edges of the movies is very challenging. Only a few software tools are capable to do such a task which is called “smart editing“.

A very simple MPEG-4 editor is Machete from Machetesoft. It’s a try-before buy-program, the current version is 4.0 build 33 released on March 22, 2013. The software is available at regnow for 15,99 euros.

Cutting videoclips or inserting other videoclips with Machete is only possible at the location of key-frames (I-pictures). Unfortunately a typical MPEG-4 videoclip has only very few key-frames (every 5 to 10 seconds).

A wellknown MPEG-4 player and editor is the pro-version of QuickTime from Apple. The current version is 7.7.3, build 1680.64. The selected part of a videoclip can be trimmed with the menu  “Edit > Trim to Selection”. The trimmed videoclip can be saved with the same parameters without re-compression with the menu “File > Export …;  Exporter > MPEG-4 sequence ; options >video and audio format > pass through”. A videoclip can be copied to the clipboard and added to another movie with the same features. I expected a clean export to MPEG-4 without affecting the audio or video streams, but this is not the case.

A very powerful and versatile video post-production tool is AviSynth, created by Ben Rudiak-Gould. It’s not a software you may usually think of programs (.exe and GUI), but it’s a video processing engine that works in the background. AviSynth uses scripts which tell the program what to do and what video to produce.

A Wiki on the main website provides some documentation and user guides about AviSynth. A more comprehensive documentation is available at the website of AnimeMusicVideos.org, a community dedicated to the creation, discussion, and general enjoyment of fan-made anime music videos. The AviSynth tutorial is part of a very useful documentation “Technical Guides to All Things Audio and Video” available on the same website.

The AviSynth syntax to program video scripts is available at the official wiki-website.

A package (AMVapp v3.1) including a lot of accessories and complementary software tools,  described in the technical guides, is available at the AimeMusicVideos website. One of the tools is a text editor specifically designed for making AviSynth, called AvsP. It has been written in Python by qwerpoi. The most recent version is 2.0.2 released on October 27th, 2007. An enhanced version called AvsPmod has been created by Zarxrax, the latest version is 2.5.1 released on June 25, 2013.

The AvsPmod editor shows not only the resolution, framerate, colorspace, frame number, time-code and aspect ratio of the videoclip in the bottom bar, but also the position and color of the videopixel defined by the mouse pointer. To play back the video in real-time, you need an external directshow media player (for example Windows Media Player) which is activated with the AvsPmod preview button (4th button from the left). The VLC-player doesn’t work because it’s not a directshow player.

VirtualDub is a video capture/processing utility for 32-bit and 64-bit Windows platforms, written by Avery Lee and licensed under the GNU General Public License (GPL). The current stable version is v1.9.11. An unofficial VirtualDub support forum is available at the website. A modified version of VirtalDub called VirtualDubMod has been discontinued since 2005.

To play and edit MPEG-4 videos in VirtualDub, specific plugins and filters are required. The most straightforward solution is to combine VirtualDub with AviSynth and AvsPmod.

Avidemux2.5 is a free video editor designed for simple cutting, filtering and encoding tasks. It supports many file types using a variety of codecs. The tool is available for Linux, BSD, Mac OS X and Microsoft Windows under the GNU GPL license. The current version is 2.6.5 released on August 29, 2013. A detailed up-to-date documentation is available on the wiki-website.

The tool shows for each frame the type of picture (I, P or B). To save a selection of a clip in the default copy mode without re-encoding, the  marker A and B must be key-frames (I-pictures). An automatic search for key-frames and for black-frames (pictures without content, often inserted between movies and commercials) is provided. To join videoclips use the menu “File > Append”. The Smart-Copy feature doesn’t work for videos encoded with the H264 codec. For some other codecs you’re asked whether you want to use Smart-Copy or not if you cut your video, and the first frame of a segment is not an I-frame, and you try to save it .

Womble MPEG Video Wizard DVD 5.0 is a commercial MPEG editor with DVD authoring and full MPEG-4 and AC-3 encoder support. The price for a single user personal license is $99. A free trial download is available. The features of this program are smart rendering, no re-encoding, fast HD MPEG editing with frame accuracy, automatic Ad detection and removal, movie conversion to iPod’s and PSP’s, intuitive User Interface (UI) and batch processing.

The current release is 5.0.1.108 from June 2013.

A tutorial “How to Edit Out Commercials? ” is available at the Womble website.
Another commercial video editing tool is SmartCutter from FameRing. The company states that SmartCutter is the world’s first H.264 AVCHD MPEG2 frame accurate cutter without re-encoding! The price is 40$, a free trial is available. Other tools as a video browser and a video framer or bundled versions are also offered. The current version is 1.8.1 released on August 28, 2013.

A tutorial how to edit H.264/AVCHD/MPEG2 videos without re-encoding is available oh the FameRing website. The name FAME stands for Frame Accurate Movie EngineeRing.

The record function of the VLC media player can be used to do a simple cutting of video clips.

My favorite editing tools are now AVS Video Remaker and AVS Video Editor from AVS4YOU, a project of Online Media Technologies Ltd, an english IT high-tech company, founded in 2004 and specialized in developing innovative video and audio solutions for end-users and professional developers. AVS4YOU is a collection of software tools (currently there are 20 tools available) for which you can purchase either an unlimited access license or a one-year access license and use aLL of the tools with that license.

I did a lot of tests with other low-price commercial and shareware video-editors and I experienced serious problems with most of them when loading my MPEG-4/H264 test videos.