First, if you don't know, there are two main things : packages & streams. The packages are AVI, MKV, MP4, MOV ; they put together image data and audio data in some way (the layer that unpacks these is often called a "splitter" or "demuxer"). The packages send their streams to codecs which convert them to some format for display. On Windows this is all supposed to go through DirectShow which is supposed to use the 4CC codes and some priority information to automatically find the right handler for the various streams and packages. In theory.
The first problem you hit on Windows is that AVI packages are handled pretty well, but AVI can't hold H264 video because AVI can't handle the flexible B-frame ordering that H264 can generate. (limitted profiles of H264 can be put in AVI, and there are hacks around this problem, but you're getting into a world of hurt). So you need MKV or MP4 boxes, and those are handled poorly; some apps handle them okay, some don't. Some apps "cheat" and don't trust DirectShow like they're supposed to (the cheating apps often work better).
Things I've installed lately :
MP4Box : MP4 stream boxer/unboxer ; pretty decent app, recommended, but help is poor
YAMB : GUI for MP4Box. Useful to help figure out command lines for MP4Box because the help is bad.
YAMB has bad bugs though and will fail to launch MP4Box, so you have to copy out the command line
and run it yourself
MKVVerify : MKV stream checker. Useful because MKV support is so fucking borked.
MediaInfo : Media info reporter. Questionable usefulness because I don't trust it and don't know where
it's getting it's info for.
Graphedit : DirectShow graph visualizer and tester from MS
GSpot : AVI info tool. Useless.
MSU VMT : Moscow State University Video Quality Measurement Tool. This is pretty neat when it works,
but far too often it fails to get the frames correctly, so you get totally bogus results.
MSU LS Codec : Moscow State University Lossless Codec. Best compressing lossless codec, seems nice
but crashes some tools when you try to use videos compressed with this. Thus useless.
Lagarith Codec : This appears to be the one good working lossless codec. Recommended.
HuffYUV Codec : Videos made with this crash me on read. Jeff says it works great for him. Avoid.
MeGUI : GUI for "mencoder" which can driver AviSynth and x264 ; like all of these big GUIs that try to
run a bunch of other products, this mysteriously fails for me. It seems to set everything up right
and then it launches ten other programs and they fail to hook up in the way MeGUI expected them to.
Garbage.
Handbrake : see MeGUI
FFDShow : hooks up the Linux video decoders (ffmpeg , libavc, etc.) to DirectShow. This thing is
pretty evil and fails to report frame rate and media info sometimes, but is also the only real
choice.
Haali Media Splitter : MKV unboxer, works with FFDShow. Difficult to install correct manually.
Even when installed correctly, does some weird shit with framerate; doesn't seem to report it
correctly through DirectShow. Probably best to get a codec pack like :
K-Lite Codec Pack : works for me but generally is considered malware
Matroska Codec Pack : didn't work for me
CCCP Codec Pack : not tried
MPlayer : Linux media player, now ported to Windows ; very flexible command line control of everything,
alternate audio/video in/out. Highly recommended.
MEncoder : video encode/decode partner to MPlayer. I've had more success running mplayer and x264 manually
than using this. Still I can't complain about MEncoder from the command line.
MPUI : GUI for MPlayer. This is horrific malware. When you install it, it takes over your system without
asking. They do provide some tools for you to change this after the fact, but still should be avoided.
Use Media Player Classic or VLC.
AviSynth : script thing to pipe video to other programs that read AVS scripts. Dear lord.
Basically I've found that all the GUI's are broken, and all the video containers (AVI,MP4,MKV) are broken. The thing I've finally discovered that actually works is using MPlayer and X264 from the command line, and only working with split frames. Trying to work with video containers caused me all kinds of hurt because so many of these apps fail to unbox the containers right and screw up the frame rate or drop frames or other mistakes. Instead now if I want to work on a video I use MPlayer to convert it to raw frames.
mplayer -benchmark -ao null -vo png:z=5 video.avi to dump frames to PNG mplayer -benchmark -ao null -vo yuv4mpeg:file=test.y4m video.avi to dump the video to YUV4MPEG format in "test.y4m" for input to x264 x264.exe --bitrate 10000 --output "out.mp4" test.y4m x264 compress to "out.mp4"Then use mp4box to put the audio back if wanted.
The cool thing about mplayer is that its audio/video decoders are the same ones used to view the video. So you can watch it, and if it plays right in the viewer, then it will extract correctly. I've found lots of videos that I can watch in MPC or VLC, but then fail to load the same way in whatever encoder/decoder when I try to process something.
The sucky thing about this method is you make ginormous temp files on your disk, which also slows things down a lot. But avoiding the fuckups of going through the borked DShow codecs and splitters is worth it.
Most of these tools now are originally Linux tools that are getting moved back to Windows. One very promising development is that many of them have the option to directly load libs for the codecs Linux-style (eg. just load libavc to play video) and avoid DirectShow completely. I haven't really tried that yet but it seems like it's almost possible to work with video just by manually picking a few of these libs and then you avoid the whole Windows borked media layer.
ADDENDUM : one of the difficulties I've seen in a lot of tools is reading the frame rate wrong. This is presumably due to the demuxers not reporting things back totally right. But there are also two related fundamental problems that make everything harder :
1. Most of these formats don't have real/useful headers. (if they do have header info, it's just added as "comment" information). This was done originally because theoretically if your AVI is being broadcast on TV and you change the channel into it, you will just start getting bytes in the middle and never see the header, thus they didn't put good headers on at all.
2. It's almost impossible to really reliably just get the frames out of video. DirectShow doesn't have a reliable call that's just "give me the next frame". Instead you have to ask for "when is the next frame" and then "give me an image at this time". The problem is that the "when" can get fucked up in various ways, and then when you say "give me an image at this time" you can either skip frames or get duplicate frames. (this is what fucks up the MSU VMT tool for me so badly, they are getting the time sampling all wrong quite often).
Even if it's not way off, this still causes subtle bugs because people don't agree exactly on how to represent the frame rate. Some people treat broadcast as exactly = 30000/1001 fps and use rational arithmetic for all timing. Some people use floats for frame rate and use 29.97003 and then wind up with floating point precision problems at high frame numbers. Many of the containers store the frame rate as a number of microseconds between frames, eg. 33367 ; so if they store "33367" in the header, should I use that as my frame time increment exactly, or should I use 33366.666666 ?
I'm guessing that tons of people get duplicate and/or dropped frames because of this and just don't notice it.
Once upon a time I used to use this when everything else failed: http://virtualdub.org/
ReplyDeleteFor added measure of good time, take a look at the hacks in the source code how they got different codecs not to crash, etc.
VirtualDub is really handy to work with AVI files; it has limited functionality, but it's fast, no-nonsense and quite robust. There's little support for other formats though. And of course their main trick is not using DShow or VfW but reading the AVI file manually and interfacing with the codecs directly.
ReplyDeleteI've never had problems with HuffYUV. I've only had crashes when trying to load HuffYUV data encoded through FFMPEG; even in strict mode, something seems to be going wrong there. In any case, by now it's a moot point; LagArith needs more CPU time, but it supports a few important formats directly that HuffYUV doesn't (YV12, most importantly), it has a significantly better compression rate, and it can split work across multiple threads, so its net runtime on a quadcore is typically faster than HuffYUV while producing files 2.5x smaller. No contest there. But last time I checked (half a year ago) it wasn't directly supported in FFMPEG, and mplayer/mencoder tends to crash when using the codec DLL directly.
Other AVI misfeatures: Variable bitrate audio is a gamble - I've never gotten it to work consistently. AVI originally has a 4GB limit, but don't ever expect a "classic AVI"-reading app to handle files bigger than 0x7fffffff bytes. They later designed an extension called OpenDML that meant to solve the problem for good by upping the limit to... 256GB (wait, what?). A lot of AVI-reading apps/libs (most notably, Video for Windows itself) will only be able to deal with the first 2GB chunk out of such a file. Which is a real pity because unlike DShow, VfW is relatively sane API-wise.
MP4 (which is basically MOV) support in most apps sadly isn't much better than MKV. Encoding to video-only MP4 then adding the audio stream via MP4Box or encoding via Quicktime are the two ways I've found of making MP4 files with audio that actually work in all players. I've tried several other external muxers and I've tried directly encoding with audio to MP4 using FFMPEG/mencoder and all those options produce files that won't play back correctly in several players.
MEncoder output support for anything but AVI is work in progress (as per the developers) and very dodgy in my experience. YMMV.
For long videos, I ran into trouble with synchronization issues using mplayer. ffmpeg (from SVN) & x264 proved to be the best combination so far.
ReplyDeleteAlex