tag:blogger.com,1999:blog-5246987755651065286.post8919217227431772604..comments2024-02-22T16:15:42.388-08:00Comments on cbloom rants: 04-07-10 - Videocbloomhttp://www.blogger.com/profile/10714564834899413045noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-5246987755651065286.post-92026403598777976432010-04-12T07:36:12.052-07:002010-04-12T07:36:12.052-07:00Yeah, I agree with that mostly.
(also BTW the gra...Yeah, I agree with that mostly.<br /><br />(also BTW the gradient of D term is most interesting if it also includes temporal gradient)<br /><br />One problem is that the improved target function that you want to optimize is itself just a heuristic. Even in the grad D case you get into "what should I tweak lambda2 to be so it looks good". The most extreme case is x264 where the SATD target function is a very strange heuristic that sort of happens to work out well due to the structure of the coder.<br /><br />I also think that the quality goal is just too complex to express and use in a practical way. But in that case even if you can't actually use it in your R/D optimizer you could use it as a verification pass to make sure you actually are optimizing something concrete.cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-83329894390233332162010-04-12T01:02:11.453-07:002010-04-12T01:02:11.453-07:00"Don't squared distortion metrics already..."Don't squared distortion metrics already achieve this, encouraging a relatively level amount of error throughout the image, and without introducing a second parameter to optimize on?"<br />They reduce overall error variance, but they don't care at all about spatial variation. For a squared distortion metric, a block with error 100 is a block with error 100 no matter where it appears in the image; if you include the gradient term, it's suddenly a lot cheaper to place that block in a neighborhood with average error around 90 than it would be to place it in a neighborhood with blocks of average error around 20.<br /><br />I agree that introducing an extra parameter is problematic; in practice you'll want to choose lambda2 as some function of lambda and only optimize for one parameter.<br /><br />However, that's not my main point; the idea is that you can view the whole process of "do RD optimization, then apply some heuristics" as "optimize simplified version of target function to get an initial solution, then iteratively perform local changes to optimize full target function". If you can write a heuristic as term in the target function (even if somewhat unwieldy), you're in good shape: doing so makes it much clearer what the model is, and there might even be a way to optimize it directly. The other way round is just as interesting - any term that is difficult to incorporate in a full optimization round but easy to evaluate "incrementally" (i.e. "does this change reduce overall error"?) translates to a local post-optimization test in the code. But unlike arbitrary heuristics, in this case you know that it's actually improving some well-defined target function. That's much better than just knowing that "this video looks a bit better when I turn this on".ryghttps://www.blogger.com/profile/03031635656201499907noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-87466002677746609582010-04-11T15:06:35.167-07:002010-04-11T15:06:35.167-07:00minimize R + lambda*D + lambda2*||grad(D)|| instea...<i>minimize R + lambda*D + lambda2*||grad(D)|| instead (using the spatial "gradient", in this case distortion differences between neighboring blocks). The higher lambda2 gets, the less attractive isolated blocks with much higher distortion than their surroundings become.</i><br /><br />Don't squared distortion metrics already achieve this, encouraging a relatively level amount of error throughout the image, and without introducing a second parameter to optimize on? (The problem being when we find that sums of absolutes seem to produce better results than squares. Maybe we should be raising to an intermediate power?)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-49837529796659254232010-04-11T10:19:48.547-07:002010-04-11T10:19:48.547-07:00One thing to consider is that not only do the opti...One thing to consider is that not only do the optimization processes only improve locally, the metrics are actually unable to measure anything else.<br /><br />Common distortion metrics for images are basically sums of error measurements over small windows. For all SSD-based metrics, the windows are as small as they can be - single pixels. SSIM uses larger windows with some overlap, but it's still the case that very visible global consistency violations (e.g. an edge being interrupted in a single block) is only detected (and hence only influences measured distortion) in a small number of those windows.<br /><br />Even without a real understanding of the HVS, it's possible to get out of that "local distortion ghetto" simply by aggregating local measures with other means than just plain summation.<br /><br />Case in point: the skip/no-skip decision you mentioned. Instead of using R + lambda*D, minimize R + lambda*D + lambda2*||grad(D)|| instead (using the spatial "gradient", in this case distortion differences between neighboring blocks). The higher lambda2 gets, the less attractive isolated blocks with much higher distortion than their surroundings become. Of course you can also get fancy and allow arbitrary distortion along boundaries (which are also determined during the minimization) - then you need to add a penalty term for the length of the boundary as well (effectively clamping the gradient term at some maximum value per block). There's tons of ways to do this kind of stuff.<br /><br />Adding this kind of term makes global optimization hairy, but it's quite easy to look at a small local change and see whether it improves the modified metric or not. In short, an actual implementation boils down to a couple of rules that "look" heuristic (and in fact might be identical to heuristics you're already using) but actually work towards locally improving a more complete distortion metric.<br /><br />I think this is the way forward - imperfectly minimizing a more complete cost function is a far better theoretical model to work from than perfectly minimizing a simple function and then doing local changes to "make it look good" without any real justification from your model.ryghttps://www.blogger.com/profile/03031635656201499907noreply@blogger.com