If you are given a low res signal L and a known down-sampler D() (in particlar, box down sampling), find an up sampler U() such that :
L = D ( U( L ) )
and U( L ) is as close as possible to the actual high res signal that L was made from (unknown).
I'm also interested in the opposite problem :
If you are given a high res signal H, and a known up-sampler U() (in particular, bilinear filtering), find a down sampler D() such that :
E = ( H - U( D( H ) ) )^2 is minized
This is a much more concrete and tractable problem. In particular in games/3d we know we are forced to use bilinear filtering as our up-sampler. If you use box down-sampling for D() as many people do, that's horrible, because bilinear filtering and box-downsampling are both interpolating and variance reducing. That, they both take noisey signals and force them towards gray. If you know that U() is going to be bilinear filtering, then you should use a D() that compensates for that. It's intuitively obvious that D should be something a bit like a sinc to bring in some neighbors with negative lobes to compensate for the blurring aspect of bilinear upsample, but what exactly I don't know yet.
(note that this is a different problem than making mips - in making mips you are actually going to be viewing the mip at a 1:1 resolution, it will not be upsampled back to the original resolution; you would use this if you were trying to substitute a lower res texture for a higher one).
I haven't tried my hand at solving this yet, maybe it's been done? Much like the previous problem, I'm surprised this isn't something well known and standard, but I haven't found anything on it.
Well, I think the signal-processing solution to the preconditioning problem is rather straightforward. For example, I think it is common for digital filter designs to compensate for the DAC/output filter. I have a book on DSP stuff at home that I can dig up, or you could probably check that wacky text document Ryg posted a month ago.
ReplyDeleteSo if you have some target super-resolution, that means you know how big the tent is for your bilinear. What you want to do is figure out what the filter looks like to deconvolve that tent.
Note a tent is just a box convolved with a box, so in fourier space, it is just a sinc modulated by a sinc -- so a tent has a sinc^2 fourier transform.
So, I think your convolution filter should look like the inverse fourier transform if 1/sinc^2.
Oh, and if you want to search for things yourself, you might want to search for "sample and hold compensation". As you can imagine, normally people are compensating for the case when the output filter is a box, not a tent. You get something like this, for example:
ReplyDeletehttp://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=112588
Yeah, I just cannot follow old signal processing papers like that. I don't know WTF they're talking about.
ReplyDeleteAnyhoo, I just did the brute force lsqr solve and I'll write it about next time I feel like blogging.