11/24/2009

11-24-09 - Feedback

One of the hard things about data compression is that when you first try an idea, chances are it will appear to be a loss. That is, say you have some working compressor like an LZ77 compressor and you try some new idea like using a reduced set of possible offsets instead of all possible offsets. When you first try this, you won't have fully worked out the idea, so your implementation is not that great, you don't yet realize the key to making it profitable, so it will appear to just hurt performance.

If you have faith in the idea and stick to it and work out the fact that you need to context-select the reduced set, and only do it when the prediction set is good, and escape out to context coded literals, etc. etc. all the subtleties, then it becomes a win and a nice technique. But for a long while there you aren't sure. You feel like you might be headed down a dead end, wasting a lot of time refining a technique that will wind up being worse than the original (of course this has happened to many people many times with techniques like LZ78 and DMC and Fractal images, etc.).

This is really the key skill that makes the practice of science an "art", and what distinguishes good practitioners from muddlers - it's the sixth sense for what is a good way to spend your time, what areas to explore are likely to be profitable, and how long to stick with it before you punt. If you punt too soon you might miss a huge discovery, but if you stick with it too long you waste your life chasing dead ends.

Of course this is a common feature of all the arcane arts. Any field where you get linear feedback to your exploits is trivial (*). That is, if you start down the right path and immediately get good feedback that it is the right path - anybody can do that, it's easy. All the difficult fields share the property of nonlinear feedback - including life of course. For example I believe that "dieting" and "working out" are really fucking easy life tasks because they provide very obvious linear feedback - you act and you see results. Harder things are things like choosing to make a big career change, since it will get much worse before it gets better, or choosing to work on a failing relationship.

* = obviously that's not true, there are "simple" fields that give linear feedback that are still quite difficult simply because they are a slog. I think Chess for example is one of them; if you spend more time studying, you get better results, pretty linearly; Chess is still difficult because the sheer mental focus and rigor to put in enough study is exhausting. Poker on the other hand is not simple because of the very large variance; you might adjust your game in a good way and immediately have some very bad results due to bad luck; you have to have intuition and faith to know it was a good adjustment even though the results don't seem to indicate such.

1 comment:

Jon Olick said...

very true and inspirational.

old rants