05-13-11 - Steady State Fallacy

One of the big paradigm shifts in data compression was around the mid 90's, people stopped talking about the steady state.

All the early papers had proofs of "asymptotic optimality" , that given infinite data streams from a static finite model, the compressor would converge to the entropy. This was considered an important thing to prove and it was done for lots of algorithms (LZ7x,PPM,CONTEXT, etc).

But in the real world, that asymptote never arrives. Most data is simply too small, and even huge data sets usually don't have static generators, rather the generating model changes over the course of the data, or perhaps it is a static model but it has internal hidden switching states that are very hard for the compressor to model so in practice you get a better result by treating it as a non-static model. We now know that the performance during the "learning phase" is all that matters, since all data is always in transition (even if the majority of the model becomes stable, the leaves are always sparse - I wrote about this before, generally you want the model to keep growing so that the leaves are always right on the boundary of being too sparse to make a reliable model from).

So modern papers rarely bother to prove aymptotic optimality.

Today I realized that I make the same mistake in decision making in life.

In general when something does not go well, my solution is to just not go back. Say some friend is shitty to me, I just stop seeing them. Some restaurant offers me a horrible table even though I see better ones sitting epty, I just won't go to that restaurant again. Some car mechanic way over-charges me, I just won't go to that mechanic again. My landlord is a nutter, I'll just move out. The idea of this behavior is that by cutting out the bad agent eventually you get into a state with only good agents in your life.

But in reality that steady state never arrives. Unless you lock yourself in a cave, you are always getting into new situations, new agents are injected into your life, and the good agents drop out, so you are always in the transition phase.

Or you know, during unpleasant phases, you think "ok I'll just work a lot right now and life will suck but then it will be better after" or "ok I'm injured it sucks I'll just do a lot of rehab and it will be better after" or "it's gray winter and life will suck but it will be better after". But the after never actually comes. There's always some new reason why now is the "hard time".


Tom said...
This comment has been removed by the author.
Tom said...

I have the same mindset: if I don't like what I have, I look for it somewhere else. Why? It is often better than trying to work with the bad apples. I feel like it is important to let go of those things that drain our energy to look for things that are more enjoyable, even thought that might mean searching forever. I'm optimistic enough to say that most of the time you'll be progressing toward a better good.

So, don't search for that steady state, look for the "better than current" state.

old rants