01-31-10 - Relationship Work

The crucial things for making a relationship work are flexibility and understanding. You also have to want to be with the person enough that you are willing to bend on anything else in your life for them. Not that you should have to give up things you love, or they should expect you to, but there has to be an overriding value of them that is greater than any loss, so that in hard moments you can think "okay this sucks but I still want them more than anything else and I'll do what's good for the relationship".

Obviously you have to be flexible and not push things; say they get really offended anytime you say "banana" ; you might think "that's fucking stupid and illogical" but still you should refrain from saying banana, you can't try to reason with them about the unreasonableness of their offense. You have to just respect it and let it go. Say you really like musical theatre, they don't, so don't try to force them to go to musical theatre. And they also have to be flexible and understanding enough to let you go to musical theatre on your own in that scenario.

A crucial aspect is being sympathetic. You have to care about their feelings and sympathize even if you think they are being illogical or unfair or whatever. I find that whole "I hear that you are saying you feel insecure because of the way I looked at that other girl" kind of repeat-back mantra to be really repulsive since it's just a surface act, maybe it helps, I dunno, but the real thing you have to do is much deeper. You have to actually care that you hurt them and not care why, even if you think they are being totally unreasonable, you have to try to see the way they are experiencing the situation and how you can make it better for them.

One of the hardest moments is when you do something that hurts your lover and they lash back at you. In this moment they can say things that are really hurtful, because they know all the things you are most sensitive about, and also because coming from their mouth it stings doubly. The ideal response is to ignore their words (since they didn't really mean them) and hear what they are really trying to say which is something like "you hurt me", and try to stop yourself from responding with anger and instead look at what you did that made them act that way. This can be very hard to do in practice, it requires great levels of self control and confidence and awareness.

Something I always struggle with is the idea of "fairness" (like wah wah it isn't fair). You feel like you are putting in all this work and the other person isn't matching, or you feel like you're compromising more than them, or you're doing so much and they don't appreciate it or whatever. You feel like they're mad at you and they don't have a good reason to be; "wah wah it's not fair, I've been so good". You feel like they should suffer as much as you. Or you know that you could smooth over a fight by compromising but you don't think they've earned the happiness that you can give them. It's important to get past all of these ideas. For one thing, you may well be wrong - it's easy to see the work you are doing, it's not always easy to appreciate how much the other person is doing for you, so your idea that you have been compromising more might just be total hogwash. For another thing, even if you have been doing the lion's share of the work, so what - if you can make the situation better, you should do it, because you should be working with your partner for improved joint happiness. There's no contract that you both have to put in the same amount of work; it's really hard not to hold a grudge about this; in the extreme case maybe your partner puts in hardly any work at all and you do all the comforting and compromising, outside observers might call you a "sucker" but if it is a pathway that leads to your joint happiness, then it's the right one.

That said, you don't want to just eat abuse. Sometimes your partner just serves you a shit sandwich and you don't want to eat that over and over. I'm still really not sure how to handle this situation. Sometimes there's an excessive tendency of the mature sensitive male to always find blame in himself. That is, once you reach the awareness that you are working towards joint happiness and that your actions are always part of the puzzle, you start thinking, oh this situation went really bad, what could *I* do better next time, or what did I do that caused this? Certainly that's a good line of thought, and you should pursue it a bit, but you can go too far and just degenerate into martyrdom.


01-30-10 - Cars

.. I know, it's getting old, I promise it's almost over. The web posting anyway, not my agonizing.

Last time I lamented how the beautiful ATTS technology in the Prelude seemed to have died out. Not so! In fact it appears to be almost identical to the Quaife Differential which is in the Ford Focus RS, which is responsible for it being such a fantastic handling FWD car (except that it seems they goofed a bit and didn't get it quite as right, or the Focus's much higher torque is just harder to control). (all that mechanical technicalia about struts and diffs is fascinating but gibberish to me).

Sadly we don't get the Ford Focus RS here. Or the Scirocco GTI. Or the VW Golf R. Or the Civic Type R. Basically car companies see America and think "there's a load of fat gits who only buy rubbish cars, no need to sell any good ones there".

One of the annoying things about buying something like Porsche is that they are severely price-inflated and quality-deflated by the devoted van boys who grew up dreaming of having one and will pay outrageous sums for a shitty product just because of the brand and the image. I on the other hand, just want a car that feels good and actually would really rather not have the brand, but I have to pay for it anyway.

It sucks when you are trying to pick something based on merit and find that the best thing is this brand which causes you to overpay. It happens all the time in clothes; I just want the jeans that look good and fit well, I would hapily buy Sears brand jeans if they were good, but I wind up buying Diesel or Seven for All Mankind or whatever the fucking thing is now just because it's the only one that's actually a decent quality product - but then I'm paying a 50% markup for the damn brand which I don't care about at all. (and in fact I try to hide the label as much as possible because it's embarassing)

I think the new hatchbacks are fucking fabulous. Unless you are buying a super high end sports car, there is absolutely no reason to buy anything but a hatchback. They are way more practical, and you sacrifice very little in weight or handling. The only possible reason to buy a sedan over a hatch these days is for vanity. And it's stupid vanity, it's trying to look boring and fit in to some old fashioned idea of what looks good which makes you totally uncool. I don't love the long hatches like the BMW 1 hatch or the Mini, a key part of the advantage is having a tiny wheel base, which makes them so nifty and fun to drive and easily parkable.

I was thinking how much I hate all the new turbo engines for their lags, and it occured to me that of course the obvious thing to do would be to have some kind of supercharger that took care of your boost before the exhaust pressure was sufficient, and then you switch to turbos after that. Of course this has been done, see for example the VW Twincharger ; the newer solution is Variable Vane Turbos ; BTW these come from the stonkeringly good Autozine Technical School ; my god that web site is good.

Today I was parallel parking my shitty rental car and of course bumping it against the cars on each side and thinking to myself how not fun that would be in a fancy car. I would much rather park a tiny thing like a Golf GTI and hell, I'd like a rubber ring all around the car like a bumper car. So anyway, for random amusement, I present some parking-related youtubes : how to repo parallel parked cars , french car bumping , FWD torque steer parallel parking


01-29-10 - Decisions

I've become so complex in my decision making that I'm getting paralyzed. This has been happening steadily over the years, but I think it's reached a critical point where I can no longer make decisions at all, so I wind up just giving up and flipping coins at some point. I feel a bit like Jimmy Carter or Obama, crippled by intelligence which makes me just wishy-washy and plodding in my contemplation. Sometimes you just have to George Bush it and make a snap decision that fucks over your whole country for many years.

Some of the things I consider in decisions are of course trade-offs and real long terms pros & cons of various options. You have to weight by how often you use things and how much of a delta there is from the other choices per use. Then you consider opportunity cost - the value of other things you could do if you make alternative decisions. You have to also consider your various life plans for your future and how your decision is rated in those different scenarios, and then weight them by their probability. You have to consider the transaction cost and how hard is to get out of if you are wrong.

Then you also have to be aware of your own emotions and prejudices; you are after all human, you can't remove irrational thoughts, but you can try to identify them and compensate for them; but be careful not to over-compensate (eg. don't exclude Bill Hayden from your search for the mole just because he's sleeping with your wife).

Obviously you aren't just rating the cold numerical factors; some things may be bad bargains but give you a good feeling; that should be assigned some value. You have to consider the probability of problems (such as ebay scams or car mechanical troubles) and give them a cost rating and weight by their probability (and obviously the cost should involving a valuation for your frustration factor and mental anguish and time cost).

Most of all, you also need to evaluate the time cost of the decision making process itself; for example people who value-hunt for cans of greenbeans are retarded, just buy the one in front of you, but with larger things it's more graded and complex.

Finally and perhaps most complex at all, you must also apply these decision processes to the meta-decision of deciding what goes in your decision criteria. You have to consider each of the decision factors and decide if the time spent considering it is worth the benefit to the quality of your decision, and then either exclude or include or approximate it.

Speaking of Jimmy Carter and George Bush - it's sort of funny to me how much people admire quick decision makers like The Decider. We don't really care that much if people around us make good decisions, we just want them to fucking pick and hurry up. This goes from the most trivial situation to the most important.

For example, if you're going out to lunch with coworkers and you say "where should we go?" and they're all like "ehh, I dunno, maybe that indian place? or thai, I'm not sure", you think "urg, you're annoying, I won't ask you again".

One thing I've learned about dating is that almost all girls want you to make all the decisions, even the independent strong liberated ones. Of course they want to have veto power and boss you around at will, but when it comes to the mundane daily decisions - where do we go to eat, what should we do tonight, etc. - you better have a decision and you better make it quick, or you are a wishy-washy loser in her eyes. A guy who can take the lead and command a situation is very attractive. Hell, I know it's wonderful to just be able to relax and have someone else lead you around and make all the decisions unless you choose to step in. It doesn't actually matter if the decisions are great, just make them without a bunch of wishy-washing or caveats (though there may be short term complaining about your bad decisions, that should be ignored).

(We also really admire people who are foolishly impulsive & do really stupid harmful things for fun whims or romance. And we love people who are "passionate" aka violent or have big tempers and scream and all that. But those are other rants.)

Sometimes you have to make the meta-decision to simply not even consider a whole class of choices. For example when I was young I used to consider the cost/benefit of food when ordering at restaurants. Most people who even consider price do it all wrong, they just see "that's too expensive" ; instead you should be looking for value, something expensive is worth it if it's special, if it leads to having a nice date, if it teaches you something about food. Anyway, doing that consideration all the time is just exhausting and not worth it once you have any money, so I decided not to make that decision any more.

Another case is when you know that you have irrational biases that are too strong and you simply cannot trust yourself to make a good decision. In that case you have to simply make rules for yourself to eliminate the decision point. Alcoholics of course know this, rather than being reasonable and letting themselves have a little drink once in a while and decide when to stop, they know they can't make that decision and just make the draconian rule to never touch the stuff. But there are lots of cases of this.

For example I know that when my future is uncertain, I am almost always too cautious; that is, I fail to commit to things when I think it's likely they may not last. e.g. I've never bought a house because I feel like I might move at any time. And I tend to never decorate or buy furniture or curtains or whatever because I feel like I might move out of the current rental at any moment, so why invest in making it nice? What I know now is that it's almost always better to just pretend that you are in fact staying in your current place forever and make the investment to make it nice and get all settled in. Worst case you have to move and then you do it all over again. That's not really that bad, the alternative of constantly living like you're about to move out is much worse.


01-28-10 - Mazdaspeed 3 test drive

Well I went and drove a 3 after all. Just some quick notes since it's very similar to the GTI and WRX from yesterday :

The handling and steering feel of the 3 is very good, better than either the GTI or the WRX. It's a little stiffer than either, close to the GTI, but more raw, less damped. In terms of comfort and noise it's between the two - not as muffled as the GTI but not as rattly as the WRX.

It is a turbo, but again it's sort of between the two - you don't really feel a big kick the way you do in the WRX, but it's definitely a big delta, it sort of whooshes in. I really don't like that nonlinear power, I had another near-incident as I pulled out of a parking lot and gassed it hard, it's pulling out slowly then suddenly the turbo kicks and I start flying and have to compensate steer. Bleck. But generally the power is great, it feels fast, and once the turbo is engaged the throttle response is good. The clutch and brakes are a little vague, but not horrible. The engine feels a lot better than the WRX even though it's a bit slower because the power is much more predictable and immediate.

Cargo space in the hatch back is very good; the rear seats are too small, near useless, but they fold down completely flat which is good. It's about the same size as the WRX overall, though the WRX has better rear seats.

However there is a big problem - it's got bucket seeds with big side bolsters. If you happen to be a wide person like me, this is fucking excruciatingly uncomfortable, as the bolsters don't fit around you, but rather dig directly into your ribs and kidneys. I fucking despise this trend to put race seats in cars that don't really need them. The standard seats in all cars should be flat, eg. not race seats, and not even buckets at all. Then if you actually want race seats you can get them aftermarket, because you need to buy ones that fit you perfectly; they need to fit you like clothing, there's no one size fits all. It's fucking retarded and it kind of blows my mind. Why don't I get a choice of seats in cars? Especially in luxury cars, there should be a variety of sizes & shapes of seats.

The cabin controls in the 3 are some of the best I've seen in any car yet. They're just plain old buttons that work fine, they are intuitive and well positioned, no computer mumbo jumo or screeny shit or weird non-standard ways of doing things. (on the other hand, they have a fucking awful locking key ignition thing which is pointless and annoying).

Aside from the seats, the 3 is my favorite of these small practical cars. It's also got great reliability marks, near Honda levels (the Subaru is also pretty good and the GTI is mediocre).

BTW it's kind of stonkering how fast these cheap cars are. The 3 has supposedly done a 0-60 in 5.3 sec , the new WRX can do it in 4.8 sec ; those numbers are like super sports cars from 10 years ago.

Head to head comparison of the "hot hatches" (not really the proper hot hatches, sadly) :

Engine : Mazdaspeed 3 , no contest. The WRX is faster, but the power is so nonlinear. The 3 does have a bit of annoying turbo wimpyness in the low revs, but kicks in quick and then is nice and linear in the high revs and has plenty of punch. The GTI is a bit slow, but that's not really the problem, it's the low-passed laggy throttle that sinks it.

Cargo room : The 3 and WRX tie , both very good. The GTI is not bad for how small the car is.

Apparent build quality : GTI wins for nice feel, the 3 is a close second, WRX is cheap and plasticky, though much improved.

Actual build quality : Mazda wins, then Subaru, then GTI last; the GTI has a reputation for being very bad, but early reports claim they've finally sorted that - we'll see.

Seats : GTI by a big margin, nice design, comfortable, adjustable lumbar support. The WRX seats are okay, the 3 seats are absurdly stupid if you're not just the perfect size body for them.

Brakes : I think I give the GTI the nod on this for nice solid feel, the WRX comes in second and the 3 last with it's vague unreassuring pedal.

Quick cornering : the WRX has to win this as it's the only one you can actually toss around a corner without the wheels skittering or veering out into understeer.

Steering feel : really all three are quite good here, the 3 is maybe the most sensitive and connected to the road, the GTI feels very solid and nifty but a bit too damped.

Ride comfort : GTI by a mile, amazingly quiet and comfortable for a small car. Both the WRX and 3 are very noisey and bumpy by comparison.

Transmission : the GTI wins, the shifter is really nice, short throws; I find the clutch to be a bit too long and soft, but it's better than the 3 and way better than the WRX whose shifter is way too long and vibratey.

There's nothing seriously wrong with any of these cars, they are all excellent values and very practical and good to drive. I recommend one of them over any Audi or Infiniti or Nissan or Toyota or etc.

01-28-10 - Fun and Safe Driving

Fun and Safe Driving is pretty hillarious. The guy is a real life "in soviet russia car drives you" charicature, and the incredible non-self-aware low rent web site is gold. He is, however, right the fuck on the money with almost everything he says. He also has a few tips that I never heard before, they are :

1. To get a FWD car up a steep slippery hill - go in reverse. This is like "duh" once you hear it.

2. If you see you're about to run into a deep pot hole - slam on the brakes and then slam on the gas before you hit it. The main thing is to not be in a big braking-induced forward-lurch when you hit it.

But I also encourage you to read around the site for a while. The humor doesn't really come through until you get into it a bit.


01-27-10 - GTI and WRX test drive

So my practical side wanted me to drive some cheap cars. My criterion was : reasonably fun to drive, hatch back with big enough boot to fit a bicycle, small enough to be a nifty city driver (eg. easy to parallel park, easy to dodge shopping carts in parking lots, and it should cost less than $25k. So I decided to check out the GTI and WRX. (you might question the omission of the Mazdaspeed 3 - in most head to heads that I've seen, the GTI beats the 3, also the VW and Subaru dealers are right next door to each other and the Mazda guy is way the fuck somewhere else, so he got left out). (also sadly, the wide variety of other "hot hatch" options that are available in Europe don't exist over here; for example, oddly, the good Ford Focus RS does not exist here, even though the Mazda 3 is actually a Focus, and the Volvo C30 is also a Focus !).

The GTI : right away the fit & finish is very impressive. The interior feels a lot like a BMW, everything clicks and fits perfectly, it feels solid; the steering wheel is just like a modern BMW wheel (thick and mushy) which IMO is not a great thing but people seem to love that. Headroom is plenty for a 6'2" man, the seats fit pretty well, and has a nice lumbar adjustment even in the manual seat. I like the feel of the cloth seats and the option of cloth is a big plus. The rear cargo room is not awesome, but you could fit a bike with front wheel off. One annoyance in the GTI interior is the fucking touch screen radio which is not optional; god damn computers in cars are such rubbish, give me regular buttons you morons.

Driving the GTI : whoah holy laggy throttle response batman. Again this is just like modern BMW's, but in a bad way. It's literally disconcerting if you glue your eyes to the RPM meter and pulse the gas pedal, it just lags by a solid second. It's like they're running a low-pass delay filter on your gas pedal inputs. I guess this is partly due to the small turbo (again similar to modern BMW's). You don't really ever feel a big turbo kick, what you feel is just a slight mushy lag in the power. Once it does come in, there's plenty of power, it feels pretty light and fast (even though it's not actually light at all), just because it's nice and small. It feels nifty; visibility is very good, steering feel is good, not great (though my standards are a bit distorted since I've been driving Porsches). Handling is generally nice though it has somewhat disturbing understeer when you corner fast. There's torque all over the rev range, which really is just a bit annoying to me because it means I don't have to downshift to punch it which removes some of the fun feel of a manual. The clutch is super easy to use, but that makes it numb and soft, it's got tons of travel and it's not clear where the engagement is exactly; again it feels like a lowpass has been run over it; it's sort of like you're just stomping your foot in pudding and after your food it out of the pudding you've shifted. The ride is amazingly smooth for how tight the body is, they've really sorted that well; it doesn't roll much at all through turns, but it also smooths out road bumps better than mushy cars like the Versa for example - again this is very similar to modern BMW's.

Conclusion on the GTI : very polished small semi-fast car, I like it a lot. As a city car with a bit of cargo room and enough power and handling to be fun, it's excellent. In tight traffic and city conditions I think this is the best of any car I've driven the whole time. If I had to just drive around downtown Seattle, I would want this car. For a real sporty car, it's a bit lacking; the understeer is a sad inevitability of FWD, the steering is not quite as raw as I'd like, and worst of all the throttle lag is very disconcerting, overall it just feels a tiny bit too softened, I'd like everything tighter by one notch.

(BTW on FWD and understeer : the Honda Prelude ATTS system had this completely sorted back in '98 ; they spin the front wheels independently, so in a turn the outside wheel spins faster, and you can actually accelerate through turns to turn *harder*, unlike normal FWD ; you can always apply the right amount of torque to get perfectly neutral steering if you can find the right amount of gas for the curve. It was fantastic, and yet nobody else has adopted it, I would love to see it in every FWD car. Whoah check out this awesome ATTS Manual ; see page 18-19 ).

The WRX hatchback is actually much bigger than the GTI when you see them both in person. I hated the WRX hatchback when it first came out (like everyone), but you know, now I actually like it better than the old WRX sedan; the shape is actually sportier looking, and the retarded hood scoop is much smaller, almost tolerable now. It's also quite practical, the rear cargo space is much bigger than the GTI, in fact it's almost as big as the old WRX station wagons. The fit & finish is much worse than the GTI; it's a lot louder and rattly on the road. It feels mushier in corners, with more body roll, but you can gas through corners more aggressively, and actually gas your way out of understeer a bit.

The turbo on the WRX is a totally different beast from the GTI's ; it's much bigger, takes longer to spin up, and then when it does it has way more punch; the WRX still has that old fashioned turbo car feel, at low RPM when you start it up you're like "omg get going, where is the power?" and then it kicks in and you go "oh, there it is!" ; when it does kick, it really does kick (apparently it now has 265 hp and does 0-60 in around 5 sec which is pretty amazing for a car this cheap - hmm they must get that number with a really hot launch). It kicks sooner than in the older WRX, but it's just nothing like the turbos the Germans are making now. That number is faster than a Cayman, but the cayman feels faster to me because it's lower and stiffer; still there's no doubt that when the turbo kicks in the WRX it feels like you just activated the afterburners in your jet and you can't help smiling or perhaps giggling. One nice thing about the WRX turbo is does reward the driver for shifting correctly to keep it in the power band (though that required you to interact with the annoyingly clutch).

The WRX actually feels less "sporty" even though it's a lot faster; it feels heavier even though GTI and WRX are almost the same weight despite the WRX being much bigger (I guess all that extra weight in the GTI is in shocks and sound dampening, and metal interior bits instead of plastic ones). In theory the GTI gets good gas mileage while the WRX does not, but I'm sure both would get terrible mileage in my hands.

The clutch and shifter are just awful; the clutch travels way too far and it's impossible to find where the gear engagement is; the shifter is way too long and rattles all over and vibrates in your hand uncomfortably. It's just rubbish. On a test drive of course I couldn't really push the WRX, but I could feel it had loads of capability, it would be fun to get to know it and try to toss it around. The WRX seats are not ideal (no lumbar and a bit of a weird bucket curve for the spine) but they're definitely tolerable.

Obviously the WRX is AWD ; it's unclear how useful that actually is vs FWD. Also for those familiar with the WRX redesign, apparently the 2008 one was rubish, but now it's good again. I dunno, I didn't drive the 2008 but this one is good, in fact there's no reason to get an STI anymore because the WRX spec is so close to it now and it's way cheaper.


01-26-10 - Retarded Cayman Complaints

1. "It's not a real porsche". First of all, who the fuck cares if it's a "real" porsche, it's a great car. Second, it's a pure driving machine, a bit low on power but very light and razor-sharp focused and not very practical. In that sense it's way closer to the old porsche heritage than a new 911 is.

2. "Your friends will ask why you didn't get a 911." Fuck your friends, they're retarded. Ask them why they got whatever horrible car they got.

3. "The engine is too quiet". WTF, you're a moron, if anything it's too loud. And if you really want that it's pretty trivial to put an exhaust kit on. And Porsche engine do not sound good, I'm not sure why you want to hear more rattle and clatter.

4. "It's too easy to drive, it's not quirky enough, it's too perfect". I guarantee that the people saying this are not hitting apexes and powering out of corners correctly. You do not have the driving skill to claim that a Cayman is "too easy" , and of course real drivers like Tiff would never say nonsense like this.

5. "It was designed without soul to fit a niche below the 911". Mmm I don't think so. There is one little niggle which was not offering an LSD, but other than that I think it's actually the purest driving machine that was designed for joy in the twisties without compromise.

6. "It could've beat the 911 in the Nurburging time with a few mods ; the 0-60 is too slow ; etc. " ; who fucking cares, you're not an elite racer; does it feel good? does it feel fast? who fucking cares if the time around some track is 1:34 or 1:36 ; only juveniles care if their car can beat a Mustang in a drag race or some ranking or whatever nonsense it is juveniles and fat bankers care about.

Valid Cayman complaints :

1. You're awfully low and visibility is only mediocre, making it a pretty bad car for traffic. It's a bit like a shark - since you only see well out the front, you need to be moving forward all the time.

2. Seats don't lean back and cabin is very small, making it a bit claustrophobic for longer trips. Even for just a nice sunday drive, you have nowhere to put your map books and snacks.

3. Engine noise and stiff ride can get annoying. Basically it's a joy when you're throttling it and not when you're just sitting in it for a long time.

If I was going to modify the Cayman, it wouldn't be more power or an LSD. I would move the engine back a tiny bit to make the cabin bigger; I would lift the roof an inch or so, shrink the rear pillars and put more glass all around. Hmmm... I think I know a car like that....

Here are some nice side-by-sides of the Cayman and 911 ; some really good pictures in these : Motivemag , evo , autoblog

It's annoying that people buy cars for all the wrong reasons. So many Porsche buyers are doing it for looks or status. The worst are people who buy fast cars because they're the "top of the line" ; these people buy fucking 911 GT3's even though they don't track. It's because of these people that the Cayman didn't get an LSD - because if it was faster than the 911 on some tracks people would complain or buy it instead. In reality the reason to buy the 911 instead of the Cayman is not speed, but because you want a bigger more comfortable car, which is what most of the fat old bankers want.

Another example is the 911 Targa. Personally I can't buy it because it's more expensive and lower performance, and I actually like to drive fast. But for a fat banker - it's fucking fantastic. The glass roof is sublime, it means you're sitting in a big glass bubble which is my dream for all cars! Hell, the Honda Civic and every little commuter car should have a glass roof like the new Targas. I think it would make everyone better drivers because it gives you so much more spatial awareness, and also makes you feel more vulnerable.

Anyway, my practical side is having second thoughts about the whole Porsche venture. If I could find a car with the right spec locally I still might go buy it right now, but I'm losing steam with this fucking annoying search, and I really don't want to buy something on the other side of the country. And I started thinking about practical issues :

1. It is a lot of money. How much exactly? 5 year cost to own of a cheap $20k car is around $40k. 5 year cost of a Cayman is around $80k. That's $8k/year for the cheap car, $16k/year for the Cayman. Assuming about 200 driving days a year, that's $80 a day instead of $40 a day. Is the Cayman really $40 a day more fun than something like a GTI or Mazdaspeed 3 ? I dunno. I could either drive the Cayman home, or I could drive the GTI and then buy a Rainier Knizia board game. Or I could drive the GTI and buy cocktails and viagra. Or I could drive the GTI and then pay a hobo to piss on a Mercedes. Or drive the GTI and give $40 to charity. Naah. It's hard to compare the fun factor of the various choices.

(aside : it's one of the classic false economies to buy a shitty old car, or even to keep yours limping along; the actual cost per year of some shitty old car is often greater than buying and running a new cheap fuel efficient car, even though the shitty old car costs $0 and the new one might be $20k)

2. It would feel pretty ridiculous taking the 911 to Trader Joe's, or parallel parking it in the city. And I would have a heart attack every time it got dinged. I kind of like having shitty cars I don't care about too much, it removes a lot of stress. I'd like to be able to hop in and pop out to a bad neighborhood and park it anywhere and not feel scared for the car.

3. It would be nice to have something I can toss my bike in, or something I can drive to go hiking and not care about the fact that I'm getting it all dirty and banging it around in a bit of gravel, or take camping. I guess this is really the same issue as #2 which is in fact just my own self-consciousness of having a fancy car, and not an actual disadvantage.

4. The 911 actually feels pretty shitty at low speed; it feels heavy and cumbersome; it doesn't open up until 30 mph or so. In traffic gridlock maneuvers (swooping in and out of lanes) I think it would not feel good. In those situations you really want a go-cart like a Lotus, or a little hot hatch kind of thing. I dunno, maybe I would get used to it, but if you're going too slow to get the back unstuck a bit, the steering feels very sluggish.

5. Seattle is a pretty shitty place to go cruising. In California I used to go out driving all the time on the wonderful country roads, just looking for new places to see. There's really not much of that at all around here. People often think of Seattle as having lots of "nature" or "wilderness" around, but it's actually one of the most confining non-wild places I've ever lived. If you look at maps it's pretty obvious what the problem is - we're stuck on a really small patch of land here between the Sound and the mountains. Almost the entire area is now suburbanized. You go from Seattle through suburbs, and then you hit the mountains. The problem is once you hit the mountains - there are no roads. You only have the 90 and the 2, which are not terribly fun. And there's really not much wilderness either unless you get on the dirt logging roads and get back into the mountains a bit. One of my favorite things in the world is crusing around on country roads, it's the whole reason I'm thinking about getting a nice car, because I just love that feeling of swooping around a curve with scenery all around, but fuck me if there is anywhere to go around here, it fucking sucks.

6. The vast majority of my driving is just shitty commuting, not nice cruising. Mmm I dunno if this is actually an argument against. There's a huge difference between a plodding neutered commute in a boring car which makes you feel like masticated oatmeal, and a ripping commute where you slash and swerve and shit all over the plebians with your chariot of gold and feel like the master of your domain.


01-24-10 - svchost

Fucking Windows and the way it runs services is the bane of my computing life. If it wasn't for that, XP would be near perfect. Every time I have a computer problem these days it's because of some fucking mystery shit happening in svchost. And I have no easy way to track down WTF is happening or block it. I cannot for the life of me figure out a good reason why they did that instead of just making each service its own process, which I could then diagnose, block, verify was legitimate, etc.

Periodically I'll get a crash from svchost, or suddenly my disk starts churning, or svchost is suddenly taking 90% of CPU. If it happens repeatedly I can usually figure out the culprit, but there's no reason it should happen at all through this mystery fucking obfuscation/anonymization host.

01-24-10 - Lessons

Something I'm only recently learning to do is to let people make mistakes and not try to teach them a lesson. The next step of maturity which I haven't quite reached yet is to help them through their mistake without begrudging the inconvenience.

I'm mainly talking about little things; like say you're going out biking with your friend and you pump your tires and grease your chain before; you offer the pump and grease to him but he says "nah I'm fine". You take off riding, and lo 10 minutes later he's having problems with his chain and tire. The immature lesson-teacher would use this as a chance to say "I tried to tell you to prepare before the ride, see what happened!". Even if you don't do that, the immature grudge-holder would be all pissy and unhelpful about taking care of his bike. The mature response is just to forget it and be nice and say "hey I know a bike shop right over here, let's stop and get you sorted". The immature person would be all tight-faced and impatient during the stop.

Maturity comes in realizing that even though this person has fucked up and inconvenienced you, now their problem is also your problem, and you're going to still be with them for a while, so you should just get the problem taken care of and be nice so you can get on with your lives happily.

The immature side of me wants to see people pay for their mistakes. I tend to put in a lot of effort considering future risks and taking the proper protections against them. I see other people just rush into things without the proper consideration, and then they get fucked. I could them help them out, but then they are getting the advantage of not having to do the consideration work and they still get bailed out, it's not fair. It's sort of like paying for someone else's insurance. But maturity is knowing that when you do the best for yourself, you might help others much more.

A crucial factor is freeing yourself from seeing life as a competition, and not needing to prove to the world that your way is the best way. You aren't in a contest with the other person about whether taking the time to prepare was a good choice or not. He will likely make comments like "god you always take so long to get ready, greasing your chain and checking your cables, let's fucking go already". You have to just ignore that and be confident in your choice of how to live your own life.

This kind of issue obviously comes up a lot in the workplace. Maybe someone is writing some threading code, and you encourage them to clearly define the channels of communication and how they are synchronized. They're like "meh, I just write code and it seems to work, so I'm not gonna bother with that". Later they have some random hard to repro timing bug. If you're an immature cock you would remind them of their bad decision, or not be very helpful to them fixing their bug. If you're mature you just smile and help them debug their mess.

It's much easier to deal with in a strict boss-subordinate relationship (I'm the boss in this fiction obviously) where you can just tell people "do this" or "you fucked up". But even as the dictator it's wise to just relax and be positive and let people do things their own way sometimes. (and of course not everything that you think is a mistake is actually a mistake). A collegial or peer level relationship at work is much much harder to deal with. When you see a peer making what you think will be a big mistake down the road, you're in a bad spot.

One funny situation where I see this happening is in parent-child relationships. I see so many parents be total cocks to their children, using their children's mistakes as "teachable moments" (every time someone says that I vomit a little and want to punch them in the cock). They're fucking children, they don't need to be worried about preparing for future risks, that's your job, just worry for them and take care of them and don't rub it in their face or groan when they're wrong.


01-22-10 - Friday

I'm spending too much time trying find a damn car, it's stressing me out and wasting massive amounts of time and concentration. I just want to buy something now to get it over with even if it's not perfect. I also need to just bite the dealer's rotting gangrenous cock and pay the $5k ridiculous markup, because the time cost of finding a decently priced car is not worth it. The problem is - I just can't find the car I want for any price.

The other day I drove a 911. Yeah, I think that's the car. I won't do a full review, but basically it has the wonderful feel of like a 370 or Cayman without all the drawbacks - it's got great visibility, tons of glass all around, the cabin feels spacious and comfortable. At low speed it feels sort of bulky and much more clumsy than the Cayman (even though the weight difference is only 300 pounds or so), but at high speed it seems to shed weight and become nimble and agile. I think I want like an '06 4S or so. And there just aren't any.

So I started looking at shit like ebay motors. That's a terrifying scam risk. I found a car that looks great, good price. I call the guy up and ask if I can have it inspected. He says no, I have to agree to the bid on ebay and put down a $1000 non-refundable deposit before I have it inspected. WTFBBQ no way jose. Just for kicks I say okay, let's ignore that problem - assuming we work that out, I would want to pay through escrow.com ; he says no they don't take escrow. Okay then.

He says I should trust him because he has "positive ebay feedback". That's a huge LOL. For one thing he has inflated his rating by obviously buying lots of shit on ebay just to get more ratings; he has like 10 seller ratings and 100 buyer ratings. Yelp and eBay ratings are also now ruined by sellers & shops who police their feedback. If you ever write anything negative, they'll try to give you some freebee to get you to change it. If that fails they go to arbitration on the site and complain and get your account banned.

I don't understand; if I was buying a $100k house, I would have agents, inspections, escrow, blah blah blah, actually way too much dilligence. On the other hand buying a $50k car I'm supposed to just wire the cash sight unseen. I'm also at the point where I would just pay a car buyer agent to take care of this shit for me, and they don't seem to exist at all. I mean there's those useless services that will give you prices for new cars from dealers for you, but not someone who will hunt for the spec you want and go inspect it and all that, eg. like a realtor but for cars.

I mentioned before that I think the best way to get accurate credit ratings for financial instruments would be to make the rating agencies offer insurance on their products - then you can look at the price they set for the insurance and know what they really think the risk is. Similarly, dealers could give you accurate condition reports by pricing insurance ("warranties").

Anyway, it's really fucking frustrating. I'm tempted to buy something completely different that I don't really want just because it's available and I can get this fucking phase of my life over with. Or buy the fucking ebay car from the shady seller and just take the risk. I'm spending my time alternating between talking to car dealers (ugh) and scanning the car listing web sites which are all so god damn broken and painful to use. (I've ranted before about the big problems with these listing sites : the biggest single thing they need is "never show me this again", but it would also be useful to have inline notes, persistent item links, and proper option filtering).

Vice Do's and Don'ts stands out for being actually smart and having a sense of humor.

Dave Moulton has a good post on how Eugene routes traffic well for both cars and bikes. San Francisco does a decent job of setting up parallel streets where one is very car friendly and one is more bike friendly, but they don't do enough to discourage the cars from going through on the bike-priority street. It's a real fucking douchey move to drive on the bike priority street, but the converse is also true. It annoys the hell out of me when bikes use Lake Washington Blvd through the Arboretum here in Seattle. It's a narrow street that's an artery for cars and has no bike lane, and there's a very good bike-priority route that runs parallel to it right over at 26th. Get off the fucking car-priority artery you damn bikes! If you're a driver on a car-priority street, do not go slow and hover behind a biker - pass him, close if need be. And fucking hell go ahead and pull out into oncoming traffic a little bit when you pass, the lane divider line is not some inviolable barrier; learn from the asians!

01-22-10 - Exponential

One of my all time pet-peeves is people who say things are "increased exponentially". Of course the worst use of all is just when people use it to mean "a lot" , eg. they're not even talking about a trend or a response curve, eg. "the 911 Turbo provides an exponential increase in power over the base spec". This abortion of usage is becoming more and more common. But even scientists and mathematicians will use "exponential" to describe a trend that's faster than linear (it's quite common in NYT math/economics/science articles for example).

Today I was reading this blog on software development organizations and my hackles immediately went up when I read :

"The real cost of complexity increases exponentially."

I started to write a snarky post about how he almost certainly meant "geometrically". But then I started thinking about it a bit more. (* correction : by "geometrically" I mean "polynomially").

Maybe software development time actually is exponential in number of features?

If you're trying to write N features and they are all completely independent, then time is O(N), eg. linear.

If each feature can be used with only one other feature, and that interaction is completely custom but independent, then time is O(N^2), eg. geometric.

Already I think there's a bit of a myth we can tackle. A lot of optimistic software devs think that they can get under even this O(N^2) complexity by making the features interact through some generic well defined pathways. eg. rather than specificially code how feature (A) and feature (B) and feature (A+B) work, they try to just write (A) and (B) but make them both aware of their circumstances and handle various cases so that (A+B) works right. The problem is - you didn't really avoid the O(N^2). At the very least, you still had to test (A+B) to make sure it worked, which meant trying all N^2 ways, so your time is still O(N^2). The code might look like it's O(N) because there's only a function for each feature, but within each function is implicit O(N^2) logic !!

What I mean by implicit logic is the blank lines which testing reveals you don't have to write! eg. :

void Feature_A( )


    if ( SelectionMode() )
        // ... custom stuff here to make (A+C) and (A+D) work

    // !!! blank lines here !
    //  this is implicit knowledge that (A+B) and (A+E) don't need custom code


You might argue that this is slightly less than quadratic complexity for the developer, and tester time is cheaper so we don't care if that's quadratic. But in any case it's geometric and above linear.

But is it actually exponential? What if all the features could be enabled or disabled, so the state of your code is a binary string indicating what features are on or off; eg. 1100 = A on, B on, C off, D off. Then there are in fact 2^N feature states, which is in fact exponential.

Another possibility is if the features can only be enabled one by one, but they have lingering effects. You have some shared state, like a data file you're processing, and you can do A then C then B then E on it. In that case the number of sequences is something like N! which is exponential for large N (actually super-exponential)

Let's concretely consider a real video game coding scenario.

You're trying to write N features. You are "sophisticated" so rather than writing N^2 hard-coded interactions, you make each feature interact with a shared world state via C "channels". (in the old Looking Glass speak, these C channels might be a set of standard "properites" on objects and ways to interact with those channels; in the old Oddworld Munch codebase there were C "component" types that could be on objects such as "SoundTrigger "Pressable" etc.). So your initial code writing time something like O(N*C).

But for the N features to really be meaningful, the C is ~= N (roughly proportional). (or at least C ~= log(N) , but certainly C is not constant as N increases - as you add features you always find you need more channels for them to communicate with each other). So just code writing time is something between O(NlogN) and O(N^2).

But your features also affect shared state - e.g. the "world" that the game takes place in, be that physical state, or state variables that can be flipped on other objects. If you have N objects each with K internal states, this creates K^N world states that have to be tested. Even with very small world state, if the features are order dependent, you're back to N! test cases.

If the bug rate was a constant percentage of test cases (eg. 0.1% of test cases produce a bug), then you are back to exponential number of bugs = exponential coder time. But I'm not sure that model of bugs is correct. If the bug rate was a constant percentage of lines of code, then bug rate would only be geometric.


01-21-10 - Confounded Macs

So Natasha plugs the Panasonic DMC-ZS3 into her Mac. iPhoto or some shit pops up and she can get the photos off, but can't get to the videos. So I go to the Panasonic web site for software & support. There's absolutely nothing for the Mac, cuz macs "just work" right? You don't have to install drivers for a mac! Uh, okay. So a little googling reveals that the new version of iMovie has support this camera (genius software plan guys, every time someone releases new hardware you have to rev your major software products!? wtfbbq). Anyhoo, I think we're in business we just have to update iMovie. So we go hit update ...

it wants $79 to update iMovie. Uhhh. In order to get the basic driver support for the camera I bought, I have to pay you for new software. Way to go Mac. You win.


01-19-10 - Tuesday

When I hit the "back" button, it's shouldn't fucking take forever as it reloads the page. It should be *INSTANT*. It should just show me what it already fucking retreived for that page the last time I was there. I know it's fucking caching all that data already, it's not like it's too much data. One of the most annoying things in the world is when you try to go back pages and you run into one that automatically kicks you forward again, so you have to try to hit back-back-back really quick, wtf.

If you're thinking about getting a 370Z or a Cayman - get a 370Z. It costs less than *half* as much ($30k vs. 60-70k) , has more power, has a limitted slip differential in the very cheap sport package with manual transmission and rev match (LSD in the Cayman is new and only available with automatic). You could get two for the same price. It's frustrating cuz the 370 is so obviously a much better deal, but they just fuck up a few tiny things. For one thing, you have to be six feet tall or less, and for another, they don't want you to actually see out of it. This is most clear when you see : Photo of 370Z and Cayman lined up so you can compare. The Cayman's visibility is just barely okay, and you can see the Z's door is higher, the hood is higher, the whole side window is smaller, the roof cuts into the view more steeply, etc.

It's like many modern designers are enamored with the 1930's Ford Coupe . Yeah, that's an interesting looking car, and maybe if you're designing cars for a movie like the Fifth Element you might copy that style. But not for the real world. We think of gangsters like Al Capone having 30's Fords (though they didn't actually) ; it's cool to be impractical; if you do stupid shit like grow your hair long so it blocks your eyes, or wear your pants so low you can barely walk, that makes you "cool". It's sort of like the animals that grow huge unweildly mating displays (peacock feathers) to demonstrate their value to females - if you can drive around without seeing you must be fucking great.

Sadly this style is becoming near ubiquitous. Cadillac CTS coupe design ; another example is the modern redesigns of classic muscle cars which lets you compare and contrast : Dodge Challenger old and new (it's not so obvious there how big the difference is because the old car rides higher, but the old car is almost 1:1 door to window, and the new one is closer to 2:1).

The other bad trend is the upsweep curve ; Mercedes moron designers give us the F700 concept to really illustrate the ugly and retarded upsweep; but real Mercedes have the same problem

Linkage :

YouTube - MonkeyLectric Video Pro bike wheel light display - when I see shit like this I can't help thinking "that would be rad at burning man", which makes me want to punch myself ; they want $2000 for it which seems pretty absurd, computer controlled LED strips can't be that expensive; skip to the last 5 seconds to see how it works; if they did something cool like self-power by the motion of the wheel rather than just use batteries I would be more excited
This guy Nazgee made his own and posted PCB specs ; I guess if you browse around youtube a bit there are shit-tons of these things. Ok, it's not cool anymore.

YouTube - Binary Star - Reality Check
TheBox Browse - BBC torrent site
The Fifth Column - if you need more vitriolic ranting
Salumi backroom lunch - meh looks like shit actually but interesting to see
FFTW Home Page - good FFT sources
Fastening Things to Drywall - best page on this topic
Dark Roasted Blend - old but I forget how good it is
cap to the hill - local lesbian blog, amusing to see another person's experience of the same places
Bra Size Calculator - best one of these
Increase in suicide rate of vets - suicide kills more soldiers than combat does
Reverse Dotty ; I'm not sure how I feel about the music (I guess it was inevitable that the retro movement would move on from synth pop to the art-punk of early Sonic Youth and such) , but anyway ghost riding the ambulance is fucking genius. I rather like their song OMD .


01-17-10 - Nob or Knob -

Is there a difference between Nob and Knob or are they just alternate spellings of the same thing ?

It appears that "knob" is generally considered more correct now, though in old english "nob" was more common. There are many meanings, I'll show with the spelling I prefer :

knob : dial/wheel control
knob : bump or protuberance
knob : small amount, usually of butter
knob : head of the penis
nob  : head of a man (archaic)
nob  : wealthy or upper class person (archaic)

Some people seem to think that either "nob" or "knob" are exclusively correct for the slang meaning of penis (they also disagree on whether it refers to the whole penis or just the head). It's unclear to me where the origin of this slang came from, since "nob" can either mean a person's head (archaic), which would suggest the slang came to refer to head of the penis, or "knob" can mean any protuberance.

Wiktionary seems to think "knob" is a common way to refer to a hill; perhaps this is British, I've never heard it in America.

Some weird uses :

"Nob Hill" - my guess is this common name refers to a hill where the wealthy people lived, not the fact that the hill was a knob, but I could be totally wrong about that. Some people seem to point Nob Hill at nabob but since nabob basically means the same thing as nob I don't see why you would point "Nob" at "nabob" when you could just point it at "nob".

"Hob Nob" - apparently this is completely unrelated if you believe this etymology it came from habbe nabbe

"For his nobs" in Cribbage is a funny one ; it appears to also be completely unrelated, coming from the game noddy which means simpleton, and since the knave of the same suit was important to the game it was referred to as the "knave noddy" or just "noddy" which must have become nob. (unrelated but the story of John Suckling, purported inventor of modern Cribbage, seems pretty fascinating; he was apparently a master gambler and cheater at cards who used his skill to get money beyond his station; he was involved in a plot to spring a prisoner from the tower of london, and received at least one beating at the handle of a nobleman tradgames ; ezinearticles ; wikipedia )

There are some funny uses : Nob Hill Knob Set and Nifty Nob Inc. maker of fine Knobs good job on the consistency, guys.


01-16-10 - Porsche Cayman S Test Drive

Yesterday on the way home from work I stopped and drove a Cayman S. It was definitely the most insane test drive of many somewhat insane test drives. I took it out and gave it some gas through a tight corner on a freeway onramp, and the back end came loose and it started to powerslide oversteer. I've never done that before in my life, but I intuitively steered out of it and merged into traffic smoothly. It was pretty fucking amazing feeling, and I think it says a lot for the composure and ease of handling of the Cayman. When it came unstuck, it was very mild mannered, it didn't go shooting off towards the side of the freeway like a bullet.

One thing I saw from my accidental tail slide was that Porsche really treats the driver like a human being, with respect. The electronic traction control (PSM) was on (in sport mode), but it didn't kick in, because it has very high tolerances before it starts interfering. This is in stark contrast to BMW and most cars these days where if your wheels start to spin at all, they kick in the electronic nanny. Just all around, the whole control set is very simple, very direct - they respect you as a driver and let you have feel and feedback and stiff linear levers.

The brakes and steering and feel for the road are all the tightest, stiffest, most linear, best feeling of any car I've driven (the only one that's really close is the 370Z). It really feels like an extension of you, without any weird laggy responses or numbness to frustrate. There's zero unnecessary electronics or beeps or screens or any of that awful shit, it just drives.

It's much smaller in real life than I thought from seeing them from afar. When you get right up to it or in it, you realize it's very low. You sit low, so your head is around the level of a bumper of a high truck. That's a bit of a disadvantage, especially annoying in traffic when you can't see around people. Also the cargo space is pretty ridiculously small, maybe even less than the 370. Like you would even have trouble fitting a large grocery shopping load, and certainly my old days of doing ridiculous things like going camping in the Prelude would never work in a Cayman. It's also really low and stiff, so the river-fording and dirt-road driving and such that I did in the Prelude would be an even worse idea in a Cayman.

Visibility is actually better than I expected. Side and front visibility are way better than the 370Z or M Roadster. Rear visibility out the back window is pretty terrible, but is okay with the side mirrors. Basically visibility while driving is perfectly fine; backing into parking spots would be the main time it would suck I think.

The engine is *right* behind your head, like 6 inches behind your head. I'm not a big fan of the sound; I think the best car sounds are old american muscle cars with that deep gurgly slow rumble that makes your bowels loose. It's got more of a Formula-1 style high banshee scream which is more annoying than sexy. It's fine when you're thrashing it for fun, but would definitely get old on a long trip.

The seating position is not exactly super comfortable, you're pretty tucked into a specific posture with not a lot of squirming options, but there is more head room for tall people than in the 370Z or Infiniti or M Roadster.

As for power - there's plenty; it's easy to think that 300 HP is not enough in the modern insane horse power wars, but it is significatly lighter than most of the competition (3000 lbs vs. 3200-3500 for the others). The Cayman easily goes faster than I could ever safely go on public roads. The gear ratios on the first few gears are very close, so to get max acceleration you need to shift quickly and smoothly.

I like it a lot, its design philosophy is exactly what I want : small, light, nimble, quick, simple, manual, elegant. A few things niggle for me :

1. After slipping the tail in a very light rain, it occurred to me that any rear wheel drive sports car is a pretty silly proposition in Seattle. There's not really much point to trying to enjoy your drive here. Or to biking here. Or to ever waking up in the morning, or being alive at all. But seriously, this is just a really foolish impractical quest that I'm on.

2. The seats are too small for me, the side bolsters stab me right in the shoulder blades. I guess there are "Adaptive Sport Seats" with adjustable bolster positions, but that makes finding a used car to correct spec much harder. It's pretty stupid that so many cars still make these tight bucket seats with fixed geometry that don't fit larger people, eg. Americans.

3. I would feel like a douchebag pulling up to thrift stores and trader joe's in it. I would feel like a loser every time I failed to parallel park it. People would look at me and wonder if I had a small penis and laugh at me if I stalled it. Cops would pull me over for going 1 mph over the limit.

BTW my god the automotive press are such fucking morons : "The second complaint is the car�s lack of soul. The Cayman is a thoroughly German sports car. In other words, it�s all about the driving, not the car" ; "For me this epitomizes what I both loved and hated the car. Compared to my MGB it lacked soul and quirkiness." ; what !?


01-15-10 - Thailand Advice and Ideas

Bangkok : personally I would skip all the temples and palaces and the whole old city area, which is dead and boring. The most interesting things in Bangkok : 1. Sukhumvit road area for street life; this is where the city is really alive now; 2. rooftop hotel bars; there are lots of good ones and they are open air with hardly any railing, it's an amazing experience; 3. old canal neighborhoods from boat taxis, and the tiny dense alleys/streets of chinatown (during the day).

Andaman Sea : don't bother with Phi Phi. Pretty much all the islands look similar, so pick them based on nice places to be, not the sights/scenery. Having your own yacht here would fucking rule Jesus though; you could go to all the little spots that the ferries don't go. There are so many islands with not much distance between them that hopping around by you own boat would be a delight. I'm told that the scuba out at the Surins and Similans is still good, dunno if that's true. Personally if I was to go again I'm interested in the Ko Phayan / Ko Chang area just as a nice place to hang out; generally I like the feel better on the islands that have a real working native population like Ko Yao, Ko Jum, Ko Sukorn, so they aren't just all tourism. I think maybe the Mergui archipelago in Myanmar is the real good place to go now. It's easy to feel like it's too crowded on the resort islands, but really it's in its infancy still; most islands only have a few little bungalow operations; it's not like Mexico or the Carribean or something where there are just hundreds of mega-resorts with all-inclusive packages and huge buildings and all that.

Street food : there are some things I didn't learn about eating street food until the end that would have been useful. A big one is just that usually the easiest way to order is just to sit down at a table. The street food people will have their cart, and some Thais will stand at the cart and order food to go; but they will have tables set up too, and you can just wave at some and say hi and sit at a table and they will come over and take your order. Ordering at the cart can be hard if there's a crowd. There are two major types of carts : noodle carts and stir fry carts. Stir fry carts are hard to order from because they make a bunch of different dishes, basically real Thai people will walk up and just name different ingredients and the wok lady will cook them all. Noodle carts are very easy to order from; usually they have a few types of noodles, and a few types of meats, all of which will be hanging in the cart for you to see, so you can just point to them (or you could learn a few words so that you can say "pork" and "egg noodle"). There are also pad thai carts that generally only make that one dish so are the easiest of all, and reliably good. The best street food areas have a bunch of tables which are shared with a few carts; in this case you can just sit at a table and order from a few different carts and get a variety meal. A lot of the times the meats are very strange bits of anus and gristle; the best way to get edible meat is from a street barbecue guy. The street barbecue guy rarely takes your order directly, and he's often 50 feet away from the seating area, so you have to point at him from your table.

01-15-10 - Relationships

One of the cognitive mistakes I make in relationships is thinking that deeds are additive to state.

I and people like me often think of the work we put in on a relationship as accumulating, building up a bank of good will that we can then draw from when we make mistakes. We imagine :

State += (deeds).

In reality, it's more like

State = (deeds)


State -= time decay
State :-> (deeds)

Where I just made up the :-> which means "drive to" in the sense of my cubic controller ; a variable is "driven" towards a target, via lerp or PD controller or cubic or whatever.

That is, doing good deeds (buying flowers, giving massages, being patient, going to the ballet) doesn't build anything. It just sets the current level, and you have to keep doing it all the time.

When I'm a fucker, I often think "I've done all these good deeds, I've proved that I love you and will stay with you, you should cut me some slack" ; I imagine that the bad deed is just like "State --" , it just takes off some of the credit that I've accumulated. In reality that's not true, again it's just more like an assignment

State = (fucker)

or really

State :-> (fucker)

In a game theory sense, we can think about how to use this. If you only have limitted budget of good deeds, when should you do them? The most important time to do them is right after you're a fucker. You certainly shouldn't do a bunch of them all at once, because they don't add up credits that you get to keep. If you've done really good, you should just coast on that for a while as the state bleeds down before you do something to replenish it. (actually it's even more extreme than that, because of relativism and baseline recalibration, doing lots of good deeds in a row can doom you, because anything you do after that will seem worse than "the good time").

One thing I struggle with is the sort of Buddhist idea of detachment. The Buddhist idea is that you shouldn't be affected by your environment; that is, your happiness should be inside your own head; you should observe and be involved in your environment, but it shouldn't pierce your inner bubble of self. This has always not made sense to me. If I'm supposed to be unaffected by the negative things people say to me or what I see, then I should also be unaffected by the positive things, right? So I should take no happiness when someone says they love me or when Richard Feynman tells me I'm the best physicist he ever met? That doesn't make any sense to me, what happiness am I supposed to take? And if I take happiness from those things, then logically I must also take sadness from the opposite.

Anyway, you do need this detachment to some extent in relationships. Your lover will inevitably have moments of bad mood or anger at you or insecurity or whatever, and they will act like a real motherfucker to you and say horrible things. You can't be too affected by it when they do that, you have to know they don't really mean it. The problem is when they do that, you don't yet know it's just a mood. The way you find out they're in a state is because they said X to you, so at the time when you first hear X, you don't yet know they're in a state. That means you need to be able to hear them say X and not react too much. You need a detachment that lets you sort of ignore them when they say something bad or weird.

This is a weird thing I haven't quite figured out, but I think I'm getting better.

01-15-10 - Friday

Seattle becomes liveable again around April 1. Until then I'm ticking off days like a prisoner. The rain is mildly annoying, not really a big deal. The constant cloud cover is pretty bad, the day just never gets bright and is just a mild gray. But perhaps the worst part of the winter is just the ridiculously short days. Eight hours of sunlight is not enough. With sunrise at 8 AM and sunset at 4:30, most working people don't get one single moment of free time in the sun.

We get our 12 hour day on Mar 17, which is also a perfectly aligned 7:18 AM sunrise 7:18 PM sunset. Time to hibernate.

Rapha is a ridiculous haute hipster bicycle clothing company. All their stuff is about 2X-3X the price it should be. It's sort of a weird market they're aiming at - their aesthetic is pure hipster, but they feature real road bikes and spandex and actual functional stuff; I guess they're aimed squarely at people like me - aging hipster wannabes with money to burn and a love of function as well as form. I'm ashamed of myself. Anyway - ignore the clothes and look at the photologs in their rides section. They're beautiful, inspiring photos; they make me want to get out in the US and ride. I mean, yeah they are ridiculous hipster photos, with that 70's-style photo effect applied to every fucking shot (or I guess it's the cameras+film they use), but it's nice.

I watched "Funny People" on the plane ; I dunno if it was just because of it being the plane, but I actually thought it was pretty good. Like, not amazing, but also not fucking cutesy and insulting like 40 Year Old Virgin or Little Miss Sunshine or Knocked Up or Juno, all of which are so fucking shmaltsy and predictable and tedious. (for reference, I think the best comedy of recent years was Superbad). I also saw about 5 minutes of "Paper Heart" on the plane ; jebus that was bad. I quite like Adam Sandler when he's in more serious roles (IMO his best was probably Punch Drunk Love) and not doing the shooba-scaba-doo hamming it up routine that made him famous.

I'm enjoying White Hinterland : Icarus quite a bit.


01-14-10 - Thursday

I can't even listen to talk radio when commuting now because it's all about Haiti. Yeah yeah natural disaster in the 3rd world, lots of people die. Is this a rerun?

Dave Moulton writes with a bit of mustard about fixie fuckers . It's a bit disturbing the way the retard fixie crowd creates so much antipathy towards cyclists. The question is - is the hipster bicycle craze as a whole good or bad for cycling? The interesting thing is that there actually two completely independent cycle booms going on right now. There's the "Green cycle commuter" boom. These guys tend to be middle aged, upper-middle-class liberals, they buy practical bikes and kit them with paniers and lights from REI and ride responsibly and put soy milk in their water bottles. This movement is pretty small, but they are politically strong because they are older and organized, they tend to be the people who join Bicycle Coalitions and raise support for better bike lanes in cities and things like that. As much as we might hate them for their oft "holier than thou" attitude, these people are clearly a force for good and give cyclists a very good name (mostly - the exception is the militant anti-car faction of this movement who are mildly damaging). OTOH the hipster/fixie movement is very large, and has gotten lots of kids into cycling. The good thing is that many of them discover they love it and become serious responsible cyclists over time. It's also just a massive benefit to have so many more people riding.

Current/recent TV watching list :

James May's * (Toy Stories, Big Ideas, Moon, etc.)

"Life" (BBC nature show)

Wild China (BBC nature show)

Tinker Tailor Soldier Spy (Alec Guiness in lots of makeup)

Full Tilt Million Dollar Cash Game Season 4 (Andrew Feldman is a huge donk)

The Thick of It Season 3

Wallander Season 2

Peep Show Season 6 (I forgot I was waiting for this)

The League : mmm not bad, in an age bereft of comedy options this is one of the least bad

Modern Family : yuck no, couldn't stand it

Top Gear Season 14 : the food's terrible and the portions are too small

Current/recent reading list :

New Ideas from Dead Economists : terrible so far, will slog through a bit more

Vanity Fair : great mag, good fun, some depth

The Economist : meh, the lack of depth and superior tone makes it frustrating and worthless

The Atlantic : liberal intelligentsia pornography

The Intrepter of Maladies : utter rubbish, terrible writing, Pulitzer my ass

The Spanish Tomb : boring boring dry dry dry

2666 : so pretentious and boring and pointless, read book 1 and couldn't continue

01-14-10 - Bankers to Blame

It's painfully ironic listening to the politicians demanding apologies from the bankers for this financial crisis. Let me clue you all in : the bankers did nothing wrong. This crisis was almost 100% the fault of politicians.

Bankers are *supposed* to make profit. You see all these retarded reporters and politicians who complain "the bankers saw an imbalance in the market and exploited it for their own profit". No shit! That's what they do! Or "the bankers sold products that were obtuse, complicated, and over-valued". Hey, if you can sell junk paper and find some moron that will buy it for millions, then of course you should do that. (one of the examples that people are citing now is the appearance that Goldman created some bogus securitiesb that they knew were very questionable and risky, sold them, and then got insurance from AIG against their failure; people imply that this was "irresponsible" or "immoral" or some shit; of course not, it's fucking brilliant; Goldman saw that AIG was being a fucking moron and took advantage of it, good play my son! The thing that was irresponsible and immoral was bailing out AIG and making the taxpayer buy those bogus securities at full value).

Now, certainly we can complain that the mortgage market was not properly regulated. Brokers like Countrywide should not have been allowed to issue bad mortgages and package them and resell them without keeping any of the debt on their books. The Maes should not have been using government-backed money to subsidize questionable mortgages. Complicated mortgages and ARMs and such never should have been allowed at all. There should have been more of a safety net to warn lenders about taking risky mortgages. The ratings agencies should have been more responsible for the ridiculously incorrect ratings they were giving to packages of mortgages. etc. etc. Nowhere in here did a banker do wrong. Instead this is pretty much 100% the fault of recent laws that liberalized the rules of the mortgage market.

Investment banks and hedge funds should not have been allowed to take so much leverage or get so big that they were a structural risk to the system. Derivatives and other securities should not be traded on markets without transparency and regulation. But this is not a fault of the bankers, it's a fault of Congress the Fed and the SEC for failing to regulate the system, for hiring regulators straight from Goldman, for taking masses of finance industry lobby money, for jumping straight from Treasury jobs to wall street & vice versa. Bankers of course should try to maximize their profit within the system they are given, the job of government is to structure the system such that they can't do so much harm in their quest for profit. It's just absurd to think that they should be altruistic and think about how their actions affect the median; that would be un-American.

The other thing that shocks me is the lack of vigilance of share holders of finance firms. If Goldman wants to take huge risks that might make it go bankrupt, or pay out huge bonuses from its profits - that's really none of our business (ignoring the issue that companies are allowed to become so big that they are a structural risk if they fail). But it *is* the business of shareholders. If I'm a shareholder, I fucking *own* that company, WTF are they doing? When they do well, they take bonuses and I don't get dividends. When they do badly, my stock value falls. WTF !? I should be demanding that those bonuses are paid out as dividends! And I should be demanding that they not take such great risks with my investment. It's another way that they have of taking big risks, profiting when the risk does well, and passing the loss to someone else when it does badly. WTF happened to active share holders !?

Largely investment bankers should be allowed to fuck around all they want, that's what they're good at. We aready mentioned the problem that they were allowed to get too big. The other problem was that ordinary FDIC insured banks were allowed to get into risky investments. Another was that public funds and pensions got into these risky investments, which meant the public was exposed to these risks. That never should have happened. What went wrong with pension investing? Basically, massive corruption. There are already a few suits underway about this, but they will surely never get accountability. The thing that happened is huge public pensions like the California state employee's pension fund had all this money they needed to invest. They didn't want to do it themselves so they'd hire someone like Merrill to do it for them. They're a financial advisor and fund manager, they're supposed to have expertise and help you invest your fund responsibly. Well, surprise surprise, Merrill would tell them that the best place to put the money was in this package of securities that Merrill had created, or in a special investment vehicle that another department of Merrill was running. Merrill takes a fee as the fund manager, they take a fee on the investment vehicle, and they get capital for their investment wing. And lo, those investments turn out to be far more risky than advertised. Pensions and public funds all over the world were suckered into paying excessive fees and investing in places they never should have invested through these mechanisms. (in reality, public pensions need to be in very safe vehicles that just track GDP with a minimum of fees, things like total stock market index funds).

01-14-10 - A small note on Trellis quantization

See reference .

I guess this is obvious, but you do get a pretty nice win from using the true floating point Dct results rather than the quantized Dct results when you do TQ.

I believe the standard practice (what I was doing before anyway) is to do your normal fast Dct + quantization, which takes your integer pixels and makes quantized integer post-Dct output. You then apply TQ on that quantized-and-transformed output, which means instead of sending the true output { 4,3,0,0 } you also consider {4,2,0,0 } and {4,0,0,0 } and so on, measure J(R,D) of each and pick the best.

Okay, but the distortion of changing "1" to 0 is not the same as the distortion of another 1 to 0 , if those were not really the same 1 before quantization.

For example, say you're quantizing with a quantizer = 1.0 for simplicity, and no deadzone, even bucket sizes, so you have quantization buckets :

[ -0.5, 0.5 ] -> 0
[ 0.5 , 1.5 ] -> 1

In that case, when TQ decides to take a quantized "1" and send instead a 0, if the true value was 0.51 , that's not so bad. If the true original value was 1.49 , that's a lot worse.

However, interestingly, if the true original value was 1.49 , then we could send it as a "2". If the value is near a quantization boundary, then the distortion doesn't care whether you kick the value up or down, but the rate might be significantly different, in which case you should make a choice based on J and get a win.

So my Dct now also does the true float -> float transform just for use in the distortion measurement for TQ. It's also useful in this application to make sure your Dct doesn't do any scaling, so that the transform is Unitary, that is, L2 norm is preserved. That way the distortion measure in post-Dct space is the same as the distortion measure in pre-Dct space which means you can use the same lambda for lagrange J decisions.


01-13-10 - Oodle Revisited

I got some emails from a friend recently that made me start thinking about Oodle again. Friend is an indie 360 developer who just shot me a query like (paraphrasing) :

"Hey, I'm loading stuff in my game and only getting like 20 MB/sec , what gives?"

So we started trying to dig into what he was doing exactly - are you opening/reading the files with all the right flags, are you on 4k alignment, are you on a thread so you aren't just stalling out for the seeks, etc. ? In the end I think we figured out that his problem was he was reading too many small files, to which he replied :

"Oh yeah, duh, I've always packed files into bundles and loaded big chunks in previous games, but I just hadn't gotten around to it yet in this and it was bugging me that my level loads were taking so long."

! Ah ha ! To me this is where Oodle comes in as a product. It's not super hard stuff, but it's something that people always put off until the very end, which is kind of a shame because it means their level loads take forever during dev. So for three years while you're making your game you suffer through annoying slow level loads. Instead you buy Oodle on day 1 and your level loads are automagically fast.

I believe in this as a product, in the sense that I think we can make it, and it would be extremely valuable, but I'm not convinced that the game industry is mature enough to buy it. The game industry has always suffered from the mistaken thinking of "I could write that myself, so I won't pay for it". That's retarded. What you should look at is what's the full cost of writing it yourself, including debugging, and perhaps most importantly including the opportunity cost of spending that time on this instead of something else, or the opportunity cost of not having feature X until you've gotten around to writing it.

For example, say you're on a one man dev team and you lay out your coding tasks for your project in order {A,B,C,D,E} . All during dev you are suffering a penalty from not having feature E done. If it would make dev a lot easier to have feature E done up front, you should pay a *lot* of money to get it immediately. Some of the classic mistakes in this vein are things like profile HUDs which people put off until the end, but would provide huge benefit if you had from the beginning; another one people aren't aware of is level save/load and memory card support - you think of that as a minor detail to do at the end, but as soon as you do it level designers are walking around with scenarios saved on memory cards to show to each other and they get a huge productivity boost, so it would have been a nice win to do early (though then you have maintenance pain).

To me the big win of Oodle is :

Client just writes code to load files one by one with plain old fopen/fread/whatever.

Behind the scenes Oodle magically robustly incrementally syncs PC files to consoles, packages files into bundles, makes prefetch lists and prefetches bundles, compresses & decompresses bundles on threads. You have ship quality fast loading from the beginning.

That's a small, simple, great product I think. The problem is to make it something compelling enough to sell we start to add lots of features : high performance paging in/out for seamless worlds, hot reloading of changed content, smooth IO integration with streaming data files like audio/video, DVD layout optimization, texture compressors, etc. which really just wind up clouding the tiny little valuable product at the core.

01-13-10 - Lagrange Rate Control Part 3

Lambda seeking. How do we choose lambda to match a desired rate? I have a few ideas.

One idea is to train up a statistical fit to make good guesses. For any given video (or other data to compress), you can create a relationship between lambda and R that's a pretty simple functional fit. There are a few ways to do this. One way would be to just go ahead and compress the video 16 times with different lambda values, observe R each time, that gives you a bunch of data points, now use those to fit a function to lambda(R) , now you can dial a lambda for any rate. If you're going to be working on the same video a lot, this might not be so bad; in particular in the games or DVD publishing business this might be tolerable.

A slight tweak on that idea is instead of just making a fit per video, instead fit to some characteristics of that video. Sample some kind of moments from the video, like do one pass with simple mocomp and measure the L2 norm of the movec and the L2 norm of the residuals. Those are just two moments. Use those to build up a fit lambda(R,m1,m2). To make this fit, we could run on a variety of sample videos to build a database of some good seeds. Furthermore, whenever a user does a run, it would add a seed to their local database. That way if they run on the same video multiple times, those moments would be matched exactly and the fit would get better. If we don't find a close enough sample point in the database to make a good fit, then we just force a test compression of the video at a given lambda and measure the R. So the end result is that rather than doing a bunch of test compressions to build a model, we just do a quick scan to sample some moments, then take the R parameter and look up the fit and get a guess for lambda. Note that rather than doing this fit per video, you could do it *per frame* which would give you a lot more dense sample data very quickly.

Another way to get a decent guess for lambda is with interpolation search. The procedure would be something like this :

pick good brackets for lambda
    high and low lambdas that you're pretty sure the desired lambda is between, but are as close together as possible
    (requires some kind of moment sampling or heuristic training)

evalue R(lambda) at the two end points and half way between them
    that's 3 compression runs

from 3 points, fit a quadratic R(lambda)

use quadratic to choose 4th point lambda
    (bias it so its not too close to the 3 we already did)

run compression at 4th point
    if that R(lambda) is close enough to target -> done

make line between 4th point and neighbor that straddles target R
use lerp to choose 5th lambda

4 or 5 evaluations of R(lambda)

In many cases with 4 runs your R may be close enough to desired rate and you are just done. The quality of interpolation search depends on how simple/smooth the R(lambda) function is. In general it is a good fit to simple curves, but you can get unlucky in your sample point choices.

Finally, if you want to hit a rate more exactly, you can do so more efficiently by relaxing the global uniform lambda requirement. Say you have some lambda which is pretty close to giving you the right rate and you want to encode to hit that rate more exactly. You let lambda drift a little bit as you code to try to adjust towards hitting the exact bit rate.

I'm not sure exactly what the ideal algorithm is for this, so if you have better ideas let me know. It's a little bit of a nasty black box feedback problem, because I'm trying to hit R by dialing L, and I have this unknown function R(L). One assumption that we will use, which most video coders use in one way or another, is that the video is locally self similar, that is, frame N is usually similar to frame N+1 , so if I observe something about the function R(L) on frame N, it is a good guess that it behaves the same way on frame N+1 ; obviously this is grossly untrue for major cuts, but those are "rare" and we just accept the innaccuracy there (you could also have a panic mode when you see you got it really wrong).

So the idea is that rather than search L around on a given frame, we use previous frames to just make a guess for L and only encode the frame once. If we get it wrong by a little bit, no big deal, we'll make up for it on the next frame. This only works when we are only trying to make small corrections.

Specifically : you did a previous pass at lambda L1 and that gave you total size R1. It also gave you a size for each frame : F1(i). You now want to hit size R2 (which is very close to R1).

At frame i you have already written W2 bits, so you have (R2 - W2) remaining. In pass one you had (R1 - W1) remaining at the same spot. Compute the desired size for the current frame as :

F2 = F1 * (R2 - W2) / (R1 - W1)

To hit frame size F2 , you know if F2 is close to F1 then you should use L = L1. If F2 is a little bigger or smaller either way you should adjust L slightly. To do that, we track a running estimate of dL/dF , call it M for the slope. So we use :

L = L1 + (F2 - F1) * M

We then compress with this L which gives us a frame size F(L). We then compute the actual observed slope :

S = (L - L1) / ( F(L) - F1 )

and then update M using S via IIR or FIR.

There are a lot of kludgy things you'd need to do here, like seed M with a decent guess, don't blend in S updates if you get a weird result ( like F(L) = F1 ), forbid L from making big steps away from L1 even if the estimate really wants to, etc.

Also I'm not sure if dF/dL is really the right variable to estimate ; maybe it would be better to do d(F/F1)/ d(L/L1) , or something. That is, for frames of different characters, what is the most consistent response of frame size to L variation?

01-13-10 - Misc

I'm trying to route my blog feed through FeedBurner so that I can hook it up to Analytics now that Google have finally got off their asses and made that work : (see 1 or 2 ). If all goes well I'll post the results in a few weeks.

My body clock is all fucked up from Thailand; I was waking up at like 4 or 5 AM, so I figured I'd just go with it and try the "early schedule" hypothesis of work days. I'm working 7-3 or so, which lets me just avoid traffic both from and to, and is actually a full length day. Those are the plusses. The minus is that it's really fucking hard to wake up and get moving in the pitch black before sunrise. Also it feels really weird to eat lunch at 10 and dinner at 5. I feel mildly tired and unhappy all the time. In theory, sleeping from 10 to 6 is the same as sleeping from 12 to 8, but it doesn't feel the same to me yet.

RANSAC is useful.

You can make your own creme fraiche apparently !? I'm a little disturbed by leaving dairy at room temperature for 4 days.

Evolution of alphabets animated gif ; awesome.

Bike Snob NYC is funny and informative. Okay, maybe not informative.

Home made Almond Milk is fucking delicious. Who knew !? I'm so turned off by retarded vegans and raw-food morons that I sometimes miss good things just because they love them. Of course almond milk is actually an ancient treat in many cultures which has only been recently sullied by foolish propaganda. It's nice Indian style with a little cinammon/nutmeg/cardamom and perhaps some cashews. The shit in a box is disgusting though.

Dj100proof best of 2009 list is cbloom approved.

The shitty A-Data SDHC card I got claims to be "Class 6" which is supposed to be a minimum speed, but the Lumix DMC-ZS3 shits out every so often and says the card wasn't fast enough. Well, it turns out those classes don't really hold a lot of water because nobody is enforcing it, so shitty fuckers like A-Data just put the label on their card even though their card isn't fast enough (the "150X" or whatever they use in marketting is pure bullshit, a lot like the "X" numbers for CD drives; the "Class" is supposed to be a reliable minimum speed). For real speeds, see : Tom's Hardware test or Rob Galbraith in-camera testing . The main thing to look at is the Tom's Write Speed test at the bottom, and look at the dark gray minimum speed bars. The Silicon Power card tests well but is not widely available. The Lexar Professional (not Platinum II) and ATP Pro cards are ridiculously expensive and small so we ignore those. The legit cards are the San Disk Extreme III (not Ultra II) and the Transcend cards. The Transcend cards are about half the price of SanDisk Extreme III cards, so that's my recommendation at the moment : Transcend 16 GB Class 6 for $42 . I also like the fact that all Transcend cards seem to perform well, unlike other brands where you have to be very careful about what exact variant you buy. Yeah I guess I did a stupid thing just buying something and trusting that it did what it claimed, but my god it's so fucking annoying how much you have to research every time you buy anything. The actual dollar cost of these purchases is dwarfed by the time cost of research and/or returns.


01-12-10 - Lagrange Rate Control Part 2

Okay, so we've talked a bit about lagrange coding decisions. What I haven't mentioned yet is that we're implicitly talking about rate control for video coders. "Rate control" just means deciding how many bits to put in each frame. That's just a coding decision. If the frames were independent (eg. as in Motion JPEG - no mocomp) then we know that lagrange multiplier R/D decisions would in fact be the optimal way to allocate bits to frames. Thus, if we ignore the frame-dependence issue, and if we pretend that all distortions are equal - lagrange rate control should be optimal.

What does lagrange rate control mean for video? It means you pick a single global lambda for the whole video. This lambda tells you how much a bit of rate should be worth in terms of distortion gain. Any bit which doesn't give you at least lambda worth of distortion gain will not be coded. (we assume bits are of monotonically decreasing value - the first bit is the most important, the more bits you send the less value they have). On each frame of video you want to code the frame to maximize J. The biggest single control parameter to do this is the quantizer. The quantizer will largely control the size of the frame. So you dial Q to maximize J on that frame, this sets a frame size. (maximizing J will also affect movec choices, macroblock mode choices, etc).

Frames of different content will wind up getting different Q's and very different sizes. So this is very far from constant bit rate or constant quantizer. What it is is "constant bit value". That is, all frames are of the size where adding one more bit does not help by lambda or more. Harder to code (noisy, fast motion) frames will thus be coded with more error, because it takes more bits to get the same amount of gain in that type of frame. Easy to code (smooth, flat) frames will be coded with much less error. Whether or not this is good perceptually is unclear, but it's how you get optimal D or a given R assuming our D choice is what we want.

Ideally you want to use a single global lambda for your whole video. In practice that might not be possible. Usually the user wants to specificy a certain bit rate, either because they actually need to meet a bit rate maximum (eg. for DVD streaming) , or because they want to specify a maximum total size (eg. for fitting on your game's ship DVD), or simply because that's an intuitive way of specifying "quality" that people are familiar with from MP3 audio and such. So your goal is to hit a rate. To do that with a single global lambda, you would have to try various lambdas, search them up and down, re-encode the whole video each time. You could use binary search (or maybe interpolation search), but this is still a lot of re-encodings of the whole video to try to hit rate. (* more on this later)

Aside : specifying lambda is really how people should encode videos for distribution as downloads via torrents or whatever. When I'm storing a bunch of videos on my hard disk, the limitting factor is my total disk size and the total download time - I don't need to limit how big each individual movie is. What I want is for the bits to go where they help me most. That's exactly what lambda does for you. It makes no sense that I have some 700 MB half hour show that would look just fine in 400 MB , while I have some other 700 MB show that looks like shit and could really use some more bits. Lambda is the right way to allocate hard drive bytes for maximum total viewing quality.

Okay. The funny thing is that I can't find anyone else on the web or in papers talking about lagrange video rate control. It's such an obvious thing that I expected it to be the standard way, but it's not.

What do other people do? The de-facto standard seems to be what x264 and FFMPEG do, which I'll try to roughly outline (though I can't say I get all the details since the only documentation is the very messy source code). Their good mode is two pass, so I'll only talk about that.

The primary thing they do is pick a size for each frame somehow, and then try to hit that size. To hit that frame size, they search QP ( the quantization parameter) a bit. The specifically only search QP in the local neighborhood of the previous QP because they want to limit QP variation between frames (the range of search is a command line parameter - in fact almost everything in this is a command line parameter so I'll stop saying that). When they choose a QP, there's a heuristic formula for H264 which specifies a lambda for lagrange decisions that corresponds to that QP. Note that this lambda is only used for inside-the-frame coding decisions, not for choosing QP or global rate allocation. Also note that the lambda-QP relationship is not necessarily optimal; it's a formula (there are a bunch of H264 papers about making good lambda-QP functional fits and searches). They also do additional funny things like run a blurring pass on QP to smooth out variation; presumably this is a heuristic for perceptual purposes.

So the main issue is how do they pick this frame size to target? So far as I can tell it's a mess of heuristics. For each frame they have a "complexity" measure C. On the first pass C is computed from entropy of the delta or some such thing, raised to the 0.8 power (purely heuristic I believe). The C's are then munged by some fudge factors (that can be set on the command line) - I frame sizes are multiplied by a factor > 1 that makes them bigger, B frame sizes are multipled by a factor < 1 that makes them smaller. Once all the "complexities" are chosen, they are all scaled by (target total size) / (total complexity) to make them add up to the total target size. This sets the desired size for each frame.

Note that this heuristic method has many of the same qualitative properties as full lagrangian allocation - that is, more complex frames will get more bytes than simpler frames, but not *enough* more bytes to give them the same error, so more complex frames will have larger errors than simpler frames. However, quantitatively there's no gaurantee that it's doing the same thing.

So far as I can tell the lagrange method is just better (I'm a little concerned about this because it just seems to vastly obviously better that it disturbs me that not everyone is doing it). Ignoring real world issues we've glossed over, the next big problem is the fact that we have to do this search for lambda, so we'll talk about that next time.

ADDENDUM : x264/ffmpeg rate control references :

ratecontrol.txt - Loren is the main dev but this is very cursory
FFmpeg RateControlContext Struct Reference
FFmpeg libavcodecratecontrol.c Source File
[Ffmpeg-user] changing bitrate on the fly - detailed rate control document, but not written by one of the main devs so beware
x264 Macroblock Tree Ratecontrol testing (committed) - Doom9's Forum - this is about the dependency issue that we haven't discussed

old rants