05-31-12 - On C++ Atomic Fences Part 2

Last post we talked about what C++0x fences are. Now we'll look at what they are not.

(NOTE : I am talking about what a C++0x fence is guaranteed to be by the standard; at the moment we will not concern ourselves with the fact that at the moment a C++0x fence always actually issues a CPU memory barrier which is somewhat stronger than what C++0x promises; I have no doubt that CPUs and compilers will change in the future to be more aggressive about allowing relaxed memory ordering).

The C++0x fence is not visible to other threads unless they specifically schedule against the fence. That maybe sounds obvious if you are thinking in terms of the C++0x definitions, but it's not true of a real CPU fence.

A very common example in old-school code is to use a CPU barrier for publication. Something like this :

Set up stuff
Barrier
Publish stuff

other threads :

if stuff is published
  then reading stuff should see set up

this does *not* work if "Barrier" is a C++0x fence. In particular we can construct this simple example :

atomic<int> x; // stuff
atomic<int> y; // publication flag

x & y initially 0

thread 0 :

    // set up stuff :
    x($).store(1,mo_relaxed);
    // barrier :
    std::atomic_thread_fence(mo_seq_cst,$);
    // publish :
    y($).store(1,mo_relaxed);

thread 1 :

    // wait for publication :
    rl::backoff bo;
    while ( y($).load(mo_relaxed) == 0 )
      bo.yield($);

    // read it :
    int xx = x($).load(mo_relaxed);
    RL_ASSERT( xx == 1 );

this does not work (xx can be zero). To make it work in C++0x you need something that can synchronize with the fence, for example this would work :

thread 1 :

    // wait for publication :
    rl::backoff bo;
    while ( y($).load(mo_relaxed) == 0 )
      bo.yield($);

    // added -
    std::atomic_thread_fence(mo_acquire,$);

    // read it :
    int xx = x($).load(mo_relaxed);
    RL_ASSERT( xx == 1 );

Why does it work on real CPU's then? (assuming the "relaxed" loads are at least C++ volatile so they go to memory and all that, but not otherwise ordered). On all current real CPU's a memory barrier sends a message to all other CPU's which creates a sync point for all of them (not exactly true, but effectively true). When thread 1 sees that y is non-zero, then the store to y on thread 0 must have happened, which means the barrier must have happened, so x must be set up, and our load of x occurs after the barrier, so it must see the set up value. That is, the barrier forms a sync point on *all* threads, not just the originator, and you don't necessarily need your own fence to tack into that sync, all you need to do is have a way of connecting a "happens before/after" to it. In this case we can say :

thread 1 reads x after
thread 1 loads y with value == 1 after
thread 0 stores y with value == 1 after
thread 0 does a barrier after
thread 0 stores x

(aside : this is of course a terrible way to do publication in the modern world, you should just use a store-release and load-acquire pair to do publish and consume; alternatively if you publish a pointer, then you don't need any synchronization at all, because causality and load-consume takes care of it for you).

Okay. We can see the exact same thing in a more complex example. This is Dmitriy's example Peterson Mutex . Here's my test harness :

struct test7 : rl::test_suite<test7, 2>
{
    // the mutex :
    std::atomic<int>    m_flag[2];
    std::atomic<int>    m_turn;
    // something protected by the mutex :
    std::atomic<int>    m_data;

    void before()
    {
        m_flag[0]($).store(0);
        m_flag[1]($).store(0);
        m_turn($).store(0);
        m_data($).store(0);
    }
    
    void after()
    {
        int d = m_data($).load();
        RL_ASSERT( d == 42 );
    }
    
    void lock(int tidx) 
    {
        int other = tidx^1;
    
        m_flag[tidx]($).store(1,std::mo_relaxed);
        m_turn($).store(other,std::mo_release);

        // ** Barrier here **       
        
        rl::backoff bo;
        for(;;)
        {
            int f = m_flag[other]($).load(std::mo_acquire);
            int t = m_turn($).load(std::mo_relaxed);
            if ( f && t == other )
            {
                bo.yield($);
                continue;
            }
            break;
        }       
        
    }
    
    void unlock(unsigned tidx) 
    {
        m_flag[tidx]($).store(0,std::mo_release);
    }
    
    void thread(unsigned tidx) 
    {
        lock(tidx);

        // do m_data += 7; m_data *= 2;
        int d = m_data($).load(std::mo_relaxed);
        m_data($).store(d+7,std::mo_relaxed);
        d = m_data($).load(std::mo_relaxed);
        m_data($).store(d*2,std::mo_relaxed);
        
        unlock(tidx);
    }
};

This is an example where if "Barrier" is an actual CPU barrier, this code works. But if "Barrier" acts like a C++0x seq_cst fence (and no more), then it doesn't work. (it doesn't work in the sense that it doesn't actually provide mutual exclusion, eg. the code doesn't do what you expect it to).

The failure is the same kind of thing as the first trivial example; all current CPU's have ordering relations that are stronger than C++0x. In particular the bad execution case that's possible in C++0x (when barrier is a seq_cst fence) goes like this :

thread 0 : starts lock
thread 0 : flag[0] = 1
thread 0 : turn = 1

thread 1 : starts lock
thread 1 : flag[1] = 1
thread 1 : turn = 0 ; (overwrites the previous turn=1)
thread 1 : barrier
thread 1 : load flag[0] ; sees 0 - old value (*!)

thread 0 : barrier
thread 0 : load flag[1] ; sees 1
thread 0 : load turn ; sees 0

(*!) is the problem point. For the code to work we must load a 1 there (which thread 0 set already). On a normal CPU you could say :

load flag[0] is after
barrier, which is after
store to turn = 0, which is after
store to turn = 1, which is after
store flag[0] = 1

therefore load flag[0] must see a 1. (I know the store to turn of 0 is after the store of 1 because the later load of turn on thread 0 sees a 0).

So part of the problem is C++0x doesn't count stores in the linear "modification order" of an atomic object. So the easy fix to ensure the "is after" relationship above actually happens is to change the store to turn into an RMW :

    void lock(int tidx) 
    {
        int other = tidx^1;
    
        m_flag[tidx]($).store(1,std::mo_relaxed);

        m_turn($).exchange(other,std::mo_acq_rel); // changed to RMW
        
        rl::backoff bo;
        for(;;)
        {
            int f = m_flag[other]($).load(std::mo_acquire);
            int t = m_turn($).load(std::mo_relaxed);
            if ( f && t == other )
            {
                bo.yield($);
                continue;
            }
            break;
        }       
        
    }

and that works (it's the same thing described here : Peterson's lock with C++0x atomics - Just Software Solutions among other places).

Some links on the topic :

Subtle difference between C++0x MM and other MMs
Subtle difference between C++0x MM and other MMs - Page 3
stdatomic_thread_fence - cppreference.com
Relacy finds fence bugs in spsc
Re questions about memory_order_seq_cst fence 2
Re questions about memory_order_seq_cst fence 1
Implementing Dekker's algorithm with Fences Just Software Solutions - Custom Software Development and Website Development in
C++0x sequentially consistent atomic operations - comp.programming.threads Google Groups 2
C++0x sequentially consistent atomic operations - comp.programming.threads Google Groups 1

05-30-12 - On C++ Atomic Fences

C++0x's atomic_thread_fence is weird. Preshing asked some questions and pointed out some errors in this blog post which has got me to look into it again.

(caveat : I'm no standards-reading afficionado, and I find the C++0x rules very odd, so this is as much me learning out loud as anything).

I'm going to do this post a bit backwards; first some random notes.

1. I never use C++0x fences. In all the lockfree atomic code I've written I've never used them or wanted them. I put the memory ordering on the operations; it's easier to understand and usually more efficient (because doing something like making an exchange be acq_rel is a no-op on most platforms, exchange is already acq_rel, whereas adding another fence requires another hard sync to be issued because compilers are not yet sophisticated enough to merge atomic ordering operations).

The only case I have ever seen where a fence is better than putting the constraint on the operation is for the optimization of doing a relaxed op to detect that you need the ordering. Something like :

if ( atomic_ticket.load( mo_relaxed ) == me )
{
  std::atomic_thread_fence( mo_acquire );  // make previous load act like acquire
  ... do stuff on my ticket ...
}

is faster than :

if ( atomic_ticket.load( mo_acquire ) == me )
{
  ... do stuff on my ticket ...
}

this can be used for example with a ref counting destructor; you don't actually need the "acquire" until the refs go to zero. Which brings us to the next note :

2. The way to understand C++0x fences is to forget everything you know about CPU's, caches, what the CPU memory fence instructions do, any connotation in your head about what "barrier" or "fence" means. The people who are most confused about it are the ones who had some background writing lockfree assembly in the pre-C++0x days, because C++0x fences are really not what you are used to.

What C++0x fences really do is provide more sync points through which other C++0x atomic ops can create "happens before" relationships. You can heuristically think of them as modifying the neighboring ops, not as being an independent operation themselves. In particular :

An "acquire" fence can make a preceding load act like a load_acquire
A "release" fence can make a following store act like a store_release

(an acq_rel fence obviously does both)

A "seq_cst" fence provides an entry in the program total order ("S")
  then preceding loads & following stores can be located relative to that point in the order S
  (eg. either "happens before" or "happens after")

Actually the fences are rather like the old-school way of marking up memory ordering constraints. eg. instead of :

x.load( mo_acquire );  // #LoadLoad follows acquire load

you used to write :

x.load();
#LoadLoad

which is more like the fence method in C++0x :

x.load( mo_relaxed );
fence( mo_acquire ); // #LoadLoad

Errkay so let's get into more specifics.

---

Section 29.8 of C++ doc N3337 :

29.8.2 : A release fence A synchronizes with an acquire fence B if there exist atomic operations X and Y, both operating on some atomic object M, such that A is sequenced before X, X modifies M, Y is sequenced before B, and Y reads the value written by X or a value written by any side effect in the hypothetical release sequence X would head if it were a release operation.

A is a release fence
X is an atomic op on M, X modifies M, after A

Y is an atomic op on M, Y reads the value set at X, before B
B is an acquire fence


is this :

    m_x and m_y initially zero
    
    void thread(unsigned tidx) 
    {
        if ( tidx == 0 )
        {
            m_x($).store(1,std::mo_relaxed);
            std::atomic_thread_fence(std::mo_release,$); // "A"
            m_y($).store(1,std::mo_relaxed); // "X" , m_y is "M"
        }
        else
        {
            while ( m_y($).load(std::mo_relaxed) == 0 )
            {
            }
            // we just read a 1 from m_y // "Y"
            
            std::atomic_thread_fence(std::mo_acquire,$); // "B"
            
            int x = m_x($).load(std::mo_relaxed);
            RL_ASSERT( x == 1 );
        }
    }

Roughly what this says is the "release" ordering of a fence synchronizes with the "acquire" ordering of another fence if there is a shared variable ("M") that connects the two threads, after the release and before the acquire. Or, if you like, the release fence before the store to M makes the store act like a store-release, and the acquire fence after the load of M makes the load of M act like a load-acquire.

(just using a store-release and a load-acquire is the normal way of doing a publish/consume pattern like this)

29.8.3 : A release fence A synchronizes with an atomic operation B that performs an acquire operation on an atomic object M if there exists an atomic operation X such that A is sequenced before X, X modifies M, and B reads the value written by X or a value written by any side effect in the hypothetical release sequence X would head if it were a release operation.

A is a release fence
X is an atomic op on M, X modifes M, after A

A synchronizes with B :

B is an atomic op on M, reads the value written by X


    void thread(unsigned tidx) 
    {
        if ( tidx == 0 )
        {
            m_x($).store(1,std::mo_relaxed);
            std::atomic_thread_fence(std::mo_release,$);        // "A"
            m_y($).store(1,std::mo_relaxed);                    // "X" , m_y is "M"
        }
        else
        {
            while ( m_y($).load(std::mo_relaxed) == 0 )
            {
            }

            int y = m_y($).load(std::mo_acquire);  // "B" synchronizes with "A"
            RL_ASSERT( y == 1 );
            
            int x = m_x($).load(std::mo_relaxed);
            RL_ASSERT( x == 1 );
        }
    }

This just says the same thing but that you can substitude the acquire fence for a load-acquire. That is, a release fence can synchronize with a load-acquire just as if it was a store-release.

29.8.4 : An atomic operation A that is a release operation on an atomic object M synchronizes with an acquire fence B if there exists some atomic operation X on M such that X is sequenced before B and reads the value written by A or a value written by any side effect in the release sequence headed by A.

A is an atomic release op on object M

X is an atomic op on M, before B, reads from A
B is an acquire fence


    void thread(unsigned tidx) 
    {
        if ( tidx == 0 )
        {
            m_x($).store(1,std::mo_relaxed);
            m_y($).store(1,std::mo_release); // "A"
        }
        else
        {
            while ( m_y($).load(std::mo_relaxed) == 0 )
            {
            }
            // we just read a 1 from m_y // "X"
            
            std::atomic_thread_fence(std::mo_acquire,$); // "B"
            
            int x = m_x($).load(std::mo_relaxed);
            RL_ASSERT( x == 1 );
        }
    }

Again the same thing, just saying an acquire fence can synchronize with a store-release.

That's about for interesting stuff in 29.8 , there's a bit more in 29.3 on fences :

29.3.5 : For atomic operations A and B on an atomic object M, where A modifies M and B takes its value, if there is a memory_order_seq_cst fence X such that A is sequenced before X and B follows X in S, then B observes either the effects of A or a later modification of M in its modification order.

A modifies M , before X
X is a seq_cst fence
B reads M , after X

Then B cannot see a value of M from before A. 

Well, duh. Note that of course this is only non-trivial when A and B are done on different threads. And B being "after X" usually means B is after something else in the total order S which you know to be after X.

This sort of says that a seq_cst fence acts as a #StoreLoad. Note however that both A and B must have "happens before/after" relationships with the seq_cst fence. If only one of them has that relation then it doesn't work. We'll revisit this in the next post when we talk about how seq_cst fence doesn't behave exactly as you think.

29.3.6 For atomic operations A and B on an atomic object M, where A modifies M and B takes its value, if there are memory_order_seq_cst fences X and Y such that A is sequenced before X, Y is sequenced before B, and X precedes Y in S, then B observes either the effects of A or a later modification of M in its modification order.

A modifies M , before X
X is a seq_cst fence
Y is a seq_cst fence, Y is after X
B reads M , after Y

Well, duh.

29.3.7 For atomic operations A and B on an atomic object M, if there are memory_order_seq_cst fences X and Y such that A is sequenced before X, Y is sequenced before B, and X precedes Y in S, then B occurs later than A in the modification order of M.

A is an RMW on M , before X
X is a seq_cst fence
Y is a seq_cst fence, Y is after X
B is an RMW on M , after Y

then B is after A on the RMW order of M.

There are two forms of very strong ordering guaranteed in C++0x. One is the total order (called "S") of seq_cst ops (aside : I believe that seq_cst fences count as independent entries in the list S, but I don't see anywhere that the standard actually says that). The other is the modification order of an atomic object. There is guaranteed to be a single consistent modification order to every atomic, though not all threads may see the same order, depending on how they view it. Furthermore, any Store without a read (not an RMW) clobbers the modification order, because it wipes out the history and makes it impossible to know what the order before the store was. But you can say something strong : as long as you only use RMW ops on a variable, then it has a single global order of changes, and every thread can only view those changes in order.

eg. say a bunch of threads are doing RMW Increment ops on a shared variable. Then the variable must go {1,2,3,...} in its timeline, and the threads that do it can only see an ordered subset of those values, eg. {1,3,4,8,...} but never {3,1,7,4,....}.

Anyhoo, 29.3.7 is just saying that the total order of seq_cst ops (S) and the total order of an RMW sequence can order against each other.

Some of the standard document points are so redundant, I'm surprised they don't have an entry for the converse :

X is a seq_cst fence, before A
A is an RMW on M
B is an RMW on M, after A
Y is a seq_cst fence, after B

then Y is after X in the total order S

which presumably is also true.

Okay, so next post we'll look at what the fence is *not*.

05-27-12 - Prefer Push-Restore to Push-Pop

It's a standard paradigm to use a Stack for a subsystem which parallels the execution stack.

Some common examples : profilers that push/pop scopes, logging scopes such as setting tab depth or subsystem, mutex lock scopers, exception handler stacks, etc. Basically the model is at the start of some function you push something on your parallel stack, and pop on exit.

I've only recently realized that this is bad programming. The problem is it's redundant with the execution stack, and any time you have two separate subsystems which must be exactly in sync to work correctly, you have bad code. In particular, it's (unnecessarily) fragile.

To be concrete lets consider a "stack" that's just an integer counting the depth for something. You might naively maintain it like :

Func1()
{
  s_depth ++; // push

  ... other stuff ..
  Func2()

  s_depth --; // pop
}

Now, sophomoric programmers may already be thinking you could use a "scoper" class to do the push/pop for you; as long as you use the scoper that ensures that you do push/pops in pairs, and it eliminates one type of bug, which is accidentally returning without popping.

But really the scoper is just putting a band-aid on the problem. There is a whole other class of bugs that you cannot fix that way - what if Func2() is written wrong and has a mismatched push/pop ? It will fuck up the stack to you. The entire mechanism is fragile because it is vulnerable to inheriting mistakes in any child code.

Now, sophomoric programmers are thinking "of course it inherits mistakes from calling broken code", they think that's just the way code is. They think the way you make working programs is by finding every single bug in every possible execution path and fixing them all. They think fragile code that has to be used "just so" is fine, because they will ensure that the code is used in exactly the right way.

This is very wrong, and it's why so many programs suck so bad. It comes from a great arrogance that is very common in programmers from beginners to experts. It's the (untrue) belief that you can write bug free code. It's the belief that fragile systems without fail-safes and inherent self-protection are okay because you are so great you will use them correctly.

It becomes obvious if we add the invariant checks that test program correctness :

Func1()
{
  int depthBefore = s_depth;
  s_depth ++; // push

  ... other stuff ..
  Func2()

  s_depth --; // pop
  ASSERT( s_depth == depthBefore );
}

The assert checks that after the pop, the stack should be back to where it was before the push.

But if that's the requirement, why not just do that directly? It's much more robust, it's not sensitive to unmatched push/pops in the children. So instead of Push/Pop we should just do Push/Restore :

Func1()
{
  int depthBefore = s_depth++; // push

  ... other stuff ..
  Func2()

  s_depth = depthBefore; // restore
}

A few quick notes on this : 1. Obviously for super-robust code you should run both methods simultaneously and check that they match; in case of mismatch, perhaps prompt the user (footnote *1*), or just prefer the more robust, and 2. it's always a good idea to write out the asserts that check the invariants you believe to be true in the code. Often you will find that the assert is simpler than the code it was checking. 3. the code is more robust if you just go ahead and make the invariant true. eg. you often see something like :

int Func3(int x)
{
  ASSERT( x == 1 || x == 3 );
  int y = x * 4 + 12;
  ASSERT( y == 16 || y == 24 );
  return y;
}

well if you know the parameters and you know what the answer should be, then just fucking return that. If you require a simple invariant, then just fucking make it true. Don't assume it's true without testing it. Asserting is slightly better, but it's still fragile. Just make it true.

Anyhoo, our s_depth example is so trivial let's do a slightly more complex one, a profiler :

struct ProfBlock { tick_t start; tickt_t end; }
vector<ProfBlock> s_profile_vec;

void Profile_Push()
{
  s_profile_vec.push_back();
  s_profile_vec.back().start = tick();
}

void Profile_Pop()
{
  s_profile_vec.back().end = tick();
  // [start,end] for this block is now set, as is stack depth, store it somewhere permanent
  s_profile_vec.pop_back();
}

Func1()
{
  Profile_Push();

  ... other stuff ..
  Func2()

  Profile_Pop();
}

and as noted in the comment, I assume you actually do something with the block once it's recorded, but the details of that are not necessary here.

Anyhoo, this is very standard, and it's sort of okay, but it's fragile. The issue is in Profile_Pop() - you are making a pure leap of faith that back() is actually the element you should be popping.

(in particular, a very common source of bugs or fragility in this type of code is if the Profiler can be enabled & disabled; even if you always use a Push/Pop scoper class to avoid unmatched pairs, people can enable/disable anywhere and that creates a mismatch (assuming you don't continue to do the push/pops when disabled)).

A better way is Push/Retore :

int Profile_Push()
{
  int i = s_profile_vec.size();
  s_profile_vec.push_back();
  s_profile_vec.back().start = tick();
  return i;
}

void Profile_Restore(int i)
{
  s_profile_vec[i].end = tick();
  // [start,end] for this block is now set, as is stack depth, store it somewhere permanent
  s_profile_vec.resize(i);
}

Func1()
{
  int p = Profile_Push();

  ... other stuff ..
  Func2()

  Profile_Restore(p);
}

If you like you can assert in Restore that it's equivalent to a Pop. But crucially, if you are doing the redundant method and asserting, the fallback behavior should be Restore. (in practice for me, once I realized that Push-Restore is what I really want, that opened the door for me to allow Pushes without Pops, so I don't check that Restore is equal to a Pop, I let them be unequal).

What is the fundamental principle that makes this so much more robust that doing push/pop? It's elimination of parallel states that must be kept in sync. That's one of the greatest causes of bugs and should be done whenever possible; I don't do it enough, it's so tempting sometimes and you don't really see the necessity of it (because we all have Programmer Arrogance that makes us think we can keep it sync correctly).

eg. you constantly see code like :

struct Blah
{
  int m_x;

  int m_y;

  // NOTE : m_y must = m_x * 2;
};

At least this example code has a note about the redundant relationship that must be kept in sync, which is better than lots of real production code, but it's still just a bug waiting to happen. The redundant state should be eliminated and any queries for m_y should just go directly to the single variable that is authoritative.

That's obvious, but the stack is the exact same situation. The thing is, the program is maintaining its own stack, the execution stack, so any other stack that mirrors the execution stack is redundant state. You should just use the execution stack, which is what Push/Restore does for you.

05-10-12 - Fix My Browser

I cannot for the life of me figure out how to do this. I want two (or three) things :

1. Make all mime types always just download. Forever. MP3, PDF, whatever the fuck, I never want any fucking handler to do anything with it. Just download it. I can't figure out a way to make this happen; I feel like the mime type database must be a text file I could just edit and set everything to =download but I can't find any info about that on the net.

2. Never ever do anything to my windows without asking me. Don't close them, resize them, move them, and certainly don't pop anything up.

3. No mouse-over popups. This is some new shit that is causing me constant frustration. Jesus christ putting mouse-overs on web pages is such a stupid broken idea. There's a bunch of web sites that I just can't use any more because of mouse-overs.

I'm currently using Firefox 2 and will be very sad when I have to switch to Firefox 3 because of HTML5 or whatever fucking new thing that makes the internet worse but which I cannot realistically opt out of.

---

In other "god the web is stupid" ranting, fucking forms that change as you enter values should be illegal. No, you are not fucking helping me. Like credit card forms that detect your type of card and then add the box for the CVV number with 3 or 4 blanks. No! You are not helping! Just make a static form.

Don't you realize that the web just doesn't work? Keep everything as simple as possible, because making it complex just makes it broken. Also, your UI design skills are rudimentary at best, so don't try anything ambitious. How about you just make a static form, and if I fail to fill in a required field you don't blank the whole thing out, mmkay?

Anyhoo, the latest trend that is fucking boiling my brain almost daily is this fucking "make your password strong" bullshit. It seems like every web site I go to these days wants me to change my password to be stronger, with some cock-ass idea of how to do that.

Of course every site has different ideas about what that should be, such as "must contain a space or !" while another is "must not contain a space or !". So I can't just use the same password in lots of places, I have to generate one per site and write them down, which of course makes them extremely non-secure. The worst of course are the places that don't tell you exactly how to generate a strong password, so you try some shit and they say "too weak" and you try some other shit and they say "funny character not allowed!" so you smash your keyboard and then try backing out the weird characters and wind up binary searching a few times through annoying forms until you get it. Look you fuck-heads, why don't you just auto-generate a random password for me that conforms with your rules, and then I can write it down on my desk and we can all pretend things are more secure even though the hackers just steal your whole user database on a near-daily basis and it doesn't matter how many numbers are in my password anyway.

---

In other computer broken news :

Somehow in DevStudio I accidentally hit "Alt-P-O" and didn't realize it. (I think all complex programs should have a log console of all your key and button presses and what they correspond to; often in Photoshop or whatever behemoth program I'll be pootling along and all of a sudden it does something wacky and I'm like "wtf what did I hit"; sometimes it's even a good result and I have no idea how to make it happen again). Anyway, apparently Alt-P-O turns on "Show All Files" in the solution explorer which is a feature I didn't even know existed. So all of a sudden a bunch of new crap appears to be in my project, and I spent a few hours deleting my NCB's and doing P4 reverts and trying to figure out WTF happened before I discovered this feature.

I've had a problem for quite a while where "Open With" didn't work; if I browsed for a program to do the open-with, it would let me pick it, but then just not open it. I finally got annoyed enough that I went and fixed it :

OpenWithAdd : Registering programs with the Open With dialog
Problems with File Association in Windows 7 64-bit - Microsoft Answers
OpenWithView - DisableEnable items in the 'Open With' dialog-box of Windows
Index of filesdips64
FileTypesMan - Alternative to 'File Types' manager of Windows
Can't get program into OPEN WITH list

Probably just using the "OpenWithAdd" utility will work if you have this problem. But fundamentally the problem arises because the registry gets itself fucked somehow. There are two different ends where it can get fucked and both seem to occur and both can cause this problem. Say you are trying to register .FLV to open with VLC. The registry for .FLV can get fucked and cause this, and there can also be a registry for VLC that gets fucked and causes this. Best bet to clean it up is just to delete everything in the registry for the file type .FLV and for the program VLC.

(the OpenWith command doesn't actually go directly to the program path that you give it, that would be too easy and robust; instead when you say "OpenWith" "VLC" what it does is go look up a registry entry for "VLC" which tells it where to get the exe and how to pass the args; that entry is what gets fucked somehow)

I still use Win XP at work (and Win 7 at home), and literally the only thing from Win 7 that I miss on the XP machine is the text-search in the start menu. I'm sure that could be added to XP and it would fully satisfy me. God damn the fucking control panel on Win 7 got so much worse, it's so fucking hard to find any advanced settings any more, and everything is split into a bajillion different pages. The typical thing I want to do when I get a new machine is to turn off all skins, turn off all graphics and backgrounds, turn off all cleartype, animations, special effects, etc; on Win 7 it takes me an hour of fumbling around in a ton of different menus to find all that stuff. There should be one single "display settings" page which has *everything* you can change about display settings all together on one page.

05-06-12 - Photos - Some Carpentry

I'm a total beginner, but I'm getting better. I find this to be pretty fun and rewarding; as long as it's not something that's on my life critical path, or that I have to do in a hurry, I enjoy it. It's fun to go out to the garage by myself and play some music and figure out how to do things. I'm sort of vaguely thinking about taking some classes, but then it starts to get too serious and I stop enjoying it.

So far the main thing I've learned is that prior to this I knew nothing about carpentry. I've made some little things in the past, but it was always just the totally uneducated "hey I'll stick some wood together and nail and screw it semi-randomly".

In reality the way modern American carpentry works is very systematic; the 2x4 stud and the 1x2 furring strip are not like multi-purpose legos that you can just fit together however you want; you use them in specific ways (and in fact the "stud" is not just a piece of wood, it is specifically engineered to be very strong in compression to bear the loads of stick framing, and not necessarily strong in other ways). You use specific sizes of nails for each specific task.

---

---

Some notes :

Garden beds are four 4x4 cedar with through bolts. I'm pretty happy with them in general but if I were to do it again I would not do the "high raised bed" style and instead just use two 4x4 posts (which gives you about a 7" raised bed, since a 4x4 is actually 3.5" , and you could use standard 8" carriage bolts in that case, I had to mail-order 15" bolts (actually better than carriage bolts are "timber bolts" which are just carriage bolts with special cutting flanges that hold the bolt better in softwoods like cedar)). Be careful when tightening bolts on wood, you do not want them to be really tight, just sort of lightly hand tight. To make the nut stick you may need two nuts that you tighten against each other.

Kitten loves the cat tower, and the fibrousy wallpaper stuff on it was pretty successful (she loves to climb straight up it, rather than jumping between the platforms usually), but it is breaking down; I'm not sure if there is any material that would be more resilient, perhaps some kind of woven bamboo mat? She can knock it over with her crazy stripper pole moves, we currently have it stabilized by leaning in a corner of the room; to really make it self-stable would require a really wide base or a lot of weight at the base, I think.

Potting table is standard apron-style table; it's 8 feet long so I used 2x6 apron rails. I pocket-holed the table top boards together, which I would not do again. If I made another table top I would probably just try to glue it (I don't have the clamps for that kind of glue-up at the moment); with an apron-style table the top is not really load-bearing anyway, the apron is the crucial structure, so the strength of the top joinery is not important. One thing you might not think of is the attachment of the top to the apron needs the ability to move side to side (but not up and down), so drill the screw shank holes oversize. (apron-style tables are super duper easy to make)

I only made the vertical trellis, the top part was already there, but with no way for a vine to reach it. One of my house pet peeves is when people have a trellis with nothing on it, so I had to do something about that. (one of the more retarded fads in house style was the "trellis top" fences that everyone was getting 5 years; hardly any of them actually have anything growing on them so there're all these stupid looking empty trelli in the fancy neighborhoods of Seattle). The only tricky thing about that trellis was trying to do the anchors to the house correctly. Piercing the skin of a house is almost always a bad idea and has to be done with care to avoid creating a water-incursion point. I used through-bolts to a block inside the sheathing. The bolts are angled down to reduce capillary action, and hopefully the caulk does the rest; the holes for the bolts must be over-size to avoid cracking the siding with seasonal flex. One mistake I made was running the bolts from the outside to the inside; I should have done it the opposite way, which would have let me install the block and bolts, and caulk and paint that assembly and let it fully dry before attaching the trellis.

Chicken coop - urr various things I would probably do differently here. It's over-engineered in some ways (more structurally sound than necessary for the weight of chickens), but also missing some important features, like a heat lamp (addendum : heat lamp was easily added via extention cord) and some easier access points. It currently has an egg retrieval door, a chicken ramp door, and a removeable wall and roof for cleaning access, but I think I have to add another door for human play time access. The run is also small compared to the coop, I may have to make a bigger run some day. Maybe the main annoyance at the moment is you have to get inside to open and close the chicken ramp door; when I made it I thought we would basically just leave that door open all the time, but we've been closing it to help them stay warm at night, and it's a pain to get in to open and close it.

05-06-12 - Photos - Pots from Q1 2012

My third quarter of classes ; Q1 2012 with ended in early April. It was a good quarter, I made a lot of progress throwing bigger and thinner. All the work from that quarter :

---

---

Most of my funky glazing experiments are failures, but notes on a few that I think worked okay :

Wacky layering of shino is great. You can also put it on super thick then scrape away some to create variation. An even more subtle way to get variation is to paint the bisque with irregular strokes of water before dipping in the glaze, because water changes the glaze adherance rate.

Rutile or iron-ox over glaze makes subtle/natural looking variations.

"Peacock eyes" on the outside of the red bowl was pretty successful. This was done by first doing the blue glaze, waxing over the eye shapes, then dipping all in red. You could use the same technique for things like clean-edge bowl rim bands.

Carved ridges on outside of small bowl does nice stuff for tenmoku.

05-06-12 - Photos - Hawaii 2011

I keep sort of day dreaming about getting a house in hawaii for frequent getaways. I'm not sure it's a good idea. Owning a house is a fucking pain in the ass; owning multiple is just extra pain. Maybe it could be made super low maintenance with a steel roof and only native/natural/xeriscape plantings. I dunno, whenever I rent a Hawaii house I always think about the shit I don't like about it (I think the vast majority of houses in hawaii are fucking gross and people just don't "get it" ; see below), but man not having to deal with the house is such a huge advantage.

(aside : I believe the proper way to build a Hawaii house (especially a vacation house that you aren't living in all year) is pretty much a roof with no walls. The weather is so good year round that you basically never need protection at all. I think the correct construction method is Japanese style, that is posts & beams that support the roof, and then any walls you may have (perhaps wood louvers) are not structural. The vast majority of houses in Hawaii are American style "stick framing" where the walls carry the roof, and they have windows that seal and air conditioning and all that vulgar tacky suburban shit that you don't need).

The pictures :

---

05-06-12 - Photos - Almost spring

This post is from back in march.

I wrote a little program to make randomized two-column image layouts. Spring is just starting; we've got lots of bulbs coming up, a few blooming, and the flowering trees are just starting. I think in a few weeks the real spring bloom will kick off and Seattle will be at its most beautiful.

Anyhoo, here's a tour of early spring around our house (featuring special guest star : the crazed kitten) :

---

05-04-12 - Toilet

WTF is wrong with you people who go and sit on the toilet for half an hour? If nothing is coming out, get the fuck up and get out of there. The only reason to park yourself on the toilet is if you have diarrhea (and I don't hear any pllfft coming from the stall so I know that's not it). If you're constipated there's no need to sit there, get the fuck up and get back to work; and hell if you're constipated on a daily basis, maybe you should think about eating a fucking vegetable once in a while.

Are you just sitting there trying to let gravity pull the poop out? You know you have to squeeze, right?

There are these very simple acts of daily life that we never get taught, and sometimes people somehow fail to grasp how to get them right. For example, up until quite late in my life, maybe high school or so, I didn't know how to drink liquid correctly. I would just pour the liquid into my mouth as if my mouth was a glass, and then I would close my lips and swallow; the result was that I frequently swallowed air with liquid and drinking anything would give me burps. Some time in high school it finally clicked that you have to actually suck as you drink and form a vacuum so that a stream runs down your throat, you don't just swallow a parcel of it. So maybe you long-sitters just don't know how to poop.

Whenever I go to someone's house and see a bunch of magazines and books in the bathroom I'm a bit freaked out and then take extra care not to touch anything in their bathroom. There should not be paper or any other non-waterproof product in a bathroom; ideally a toilet room should have a drain on the floor and tile walls so you can just hose the whole thing down.

The only time I've felt the desire to hang out in the bathroom for longer than the bare minimum of time was once when I had house guests staying over, and I was getting quite fed up with them (not that there was anything wrong with them per se, but after a few hours of entertaining people I'm ready to lobotomize myself). I randomly needed a poo and had a sit down and was going to do my usual one minute express, but I suddenly realized, whoah this is nice, I'm alone, nobody is trying to talk to me, I can just chill, and I sat there a few minutes and relaxed. So maybe you toilet campers are doing this all the time, hiding out?

I do think I have seen that behavior in married men. You can see in certain married men (especially with kids) a desperation to get some alone time. If you mention some group activity that their wife wouldn't want to do they instantly are like "oh yes please I'd like to do that". I see them at our local neighborhood pizza place waiting for take out; the desperate married men always show up early and just hang out in the bar for a while.

In other random bathroom ranting, I think that America has the worst designed bathrooms, perhaps in the whole world, certainly in the civilized world. The bathroom is not an appropriate place for carpet. The toilet should not be in the same room as your shower/bathtub and makeup/medicine cabinet, the toilet should be in a little closet all by itself. Almost every other country in the world gets this, but for some reason Americans just love to hang out in a room full of shit and piss.

I think the 80's were probably the pinnacle of terrible bathroom ideas in America, when people were actually putting plush covers on toilet seats (not even just the top part, but the actual horseshoe sitting part). Here's an example of the retarded type of bathroom you see in America :

Actually that one is not even close to the worst because it actually has tile on the walls, a lot (probably a majority) of american baths have drywall walls which is super fucking retarded. I know, let's put some porous, absorbant, non-waterproof material in the bathroom! good idea!

Even aside from the specific horrible design choices, the whole trend of spending a bunch of money to make your bathroom fancy is so fucking retarded. Of course most home improvement that the suburban wives do is not actually connected to functionality or even their own taste in any way, it's entirely based on what is popular at the time, and a kind of competition. Everyone is just so fucking disgustingly mindless, nobody has their own ideas, they don't think for themselves and do what they actually want; you can watch the builders around town and they all wind up doing the same thing, and then the fad changes and they all do the new thing; like for a while recently the big fad is chef's kitchens; it doesn't matter that the family basically never cooks and has no idea how to use a knife, yeah we need the fanciest chef kitchen; right now around Seattle I'm watching horizontal board fences go up at a blistering pace, it's suddenly become a mass fad and everybody needs a fucking horizontal plank fence, which will look tacky and dated in a few years (and actually usually already looks bad, because it looks fucking awful to put one modern-style feature on an old house that doesn't match it at all).

05-03-12 - Doing Work For Another

It's very pleasurable to do some hard work for someone else, when you can do something for them, push yourself, work at it for a while, and you feel like they really appreciate it. It's one of the principle pleasures of the human pack, it's part of what makes being single so horrible. (when you're single all the hard work you're doing is only for yourself, and you inevitably have to ask yourself "why am I doing this? I don't really care" and you wind up putting on a bath robe and becoming The Dude).

This hard work for someone else principle is huge in domestic bliss, from the smallest level to the overall. On the small scale, even tiny things like doing dishes are so much better if you feel like the family appreciates it, if they acknowledge and are kind about your contribution to the group. Of course children and husbands are consistently dickish about this, they don't want to give the respect and status elevation associated with gratitude.

Of course if you are a good person you should gladly let the other person have the pleasure of doing something for you. The tricky thing is knowing when they want to do it, and when they are just offering because they think you want it. When someone offers to do something for you, and you don't really particularly care, it's super dicky to just say so. One of the great social disfunctions of the nerdy computer guy is being too literal in conversation. You may think you are being clear by listening and responding to the content of the words, but in fact you are being a huge dick by not picking up what they really mean.

(in fact, often people use this as a way of intentionally being a dick; for example when someone is clearly trying to tell you something but they just can't say it directly, and you respond with "what are you saying" or "if you want something just tell me" or whatever, it's not their fault for failing to spit it out, it's your fault for failing to pick up the very obvious message that they aren't saying.)

Back to the case in point, when someone asks you "hey I was thinking of doing this thing for you which is quite a lot of work for me, would you like me to do it?" you are now faced with a puzzle (assuming you don't really care about the actual work product very much). Are they just offering for your sake (and would be happy if you said no, don't do it), or are they offering because they want to do it, and they want to do it for someone. eg. when your auntie offers to bake you cookies it's actually because she wants to bake cookies for you, it's not about what you want. In some cases it's tricky because it's hard to tell what's actually going on with their intention. You also can't just ask, of course, like "are you offering for my benefit, or do you just want to do that?" because people are not so self-aware, nor are they so open in their admissions.

Even once you are sophisticated enough to navigate that minefield, the next step is equally treacherous, when you get to the details of the act. Depending on whether you think the person was offering for your benefit or their benefit, you have to tailor your requests to what *they* want to do, even though they will ask you what *you* want to do. eg. when auntie offers to make you cookies, she will ask what kind; this case is easy because you know it's for her benefit, so you should choose a type of cookie that she likes to make; in some cases she has a "famous" variety (it's pretty fucking hillarious how many American secret family recipes are actually just the Betty Crocker recipe or some shit like that).

As another example, if I offer to throw a pot for someone, it's not because I think they really want me to, it's because I want to throw a pot, and it's nice to do it with someone in mind. Your only task in the job is to act grateful. You have about 30 seconds of work to do, if you aren't actually grateful you have to put on a smile and say thanks; it's just outrageously dickish if you fail to do that 30 seconds of work for me. (the same is true of most present receiving of course). Furthermore, if I offer to throw you a pot and you know I make sort of rustic ceramics don't say "what I really like are the cast art deco ceramics" ; you fucking asshole, if the conversation was entirely literal that would be a fine response, but of course it's not; the subtextual conversation went something like "I want to make a pot, can I pretend it's for you?" and you responded with "you can make it, but I won't let you pretend that I'm happy about it, and I'm going to belittle your work first". That example may be overly obvious, but a lot of the time people tell themselves they are being constructive or helpful by trying to push you with weird requests, or by pointing out ways your stuff could be better; that in fact is not helpful and is probably even more dicky (and in fact I believe they subconsciously know they are being dicky and it's sort of intentional as some sort of dominance bullshit).

Okay. So in real life you can never be completely sure what the subtext is. You can only make a probability estimate; you aren't entirely sure what the other person's intentions were. Then you have to do an analysis like a poker player. Human conversation is a game of imperfect information. Let's outline the game in this case :

Player 1 has a hand which contains either the "I do it for you" card or the "I do it for myself" card
Player 1 makes the offer which reveals some information in subtext
Player 2 has to make an estimate of 1's holdings; eg. based on the limited information I got, I
    esimated they have "for you" 20% of the time and "for me" 80% of the time
Player 2 says yes or no
Player 1 asks about the details ; this reveals a little bit more information
    and player 2 revises their estimates.

Player 2 must now consider their various possible responses;
fold = "forget it"
call = "do it your way"
raise = "actually please do it this specific way"

What you have to do is a weighted EV analysis; that is,
for each possible action I can take, what is the outcome in each of the hidden cases ("for you" or "for myself")
then weight the value by my current estimate of what the underlying truth is

People with very sophisticated social intelligence are actually doing this kind of analysis all the time, though I think they don't realize it. (I've written a rant very similar to this before).

Anyhoo, aside from the sort of micro, conversational cases like this, this principle of "doing it for myself but pretending to do it for you" is one of the primary macro forces in relationships.

The most obvious example is the stereotypical male/female family roles. The man goes off to work and puts in lots of hours and grinds himself down "for her", and the woman stays home and cleans and raises the kids and dreams up home improvements "for him". Of course in reality, the other person typically doesn't actually want that. eg. she would be happier if he didn't work so hard and come home so tired and grumpy, and she would be just fine if they made a bit less money. He in fact is not doing it for her, but because he wants the challenge, or wants the good job, or thinks he's supposed to for some reason or whatever. But he can be much happier if he is allowed to pretend that it is "for her". In a healthy relationship, she would allow him to have that illusion and support him to some extent (though in a truly healthy relationship you should also be able to have the conversation about "hey you're working too hard and it's not actually for the family's benefit" if it gets out of hand).

(there's an obvious sort of "Gift of the Magi" in the old 50's stereotype family roles; the man works his ass off "for the family" (not really), which just alienates him from his wife, she wishes he would come home with more energy for her, while the wife cooks and cleans and pretties up the house, which just annoys the man, he wishes she would just relax and leave well enough alone).

This macro feeling that what you're doing is "for the family" shouldn't really be in your mind every day (if it is, you might be one of those narcissistic psychopaths who constantly talks about how they would do anything for their family, including fucking over anyone outside their family, etc. (I recall reading about the Costa Concordia accident where some douchebag was proud to say that he had punched his way to a lifeboat to secure a seat for his child; it's sort of weird that these dangerous psychopaths are admired by a decent chunk of society (sort of like the surreal psychological dissonance I feel when I see admiration for someone who is just so obviously overtly evil like Dick Cheney or Rupert Murdoch))) - but it is a nice undertone to your whole life. Once a week you have a moment where you think "fuck, why am I doing all this? I'm so tired, maybe I should just run away to Asia" and in that moment you can think "it's for my family" and it makes you feel better. It's incredibly dickish (and a failed relationship) if the other person doesn't allow you to feel like you are making a contribution to the unit.

(it would be nice if there was syntax coloring for deeply parenthesized writing, I might do it more)

When one party complains about their workload in the partnership, often what it really means is that they aren't getting enough gratitude. In our stereotypical sexist couple, when the man comes home and complains about how hard work is, the correct response is not "well maybe you should work less, I don't want you to do this, don't pretend it's for me". When the stereotypical woman says "I'm sick of doing the dishes all the time" or whatever, there is some aspect of literalness in that, but the more important subtext is "I'm not receiving sufficient gratitude (or sufficient cohesion) to have the pleasure of feeling like I'm doing this work for our family unit".

---

Semi-related topic : a lot of the work that people consider "hard" is not actually. eg. things like being in the Army, or raising a child. They may be difficult, they may be grueling, they may be exhausting; but if you feel like your work is for a reason, if it's very clear to you what you are supposed to do, and you just go do it, and then you feel like it was for good - that's easy. That's the easiest fucking thing in the world. It doesn't matter how tiring the actual work activity is. When someone hands you a todo list and says do this then this then this, that's so peaceful and relaxing and easy. When you wake up in the morning and you know your todo list for the day (feed the kids, take em to school), that's easy.

The truly difficult thing is doing work that is only for yourself, that you're not really sure if it's a good idea or not, and you're not really sure if you're doing it the right way. That is fucking horrible and exhausting. Every few minutes you have to stop and think "wait, why am I doing this? maybe I should stop".

A lot of people get themselves through life by making up goals that they "have to" do. I'm not really sure how self-aware the average person is (for example, do bird watchers know that bird watching is fucking retarded? it is nice to get outside, and it's nice to have a reason to get outside, and it's nice to make up an excuse about why you "have to" go to the jungles of New Guinea, but doing it to see some bird is fucking ridiculous, right? do bird watchers actually realize why they are doing? and how arrogant am I that I think I know more about other people's real motivations than they do?) - but clearly people do this to themselves. Like I "have to" lose weight for my bikini vacation, or I have to do this kitchen remodel; it's like they create this task for themselves, and then for a while they act like it was an order given from on high that they cannot question, and that makes life easier because you just do that task and stop facing the cripling void of self-determination.

(another aside on bird-watching : in general I think the people who get obsessed with something minor which obviously is not deserving of that obsession are very silly; like triathletes, or hikers around here who try to "bag all the peaks", or people who travel around the world for one specific weird thing like birding or whatever. But in the end it seems to work for them in terms of quality of life. That is, I think it would be more reasonable if instead of training for your triathlon (which will often be miserable exercise and is not particularly good for your body), you instead just did some pleasant exercise that was actually more useful. The problem is that if you are not a wacko over-committed person who has this invented "has to", you just won't do it. Like I'm sure I would be happier if I went on a hike every week, but since it's just up to me, and I'm doing it only because I choose to, I wind up not doing it very often; on the other hand the goof-ball who decided he "has to" bag every peak in one year does do the hikes, and while that specific goal is retarded, it does force him to get out there. Obviously the standard "everything in moderation" is sound in theory, but in practice it just doesn't work, because if you are reasonable and moderate, it's hard to get out there).

(another example to beat this point to a pulp : it's easy to make fun of the "freds" on bicycles who track their heart rate and buy the fanciest bike shit, and generally over-obsess and make up goals for no reason; it would be nice if you could be more reasonable, you get just as much exercise on a slower bike, you don't need to fucking test your blood sugar if you are a recreational rider. It's very easy and trite for people like Bike Snob or Grant Peterson or Me to make fun of the freds and say that you should just "go ride". (as another example, obviously doing something like "bike to work month" is silly; if you want to bike to work, you can make that decision for yourself each day of the year, there's no reason to follow some organization's choice of month). However, if you are just reasonable and moderate and make your own decisions, you simply won't do it. It's too hard to make the right decision all the time, you won't have the motivation, you'll get lazy. In the end when you see the people who are out there riding at 6 AM in the rain, it's the nut-jobs, it's the people who invented some silly "have to", and in the end they are happier for it)

(of course the best form of made of "have to" is one that's at least semi-useful; like I "have to study hard and get into a good grad school"; having that goal can make the work fun and rewarding for a while; of course you eventually get disillusioned with that goal and realize it was perhaps all pointless; but the correct response is not to just give up on goals, but rather to invent a new one. (sometimes I try to get myself excited about some kind of made up goal, like I "have to" learn to fly an airplane, or I have to take a lot of artistic photographs, or I have to write some music, but it's hard for me to sustain the mental illusion).

Anyhoo, the relation is that some people use "do it for the family" as one of these made up goals to make their life easier. Like I "have to" get an SUV so I can drive all these kids around to their soccer and piano lessons and so on. Well, no you actually don't have to, but if it makes your life easier to pretend that you have to, then okay. This form of "do it for the family" is NOT what I was talking about in the main part of the post, but it is semi-related.