10-29-08 Blog Ho!

Sweet! Somebody at Google flipped my switch and my blog is now getting up. (URG; update : it looks like my limit is now 1000 posts a day and I just ran out). I did some fixes in TextBlog and it's running pretty nicely now. One fix was to try-catch every network op and retry them a few times, because the server can just randomly drop connection once in a while. The other fix was to only query the list of entries once instead of doing a stupid N^2 loop.

It's still a bit slow to just upload one post. The problem is that I have to get the list of all the blog entries to see if you're doing an update or a new post. Getting the list is slow because I can only get 25 at a time and I have to get 25, follow the next page link, wash rinse repeat. Just getting the list of all the posts takes something like 10 seconds. I guess if I was confident that my local disk cache of the posts was accurate, then I could just use the local cache and skip that completely, but I'm trying to be extra safe for now.

Speaking of extra safety, it's something I'm a little worried about in Oodle. The issue arises with "baked" content and original file syncing, and different versions of the game, different versions of the baker, and different versions of the original file. There are a huge host of possible problems.

The basic bad situation goes like this :

Artist updates a file on their machine

Oodle does an automatic local Bake of that file to get it in the game

They see the file in the game and are happy

At the same time, a server is running an auto-bake of the whole game using an older rev of the source file

The server checks in the bundles from the older file, but the bundle itself from the server is newer

Artist sync to the server, the newer bundle comes down

The game prefers the newer bundle, and the artist no longer sees their edit
The fix is for the bundles to track a lot of information about how they there made. The mod time of the files they were made from, maybe the Perforce revision #, who made them, what version of the code made them, etc. The disadvantage is that I can't be minimal about loading bundles then, I have to just go ahead and load all of them to look inside and who's got the newest version of everything.

I think during development it's better to be safe and have a slightly slower load that does scan everything and give you the best content.


Autodidactic Asphyxiation said...

This reminds me of an issue I had at work where I was checking in compiled code into source control. It could be a real disaster waiting to happen, especially since things can bite you in the ass if you do branching and integrating, which p4 is normally pretty decent at but will have no idea how to deal with your baked files, no matter what kind of manifest you keep around.

I don't know anything about art toolchains for games, but I don't see why the artist would ever need to do a local bake. Couldn't there just be a way to override stuff in Oodle with stuff from a plain-old filesystem? Or maybe the local bake creates an overlay thing, in case you still need to do some kind of transcoding.

cbloom said...

The local bake is just a cache; I don't think artists should ever check that stuff in.

In fact if you prefer, you can pretend it doesn't exist at all. The existence of the local bake doesn't really change anything, it's equivalent to loading local source files. You still have to tell whether to load the server compiled level or the local source files.

It's exactly equivalent to building code with "make".

The artist source content is "c" files. The local bake are your "obj" files. Then you have an exe. You're working on a collaborative project and you have a build machine that makes an exe and checks it in. You want to know whether you can just use the checked in exe, or whether any of the local files are newer and you need to use the local data.

old rants