09-22-12 - Oodle Beta and Roadmap

Oodle went Beta a few weeks ago (yay). If you're a game developer interested in Oodle you can mail oodle at rad.

I wanted to write about what's in Oodle new and what's coming, as a sort of roadmap for myself and others. This is not a detailed feature list; contact RAD for documents with more details about Oodle features.

Oodle Beta : (now)

What's in Oodle at the moment :

• OodleLZH and LZHLW : the LZ-huff compressors I have written about repeatedly. Designed for offline compression and very fast low memory decompression in game.

• PC, PS3 (+SPU) and Xenon support

• OodleIOQ : Fast low-level async IO

• OodleIOQStream : Streaming async IO (keeping buffers full for media playback) ; with Bink and Miles drop-in IO layers.

• OodleFile : Synchronous/buffered IO built on the async IO layer. This is a pluggable thing and there are memory OodleFiles and transparent LZ decompression OodleFiles and so on. Currently the ability to make your own OodleFile type is not exposed, but maybe it will be in 1.1 or so.

• OodleWork : Work-stealing thread pool system. Includes stackless coroutines. A bit more conscientiously designed than most.

• OodleHandle : Generic handle/dependency system so you can set up async chains (like load X then decompress it, then process it, etc.), WFMO mixing IO and work and SPU jobs and so on.

• OodlePackage : "Packages" (archives) with solid archiving and very async decompressors.

• OodleVFS : (virtual file system) for drive name mapping and per-drive file system overrides; also supports transparent mirroring from one drive to another.

• VFS patches, for overlaying a set of files over an existing set.

• Multithreaded memory allocator ; some other handy threading stuff like a tiny mutex; lots of other threading stuff that's in Oodle but not exposed to public API.

• Oodle Thread Viewer (super-lite Telemetry of sorts) (and real Telemetry integration for people who have that)

Oodle RC / 1.0 : (2012-2013)

• More platforms (Mac, Linux, iOS)

• Mostly just loose ends from stuff in the Beta. eg. Packages need to have update scripts to rebuild them when the source content changes. Packages need to be able to have their header in a separate file from the body (for header aggregation). etc.

• Lots of little stuff needs to get wrapped up; for example I need the ability to cache open file handles so that multiple opens for read on the same handle just use the cache, with some kind of LRU type of thing and a limit on the number of open handles kept.

Oodle 1.1 : (around GDC 2013)

• All major/current platforms (Mac, Linux, WiiU, ARM/mobile, next-gen consoles)

• A better cross-platform DVD to HD transparent mirroring system.

• OodleLZB and LZA (LZ-bytewise and LZ-arithmetic) , for when the LZH's don't quite hit the space/speed tradeoff you need.

• DVD layout tool and DVD emulator ; back when I was a real game dev I was dismayed at using the various platforms DVD emulators and layout tools, so I think doing a cross-platform one would be awesome. My goal for the emulator is that you should be able to just use your normal content on your disk, but with a layout file to tell me where on the virtual disc each file is I can emulate seeking and such. It should be possible to just flip a switch and test your game as if it was on DVD.

• PC File Serving Host. Basically a server you can run on your dev PC that will serve files to Oodle on a console so that you get transparent mirroring. (some consoles provide support for this already, some don't; it should be possible to do it for all consoles in a unified way).

• High level paging / resource manager interface. Oodle 1.0 is very imperative ; C-style. You can certainly make your own paging system on top of the Oodle 1.0 interfaces, but there's been a reasonable amount of queries about an out of the box ready to go resource manager. This would probably be a C++ style interface (perhaps in C though using function pointers).

• Paging transition package optimizer : you give me a list of resources for each region and a region transition map, I generate transition bundles with constrained optimization.

• OodleHandle for GPU Compute jobs ; be able to spawn & wait on them in the unified IO/Work system.

Oodle 1.2/2.0 :

• Image stuff is probably the biggest thing for the major rev of Oodle. DXTC, R/D-lossy DXTC, some kind of lossy image codec, perhaps just a fast JPEG decoder. I have lots of this code already, but it needs a big cleanup and "publicating".

• Patch generator.

• Stackful coroutines ? If so, then also a set of primitives that yield the coroutine instead of doing an OS wait; eg. "coMutex" and such that yield if blocked, and of course "coFRead" for files that yields if it can't return immediately.