So I started reading about stutters and of course there are a million pages on it where people randomly suggest things that don't actually help (change this or that windows setting, install new codecs, blah blah blah). I tried some of the random things that are supposed to maybe help, such as the Windows dual-core TSC hot fix, the AMD dual core optimizer, turning off cool & quiet, turning DXVA on and off, etc. None of that helped.
Of course the real answer is to diagnose the problem and fix the actual issue instead of just randomly changing things. The difficult thing about diagnosing a stutter is you can't just look at what is taking too much CPU in Resource Mon or whatever, because in the steady state it's not happening.
Every video game should have some kind of "long frame trap" mode. What this would do is trace every frame, and if the frame was faster than some threshold (33 millis for example) it would just discard the trace; if it was longer it would parse the trace and pause the game and let you see a hiprof. The idea is that what you care about is the rare spike frame, not the average. (this is a little harder on modern games with many threads and an unclear concept of "one frame of takes X millis" due to the fact that many operations are deferred or spread across several frames)
Anyway, there's a new Windows tool I didn't know about which does something like this : xperf ; which is in the Windows Performance Analysis Tools, which is in the Windows SDK. They are supposed to be Vista+ but you can actually copy xperf.exe onto an XP system and use it in limited ways.
So xperf is cool, but I didn't actually end up needing it because some digging revealed the spikes were due to DPC's. The driver DPC just seems like one of the design flaws in windows. It means that badly written drivers can ruin your computer by causing this stuttering.
The old DPC Latency Checker for XP+ is cool, but to figure out what driver exactly is causing the stutters, you have to go to Device Man and turn on and off devices one by one. There's a new awesome one for Vista+ called LatencyMon that tells you exactly what driver is taking the time. Radical. My newer Dell Precision lappy has no DPC problems, the longest time taken is by the MS network layer (ndis.sys).
So I found the problem on the HTPC was the Asus/Ralink 802.11n Wifi card in the HTPC. The card is branded Asus, but apparently the key bits are actually Ralink. I tried various versions of the drivers for it (from Asus and from Ralink) and all had the problem more or less. I tried switching my network to 802.11g , still the problem. I tried the "WLan Optimizer" which turns off the periodic Wifi scan for networks, no help. So I yanked the card out of the machine and stomped on it. Stutters gone!
Now I have a 50 foot net cable running across the house and I love it.
Some related links :
Measuring DPC time with XPerf - Pointless Blathering - Site Home - MSDN Blogs
Watching H.264 videos using DirectX Video Acceleration (DXVA) My collection of short anime reviews
Stuttering Playback - MediaPortal Wiki
SAF v4.00 ''stable'' (StandAlone Filters) - DXVA ready (H.264 and VC-1). - MediaPortal Forum
Resplendence Software - LatencyMon DPC, ISR and pagefault execution monitor
Resplendence Software - Free Downloads - LatencyMon
ReClock 188.8.131.52 - VideoHelp.com Downloads
Ralink corp. Windows
My Solution for Dell XPS M1530 DPC Latency
What video stuff can Reclock do! [Archive] - SlySoft Forum
How to config ffdshow and reclock for HD audio, - Doom9's Forum
FAQ-WLAN Optimizer - Optimize wireless gaming, audio and video streaming...
Codec FAQ - Playback issues
DXVA - Free with the new DivX Plus H.264 Decoder DivX Labs
M-Audio Delta 1010 Troubleshooting - CPU Throttling
ATI HD Hardware Accelerated DXVA for H.264 AVC L5.0 L5.1 Zach Saw's Blog
ASUS Driver Download ASUS Driver Upgrade (Wireless LAN Security & Wardriving - 802.11)