tag:blogger.com,1999:blog-5246987755651065286.post8343464429004267782..comments2024-02-22T16:15:42.388-08:00Comments on cbloom rants: 09-12-10 - Challenges in Data Compression 1 - Finite State Correlationscbloomhttp://www.blogger.com/profile/10714564834899413045noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-5246987755651065286.post-50814762579785497102010-09-14T12:01:59.828-07:002010-09-14T12:01:59.828-07:00Yes. Once RecordModel figures out the cycle length...Yes. Once RecordModel figures out the cycle length, it uses combinations of the bytes to the left and above and the cycle position as context. Figuring out the cycle length has a lot of heuristics and doesn't work perfectly.Matt Mahoneyhttps://www.blogger.com/profile/13946883164366534088noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-10233826793873577392010-09-13T18:11:36.915-07:002010-09-13T18:11:36.915-07:00This is "recordModel" in PAQ8 I assume?
...This is "recordModel" in PAQ8 I assume?<br /><br />It is a form of what I describe as hard-coding a certain type of structure and then seeing if your data matches that structure.<br /><br />PAQ is pretty strong in that way, because you can hard code a variety of structures and then the mixer will pick the ones that fit the data.<br /><br />I haven't figured out everything that's in PAQ8 yet, there's a lot!cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-57946502661375179322010-09-13T18:00:00.966-07:002010-09-13T18:00:00.966-07:00Of course the general problem is not computable. H...Of course the general problem is not computable. However, PAQ is able to find the record length in many files with fixed sized records. It looks for characters or pairs of characters that repeat at regular intervals. For example, if it finds "X" at offsets 0, 72, 144, and 216, it would guess that the record length is 72 and then model appropriately using contexts like offset mod 72 and neighboring contexts in 2-D.Matt Mahoneyhttps://www.blogger.com/profile/13946883164366534088noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-83920097032591418212010-09-13T00:07:17.904-07:002010-09-13T00:07:17.904-07:00http://encode.ru/threads/1127-Structure-detection<a href="http://encode.ru/threads/1127-Structure-detection" rel="nofollow">http://encode.ru/threads/1127-Structure-detection</a>Shelwienhttps://www.blogger.com/profile/15845762957306674934noreply@blogger.com