tag:blogger.com,1999:blog-5246987755651065286.post2466956311745811739..comments2024-02-22T16:15:42.388-08:00Comments on cbloom rants: 09-30-11 - Don't use memset to zerocbloomhttp://www.blogger.com/profile/10714564834899413045noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-5246987755651065286.post-49782257602539656582011-10-03T12:01:48.952-07:002011-10-03T12:01:48.952-07:00Also, if you look at the example in the original p...Also, if you look at the example in the original post it's a malloc of 20 MB. I'm really talking about large tables here.cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-15152926025767346182011-10-03T12:00:53.295-07:002011-10-03T12:00:53.295-07:00Sure sure; but if it fits in L1 it's tiny tiny...Sure sure; but if it fits in L1 it's tiny tiny. You can't rely on L1 being bigger than 32k or so, and you have to share that with lots of other stuff, so you can only fit maybe a 4k table reliably in L1.<br /><br />Also, I try to always write code that doesn't use the stack because unfortunately other platforms are not as stack-friendly as Windows.cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-31732005353557100852011-10-03T11:55:52.722-07:002011-10-03T11:55:52.722-07:00Fast match-tables can be small enough to fit into ...Fast match-tables can be small enough to fit into the stack, since they are designed to use L1 cache.<br /><br />In such case, i'm not sure memset() can be avoided. Stack memory is typically not zeroed by OS.Cyanhttps://www.blogger.com/profile/02905407922640810117noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-76444517128048081332011-10-02T09:53:21.630-07:002011-10-02T09:53:21.630-07:00"memset is not bad if you're going to use..."memset is not bad if you're going to use the memory soon afterwards anyway, because it just warms the cache for you."<br /><br />This is only true for tiny allocations. And I'm clearly not talking about tiny allocations.<br /><br /><br />BTW It's particularly compelling for uses like "cache tables" ala LZRW or LZP matching, because you might allocate a 512MB cache table, and then large sections of it might never be touched at all. (especially on small files)cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-85680247323817599572011-10-02T08:30:15.335-07:002011-10-02T08:30:15.335-07:00Relying on the OS to clear pages is very expensive...Relying on the OS to clear pages is very expensive if it results in a pagefault on every initial page acess. It's much cheaper to use hot pages already in your heap than to map new ones in that case, even if you have to memset them yourself.<br /><br />memset is not bad if you're going to use the memory soon afterwards anyway, because it just warms the cache for you.<br />(BTW, offline page zeroing daemons are often a bad idea because of this; they zero the pages then let them cool down before using them; much better to zero the page just before you're going to use it.)<br /><br />The benchmark in the stackoverflow question is completely stupid because it doesn't actually touch the memory after calloc. If it did, it would be about the same as the memset one because of all the pagefaults.jsgfhttps://www.blogger.com/profile/01559704696397678598noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-58957666239330077362011-09-30T17:00:23.761-07:002011-09-30T17:00:23.761-07:00word.word.cbloomhttps://www.blogger.com/profile/10714564834899413045noreply@blogger.comtag:blogger.com,1999:blog-5246987755651065286.post-92055036969617818222011-09-30T12:50:57.189-07:002011-09-30T12:50:57.189-07:00http://stackoverflow.com/questions/2688466/why-mal...http://stackoverflow.com/questions/2688466/why-mallocmemset-slower-than-calloc<br /><br />It looks like glibc likely uses virtual memory tricks for calloc. This isn't surprising since it will use mremap for large, aligned reallocs.won3dhttps://www.blogger.com/profile/09787472194187459747noreply@blogger.com