Parallel make faster than serial/N? - Re: non-x86 multicore - Re: CPU monoculture

On 21/02/12 3:04 PM, David Riley wrote: (still amazing; can you explain why it's superlinear?) off the disk) vs. CPU (compiling the file). Actually compiling the file Wouldn't a large buffer cache pretty much make that disparity vanish? Most systems can buffer many times more than the total size of source. I still don't see a clear explanation, because my reasoning is: - assume a very small cache in relation to source - parallel job kicks off 1..N compiles, let's say N unique source files and 20*N common include files. So each compile process works through 21 files. - the problem is, if your cache is smaller than that overall job, ANY of these compiles is going to blow the cache. Running in parallel doesn't help. On the other hand, if ALL of the source is cached, again, parallel make doesn't win, because serial make gets exactly the same advantage. Did I miss something - exactly what mechanism is helping if it's not cache? --Toby (depending on language, of course) can be miniscule compared to the overhead of reading off the disk. I typically use a number of jobs equal to 1.5 the number of processors in the system; I have a 4-core i7 which masquerades as 8 due to hyperthreading (which they actually got right in the i7, as opposed to the P4). I don't see a *lot* of difference between 9 and 12 jobs. would imagine it's because the header files then wind up in the file cache while you're using them. I've never profiled it.