- you don't drop the disk cache; so you're loading from RAM (echo 3 > /proc/sys/vm/drop_caches)
- you don't have very complex code
- you timing includes starting/loading the intepreter itself.
Yes that is why I said the benchmark measures "just CPU impacts."
I don't think dropping cache alone would be sufficient to simulate loading of a large project -- I think the benchmark would also need to split the input across many files to recreate the seek time effects.
> you don't have very complex code
Given what I know about parsers, I highly doubt that the performance profile of parsing real code differs from my benchmark very much (ie. >30%). But I'm happy to be proven wrong on this, if anyone wants to try.
> you timing includes starting/loading the intepreter itself
That is why the benchmark makes the files pretty big, so the constant interpreter startup overhead is not a significant factor. Again, I don't think that you're going to be able to significantly change the shape of the results by adjusting for this.
