May 3, 2010

When you hit a strange bug, make sure to rule out ccache

ccache is a very useful tool, especially on a from-source distro like Gentoo. It may make repeating compilations several times faster. However, when using it, you have to be aware of the cache corruption issues.

Recently I started a thread on gentoo-dev on that topic. If you just want examples of weird behavior, look at https://bugs.gentoo.org/show_bug.cgi?id=316657 and https://forums.gentoo.org/viewtopic.php?p=6262495#6262495.

If you are using ccache and hit a hard-to-reproduce compile failure, try re-running the compile with ccache disabled (FEATURES="-ccache" emerge ... should be fine; if you want to be extra sure unmerge ccache temporarily). If that fixes the problem, you should clean your ccache cache and then you can continue using it... up to the next failure. If you are adventurous and want to get to the root cause of the problem, see robbat2's hints.

I wonder about the idea of adding cache integrity checksums to ccache, so that the corruption can be detected. If you're looking for a way to contribute to Open Source, that may be an interesting project.