March 5, 2015

More reliable handling of bash history across terminals and crashes

I've been occasionally hitting frustrating issues with bash history getting lost after a crash. Then I found this great blog post about keeping bash history in sync on disk and between multiple terminals.

tl;dr is to use "shopt -s histappend" and PROMPT_COMMAND="${PROMPT_COMMAND};history -a"

The first is usually default, and results in sane behavior when you have multiple bash sessions at the same time. Now the second one ("history -a") is really useful to flush the history to disk in case of crashes.

I'm happy to announce that both are now default in Gentoo! Please see bug #517342 for reference.

August 7, 2014

Can your distro compile Chromium?

Chromium is moving towards using C++11. Even more, it's going to require either gcc-4.8 or clang.

Distros like Ubuntu, Mageia, Fedora, openSUSE, Arch, CentOS, and Slackware are already using gcc-4.8 or later is their latest stable release.

On the other hand, Debian Wheezy (7.0) has gcc-4.7.2. Gentoo is using gcc-4.7.3 in stable.

I started a thread on gentoo-dev, gcc-4.8 may be needed in stable for www-client/chromium-38.x. There is a tracker for gcc-4.8 stabilization, bug #516152. There is also gcc-4.8 porting tracker, bug #461954.

Please consider testing gcc-4.8 on your stable Gentoo system, and file bugs for any package that fails to compile or needs to have a newer version stabilized to work with new gcc. I have recompiled all packages, the kernel, and GRUB without problems.

The title of this post is deliberately a bit similar to my earlier post Is your distro fast enough for Chromium? This browser project is pushing a lot towards shorter release cycles and latest software. I consider that a good thing. Now we just need to keep up with the updates, and any help is welcome.

July 19, 2014

Recovering from removed (and missing busybox)

I was experimenting in my arm chroot, and after a gcc upgrade and emerge --depclean --ask that removed the old gcc I got the following error:

# ls -l
ls: error while loading shared libraries: cannot open shared object file: No such file or directory

Fortunately the newer working gcc was present, so the steps to make things work again were:

# LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/lib/gcc/armv7a-hardfloat-linux-gnueabi/4.8.2/" gcc-config -l
 * gcc-config: Active gcc profile is invalid!

 [1] armv7a-hardfloat-linux-gnueabi-4.8.2

# LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/lib/gcc/armv7a-hardfloat-linux-gnueabi/4.8.2/" gcc-config 1 
 * Switching native-compiler to armv7a-hardfloat-linux-gnueabi-4.8.2 ...

Actually my first thought was using busybox. The unexpected breakage during a routine gcc upgrade made me do some research in case I can't rely on /bin/busybox being present and working.

I highly recommend the following links for further reading:

June 1, 2014

perl-cleaner slot conflict when upgrading perl (5.16 -> 5.18)

If you tried upgrading from stable amd64 to ~amd64 or otherwise done a big update of perl, you probably hit this weird perl-cleaner slot conflict:

# perl-cleaner --all
!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:


  (dev-lang/perl-5.18.2:0/5.18::gentoo, installed) pulled in by
    =dev-lang/perl-5.18* required by (virtual/perl-IO-1.280.0:0/0::gentoo, ebuild scheduled for merge)
    ^              ^^^^^                                                                                                                                  
    dev-lang/perl:0/5.18=[-build(-)] required by (perl-core/version-0.990.800:0/0::gentoo, installed)
    (and 7 more with the same problems)

  (dev-lang/perl-5.16.3:0/5.16::gentoo, ebuild scheduled for merge) pulled in by
    =dev-lang/perl-5.16* required by (virtual/perl-Package-Constants-0.20.0-r3:0/0::gentoo, installed)
    ^              ^^^^^                                                                                                                                  
    (and 6 more with the same problem)

This is bug #506616, and the solution is to run the following command:

perl-cleaner --all -- --backtrack=30

January 20, 2014

System update report (last update a year ago)

I have some systems that I update more rarely than the others. Of course I highly recommend keeping up to date, especially for the security fixes. I've also written similar update reports in the past. You may want to read them and compare the experience:

Another 5-month update (December 2011)
Another report from rarely updated system (September 2012)

December 4, 2013

Testers wanted: Aura in Chromium Dev channel (33.x)

If you're using hard masked www-client/chromium dev channel packages (currently at version 33.x) you're probably used to testing things and encountering breakages from time to time.

If you feel bored or adventurous, or both, or would just like to try the new Aura UI of Chromium (which will become default at some point), I encourage you to enable USE="aura" for www-client/chromium. Upstream still keeps it disabled by default, and that includes Google Chrome. In Gentoo it's easy to make a choice about this.

Aura is a new UI architecture which is GPU accelerated. You can read about the technical details in its documentation. Especially because of the use of hardware acceleration it'd be useful to get many people to test it. If these changes break in your configuration, you can know now (while it's still early in development process), report a bug (consider posting the bug link in the comments so I can ensure it gets proper attention), and get back to the working configuration.

Possible workaround for GPU problems is using --disable-gpu command-line flag. Note that this still means there is a bug that is unlikely to get fixed if you don't report it. It's also useful to include contents of about:gpu page in your bug report.

Finally, see the screenshots below for an idea how it looks like (click thumbnails to see original images):

with aura
without aura
You can start compiling now, chromium-33.0.1726.0 has just been added to the tree.

November 26, 2013

Third party libraries in Chromium sources

One of the most common obstacles or issues related to inclusion of Chromium in Linux distro repositories are bundled libraries. Last attempt to blog about it I know about is Evan Martin's Forking upstream software post. I decided to take another look.

It is important to know that even if something appears in a third_party directory in Chromium codebase, it is not necessarily a bundled library. Third party code - yes, but not necessarily a bundled library. What's the difference? Well, even Fedora in its excellent No Bundled Libraries article lists e.g. copylibs as a possible exception. What about code that was never intended to be used as a shared library, is part of larger codebase, but is still useful? This will come up in some examples below.

Here is my list of third_party code still present in Gentoo's Chromium packages as of version 33.0.1711.3 (dev channel). This means that the libraries that have been successfully unbundled are not included. Similarly, code that is not used on Linux is not included. This takes into account intended audience - mostly Linux users, some of them packagers. A star (*) means that the code was already there in 2009.
  1. base/third_party/dmg_fp (*) - David M. Gay's floating point routines (dtoa, g_fmt). I don't think there is a shared library for these. There is also about using V8's routines.
  2. base/third_party/dynamic_annotations - single .c file with corresponding header containing annotations for dynamic tools like valgrind, tsan. Doesn't seem to be worth extracting, which could likely add unwanted dependencies on these tools.
  3. base/third_party/icu (*) - this could be extracted to use system icu; supporting nacl may be a challenge there (base is most likely also compiled by the nacl toolchain, and it's not obvious to me how shared libraries would work there - if at all, or whether it would make sense).
  4. base/third_party/nspr (*) - it may be possible to remove it now that Gentoo dropped nacl support for other reasons (
  5. base/third_party/symbolize - part of another Google's project, google-glog. Technically should be possible to extract, and glog even made a release with a tarball.
  6. base/third_party/valgrind (*) - bundled to avoid depending on valgrind just for build... IMHO fine.
  7. base/third_party/xdg_mime (*) - looks like the code was not intended to be used as a library, but maybe the intention was to avoid forking a process. Probably worth a closer look.
  8. base/third_party/xdg_user_dirs (*) - see this comment in the source code:
      This file is not licenced under the GPL like the rest of the code.
      Its is under the MIT license, to encourage reuse by cut-and-paste.
      Copyright (c) 2007 Red Hat, inc  ...*/
  9. breakpad/src/third_party/curl - great candidate for unbundling (or just disabling breakpad for Chromium builds in a way that doesn't try to touch curl even when disabled).
  10. chrome/third_party/mozilla_security_manager - parts of Mozilla code; doesn't seem to be designed as a shared library; has local modifications.
  11. crypto/third_party/nss - selected files extracted from NSS; there are some modifications, but with enough effort it may be possible to unbundle.
  12. net/third_party/mozilla_security_manager - parts of Mozilla code, different from the chrome bits above.
  13. net/third_party/nss (*) - parts of Mozilla's NSS (libssl) with experimental patches. Note that NSS developer is working there, so this can be seen as even more bleeding-edge than NSS trunk.
  14. third_party/WebKit (*) - now Blink, developed as part of the same Chromium project, but a fork of third party code. Not designed to be used as a shared library.
  15. third_party/angle_dx11 - developed by Google/Chromium developers; doesn't seem to be designed to be used as a shared library, but with enough effort it should be possible.
  16. third_party/cacheinvalidation - same as above.
  17. third_party/cld (*) - developed by Google/Chromium developers, probably to be replaced with cld2, which will hopefully be closer to a shared library design.
  18. third_party/cros_system_api - related to ChromeOS, not really bundled but rather just part of the project.
  19. third_party/ffmpeg (*) - Chrome uses very recent ffmpeg; I think the local modifications status has improved greatly since 2009: looks like patches make it upstream pretty quickly.
  20. third_party/flot - JS library, AFAIK there isn't really a concept of having system JS libraries. It could actually be useful to have one, but it's not obvious.
  21. third_party/hunspell (*) - modified to support running under sandbox and loading dictionaries in a different format; maintainers do respond but are very busy. This is doable but requires a fair amount of effort to figure out what to do with the different dictionary format.
  22. third_party/iccjpeg - taken out of lcms library, and the maintainers don't want to expose it.
  23. third_party/jstemplate (*) - Google's JS templating library.
  24. third_party/khronos - GL headers, unfortunately with local modifications.
  25. third_party/leveldatabase - needs a redesign to allow applying Chromium-specific behavior ( at run-time instead of at compile time. I've seen a Debian package for leveldb, looks like there is some interest in using it as a library.
  26. third_party/libjingle (*) - used to have semi-inactive upstream, now seems to become a part of WebRTC. When things stabilize more, worth another look.
  27. third_party/libphonenumber - upstream seems to be more focused on Java version of it, which actually has releases; C++ version doesn't seem to be designed to be used as a shared library.
  28. third_party/libsrtp - used to be inactive but now has a new home at and there are Googlers helping out with it. Worth taking another look when things stabilize. Note that even if it compiles it doesn't mean it works, see bug #459932.
  29. third_party/libusb - locally made incompatible change needs to be upstreamed (
  30. third_party/libvpx - waiting for upstream release supporting vp9, see bug #487926.
  31. third_party/libwebp - waiting for upstream release supporting APIs Chromium depends on, see
  32. third_party/libxml/chromium - this is ugly: code is actually part of Chromium codebase; at least it's not really bundled.
  33. third_party/libXNVCtrl - part of nvidia-settings. Not sure if it's intended to be used as a shared library, but it seems totally possible technically, and I even remember some success reports with it.
  34. third_party/libyuv - Google/Chromium project. Should be possible to use as a shared library, but doesn't seem to make releases.
  35. third_party/lss - Linux Syscall Support; a header based on Linux kernel headers.
  36. third_party/lzma_sdk (*) - lzma library from ; it would be great to replace it with xz-utils which distros package.
  37. third_party/mesa - I think only headers are used, but it's complicated.
  38. third_party/modp_b64 (*) - README.chromium points to Doesn't seem to be design to be used as a shared library, but it seems possible.
  39. third_party/mt19937ar - not designed as a shared library, rather small; can be removed after move to C++11 (looks like <random> would support needed functionality).
  40. third_party/npapi (*) - NPAPI headers with modifications.
  41. third_party/ots (*) - OpenType sanitizer, may be possible to package as a shared library, although it doesn't seem to have releases.
  42. third_party/polymer - JS library by Google, see
  43. third_party/pywebsocket (*) - Python WebSocket server used for testing. Should be possible to package it separately.
  44. third_party/qcms - color management library. Last upstream commits seem to be over a year ago, but the bundled copy continued to receive various updates, at least for more recent toolchain support.
  45. third_party/sfntly - font-related library; doesn't seem to have releases, doesn't seem to be designed to be a shared library.
  46. third_party/skia (*) - graphics library, changes very often.
  47. third_party/smhasher - hash function library - doesn't seem to have releases or be designed to be a shared library.
  48. third_party/sqlite (*) - available as a package, the biggest obstacle is lack of a good API to use it in a multi-process sandboxed context and also test it. See That obstacle would disappear however when Chromium drops support for abandoned webdatabase spec.
  49. third_party/tcmalloc (*) - although theoretically available separately, the Chromium copy is heavily modified, and that includes hardening changes important for security.
  50. third_party/tlslite (*) - Python crypto library, only used for testing but appears to be modified in a non-compatible way.
  51. third_party/trace-viewer - not obvious what it really is, and it contains several more bundled libraries inside.
  52. third_party/undoview - code extracted from gtksourceview.
  53. third_party/usrsctp - user-space SCTP implementation with local changes.
  54. third_party/webdriver - mostly some minified JS embedded in C++ code.
  55. third_party/webrtc - Real-Time Communications library - doesn't seem to have releases, and seems to be moving pretty fast.
  56. third_party/widevine - stubs for proprietary content distribution module.
  57. third_party/x86inc - asm code extracted from x264 with local modifications; I don't really see a good way to provide that as a system package.
  58. third_party/zlib/google - this is ugly: code is actually part of Chromium codebase; at least it's not really bundled.
  59. url/third_party/mozilla - parts of Mozilla code; doesn't seem to be designed as a shared library; has local modifications.
  60. v8 (*) - although the path doesn't contain third_party, I consider it bundled code. See When the libraries you use are moving too fast for the reasons it's there. While technically not part of Evan's 2009 list, it was obviously there since the beginning.
60 entries look like a lot. I would like that number to be smaller. On the other hand, note that many of these codebases were not designed to be used as shared libraries, some were developed as part of Chromium project, and that the project is very careful to put code it borrows from outside in third_party directories, whereas it's not uncommon for open source projects in general to incorporate such code directly into their codebases. In Chromium it's just much more visible.

Also note that while 23 of these items still exist, for some entries from 2009 we're now using system libraries, at least in Gentoo. Just to give you a few examples (the list is not necessarily complete - star means it's present on the 2009 list):
  1. flac
  2. harfbuzz (*)
  3. icu (*)
  4. jsoncpp
  5. libevent (*)
  6. libjpeg (*)
  7. libpng (*)
  8. libxml (*)
  9. libxslt (*)
  10. minizip
  11. nspr (*)
  12. openssl
  13. opus
  14. protobuf (*)
  15. re2
  16. snappy
  17. speex
  18. xdg-utils (*)
  19. yasm (*)
  20. zlib (*)
I'm interested in your opinions, so feel free to add your comment below. If you liked this post, you may also like State of Chromium Open Source packages.