Watch out for shadow-4.1.5-r1 and pambase-20120417

I recently bumped shadow-4.1.5-r1 and pambase-20120417. bug #412721 has been filed about people being locked out after the update.

The fix is to run dispatch-conf. The reports are unclear, but you might not get any message from emerge that it is necessary. If dispatch-conf doesn't update anything, make sure to manually re-emerge pambase.

What's going on? /etc/pam.d/{login,passwd,su} are being moved from shadow package to pambase, so they can be shared with hardened-shadow. They are not really shadow-specific, but Gentoo-specific, so it makes sense to make pambase own those files.

www-client/chromium and libjpeg-turbo

Recently Gentoo bug #412031 (">=www-client/chromium-18.0 should not force the media-libs/libjpeg-turbo dependency") has been filed. Samuli did the right thing and closed it, but I'd like to use this as an opportunity to explain the issue in more detail.

The original bug is #393471, filed by Gentoo Chromium team member Mike Gilbert four months ago and fixed in February. An optimization done in WebKit JPEG decoding code (which speeds up JPEG decoding by up to 2x) assumed that the code will run against the same version of JPEG library it has been compiled with. In general such assumptions do not hold (with the note of SONAMEs, i.e. same library SONAME should mean it's binary-compatible; more on that later), but Google Chrome bundles all of those third-party libraries anyway, so for Google it's true.

I started a discussion on chromium-dev mailing list, and also had direct e-mail exchange with developers doing the WebKit and libjpeg-turbo changes. The end result is we'd either have to keep a local patch in Gentoo to revert the optimization, or just always depend on libjpeg-turbo.

Of course the most cool would be to do the optimization The Right Way, by doing runtime detection of JPEG extensions, as made possible by DRC, libjpeg-turbo developer and explained in bug #393471 comment 23. jcstest.c in libjpeg-turbo source tree indeed shows how to do that. The only reason I haven't done that yet in WebKit is lack of time (it's not just producing the patch; compiling WebKit and running its layout tests takes a lot of it; and then dealing with any possible breakages), but I plan to do that in the future (help is welcome).

Speaking about The Right Way, another broken thing is that libraries that have the same SONAME (in this case libjpeg and libjpeg-turbo) behave differently when using JPEG extensions. You may recognize the libpng "breakages", but it seems to me libpng upstream actively cleans up the interface of the library, and it obviously updates SONAME accordingly. That's a good thing.

Hopefully it's now more clear why the Gentoo package www-client/chromium depends on libjpeg-turbo and not virtual/jpeg.

Experimenting with SELinux

Recently I started experimenting with SELinux and created a new VM for that. I've used the excellent Gentoo SELinux Handbook and mostly got things running. I also submitted bug #411005, bug #411365, and bug #411377 to make it even better.

SELinux may seem scary or complicated at the first glance, but my experience so far was pretty good. I did encounter some AVC denials, but that was fine. Here are my general notes about running a SELinux system:

1. UNIX permissions work normally. If UNIX permissions deny access, SELinux rules are not checked. Access is just denied, as expected. SELinux rules can result in more denials, but can never allow more than UNIX permissions.
   
2. Permissive mode is really useful. In that mode the denials are only logged, not enforced. The system should work as usual, and you have chance to note and fix eventual problems. You can actually switch between permissive and enforcing on the fly (setenforce 1, setenforce 0), so you can experiment whether things would work in enforcing mode and quickly switch back otherwise.
   
3. Targeted policy is recommended for desktop systems. It's part of general advice to "start small", but desktop systems have many more available applications anyway. Usually servers are minimal installations, so it's easier to write complete strict policy for them. Targeted SELinux policy uses unconfined_t context by default, which has no restrictions at all. Only selected applications have a specific policy just for them.
   
4. You don't need to worry about SELinux users with targeted policy. Another part that makes SELinux more complicated is that normal Linux users are mapped to SELinux users. This is important when using strict policy, but with the targeted one everyone is just mapped to unconfined_u.
   
5. Most access decisions are made based just on the type. Usual SELinux context, e.g. unconfined_u:object_r:user_home_t may seem quite complicated. However, corresponding allow rule would most likely only need to mention the type, i.e. user_home_t (to be precise, two types: types of the object and type of the subject).
   
6. Many confusions are just matter of vocabulary. For example, there is no practical difference between domain and type (source). My understanding is that domains are types used to confine processes, so every domain is a type, but not every type is a domain.
   
7. Allow rules are actually pretty simple. For example "allow example_t self:process execmem;". Ignore the syntax, the meaning of each rule is really straightforward, and I think it's no different with alternative MAC (Mandatory Access Control) implementations. And when writing a policy for specific domain (in this case example_t), all of the rules are going to start with "allow example_t ...".

For further reading, I recommend SELinux from the inside out and Breaking the Ice with SELinux.

Some bugs detected during arch testing

Just wanted to share some list of bugs I've recently hit during arch testing / stabilization. Especially useful for that is testing of reverse dependencies.

bug #387789 - app-office/passepartout-0.6_p1 failed (compile phase)
bug #390805 - dev-embedded/ftdi_eeprom-0.2 fails to compile
bug #397513 - dev-util/netbeans-6.8-r1 fails to compile (JavacTask.java:107: package CheckForCleanBuilds does not exist)
bug #398881 - media-libs/libxspf-1.2.0 fails to install (/usr/bin/install: cannot stat `html/*.gif': No such file or directory)
bug #398887 - dev-db/pgadmin3-1.14.1 pre-merge check failed (postgresql-config: command not found)
bug #399469 - dev-python/matplotlib-1.0.1-r1 fails to build with USE=doc
bug #403851 - dev-java/xmlunit-1.3 uses network during src_compile
bug #403859 - dev-db/slony1-1.2.10 fails to compile (error: 'SerializableSnapshot' undeclared)

And bugs that are fixed now:

bug #385403 - dev-php5/pecl-http-{1.7.0-r1,1.7.1} sandbox violation
bug #385423 - games-simulation/crashtest-1.1 fails to build (fltk-config misuse?)
bug #387785 - dev-cpp/gtksourceviewmm-2.10.1 failed (glibmm missing shared doc utilities?)
bug #390831 - media-sound/amarok-2.4.3 fails to compile (config-amarok.h: No such file or directory)
bug #394995 - emerge: InvalidDependString: USE flag 'birdstep' is not in IUSE (REQUIRED_USE contains flag not in IUSE)
bug #399513 - dev-lang/xsb-3.3.2 fails to compile
bug #399621 - media-plugins/vdr-dvd-0.3.7_pre20071113-r1 fails to compile against media-libs/libdvdnav-4.2.0
bug #401137 - dev-java/mx4j-tools-3.0.2 fails to unpack with jython

Help wanted with www-client/chromium bugs

Recently some new bugs have been filed against www-client/chromium, mostly related to HTML5 video issues. More feedback on those bugs would help fixing them faster and preventing possible regressions for Gentoo stable users.

By the way, helping with those bugs may be one step for you towards becoming a Herd Tester, and getting a cool badge on Gentoo Bugzilla like this one:

Back to the bugs then:

bug #402787 - html5 video loading progression bar issue
bug #402995 - html5 video loading but not playing
bug #402997 - html5 video sound playback issue

I have not yet tried reproducing above bugs, but the steps to reproduce are not specific enough. Any help with making the bugs more specific and reproducible is welcome, as well as reports of success (i.e. no bug + version of the browser) or failure (i.e. bug reproduced + details).

bug #403449 - doesn't release memory if sandboxed and swap is used

This is a weird bug, and may be kernel-related. I have not yet tried to reproduce, but help is very welcome there, and remember to post the version number of your kernel (uname -r). Always remember to post the version number of www-client/chromium as well.

If you like this post, please use the +1 button or one of the other ways to share. I appreciate your feedback.

Kernel 3.1.6 - watch out for "Unable to mount root fs" panic

While testing kernel 3.1.6 for bug #396469 I got the common kernel panic "not syncing: VFS: Unable to mount root fs".

It's easy to fix, here's the grub config before:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo

And fixed one:


title Gentoo Linux
root (hd0,0)
kernel /boot/vmlinuz-3.1.6-gentoo root=/dev/sda1

I had to pass an explicit root= parameter. How to figure it out? mount -l or cat /proc/mounts are not so helpful:


rootfs on / type rootfs (rw)
/dev/root on / type ext3 (rw,noatime,errors=continue,barrier=1,data=writeback)

So I used "fdisk -l" just to make sure whether it's sda or something else...

   Device Boot      Start         End      Blocks   Id  System

/dev/sda1              63    59006744    29503341   83  Linux

/dev/sda2        59006745    62910539     1951897+  82  Linux swap / Solaris

How to batch-stabilize big things like KDE 4.7.4

I've just finished stabilizing KDE 4.7.4 on x86, and it was about 300 packages, so it would be very hard to do without good tools. I have some pretty nice scripts, so it took me maybe a few supervision and everything has been done automatically (even then it can take 3-4 hours, so trying to do that manually would be a huge waste of time and take even longer).

Here's how I did that. First I created a ~/kde-stable.txt file with the following contents:


# bug #396359
=app-misc/strigi-0.7.7
=app-office/akonadi-server-1.6.2-r1
=dev-libs/shared-desktop-ontologies-0.8.1
=kde-base/activitymanager-4.7.4
=kde-base/akonadiconsole-4.7.4
=kde-base/akregator-4.7.4

... hundreds of packages omitted ...

And then:

~/arch-tools/batch-stabilize.py --arch x86 -i kde-stable.txt --repo ~/gentoo-x86/

You can do that too! Just clone my arch-tools repo, hosted by the Gentoo Overlays Team.

The tool creates a file batch-stabilize.log in the current directory (if it exists, new entries are appended to it), and the file contains all executed commands, their exit codes and output, so you can figure out what happened if something goes wrong.

It's also easy to create a list of packages in format accepted by batch-stabilize.py from the bug list. The bugzilla viewer is interactive, ncurses-based, and is simple to run:

~/arch-tools/bugzilla-viewer.py --arch x86 --repo ~/gentoo-x86/ --verbose

It's OK to omit --repo and --verbose parameters - it'll run faster and do less checks. You can also append --security parameter to only review security bugs.

I hope you like that, and enjoy either the tools, or they results - more up to date Gentoo!