# Pawel Hajdan jr (13 Nov 2013) # Masked for removal in 30 days. Does not have stable # API resulting in compile breakages in reverse # dependencies. Combined with short release cycle # (6 weeks) this makes it pretty much unusable as # a shared library. See bug #417879, bug #420995, # bug #471582, bug #477300, bug #484786, bug #490214. # Also, the following discussions: # - http://thread.gmane.org/gmane.linux.gentoo.devel/88222 # - http://thread.gmane.org/gmane.linux.gentoo.devel/88811 dev-lang/v8All packages depending on shared v8 library are now either bundling it or are masked.
This is obviously a quite controversial change. People are opposed to bundling libraries for good reasons. I'd like to make it clear that I'm also strongly in favor of using system libraries when possible. I'm also pragmatic though: in case of v8 this resulted in multiple bug experienced by users - just see the links above. With the API of v8 changing every 6 weeks and security fixes being pushed every now and then, these other packages depending on v8 just don't keep up.
Now the v8 team has made some nice improvements, as you can see on https://code.google.com/p/v8/wiki/Source:
V8 public API (basically the files under include/ directory) may change over time. New types/methods may be added without breaking existing functionality. When we decide that want to drop some existing class/methods, we first mark it with V8_DEPRECATED macro which will cause compile time warnings when the deprecated methods are called by the embedder. We keep deprecated method for one branch and then remove it. E.g. if v8::CpuProfiler::FindCpuProfile was plain non deprecated in 3.17 branch, marked as V8_DEPRECATED in 3.18, it may well be removed in 3.19 branch.Indeed I see V8_DEPRECATED being used in new v8 changes instead of removing APIs immediately. I heard sometimes they need to remove things immediately anyway, when some APIs are inherently buggy and lead to memory errors like leaks or double-frees.
Ideally I'd like to see a longer grace period for removal of APIs (not just one release, since it's only 6 weeks). Then maybe ABI stability in the scope of one release could become a consideration. The consistent usage of V8_DEPRECATED will for sure lead to more data about how much more effort is now needed to maintain the V8 API. If it turns out to be manageable, I hope to see these further improvements being tried.
By the way, with ffmpeg/libav we're pretty much in a very similar situation: Chromium uses bleeding-edge ffmpeg code, and other packages just can't keep up with API updates. Sometimes the APIs Chromium depends on are not part of any ffmpeg/libav release: Chromium developers actively contribute to upstream ffmpeg codebase and it's reasonable to iterate quickly instead of waiting for a release. I think even Fedora has exceptions for bundling libraries in such circumstances.
I don't expect this post to appease most people from the packaging community. This is just stating where we currently are. Maybe having some more stable layer that could be added on top of Chromium codebase in a manner similar to Ozone-Wayland would be one way. Still, that'd be a considerable engineering effort, mostly to keep old things working rather than developing the future. I think it has some value, the main question is just what to do with limited time developers have.
I'm experimentally enabling comments for this post - feel free to share your thoughts.
8 comments:
Good work.
This may be annoying, bur your reasons are sound.
This is what slotting is for. Gentoo has had it forever. Let them each link against the appropriate version.
@Joshua - yes, that sounds like a good idea at first. Other Gentoo developers also suggested it, and I was thinking whether it would fit.
There are several problems with that though:
1. Vulnerable versions of dev-lang/v8 in the tree. We should be deleting them, and that breaks reverse dependencies.
2. Slotting v8 is non-trivial (upstream is not really designed for slotting, and this is really something that should work in a compatible way across distros), and when each package on your system uses a different slot, it's not really different from bundling.
Slotting works great in cases of e.g. gtk2 vs. gtk3, or when there's a limited number of major incompatible versions like 1.x and 2.x, where minor bumps are made but major versions stay the same. v8 is different - it keeps moving forward at a great speed, changing the public API every 6 weeks. No other library that slotting was designed for works this way.
That sounds like a reasonable response, and there is nothing inherently evil about library bundling, other than some bloat, and the fact that it is not consistent with the "Gentoo way".
I have a couple of thoughts and then I'll leave you alone.
1. Bundling doesn't solve the vulnerability issues unless each bundler is individually patching their version of the lib, which then multiplies the chance of a vulnerable version.
2. Temporary solutions tend to become permanent. It might be a bigger headache in the short run, but it will be really hard to disentangle all the bundled libs when/if v8 stabilizes their API.
Thanks for your efforts!
Thanks, and actually I welcome comments - I enabled them precisely to have more discussions.
1. Correct - however, it's more tricky since e.g. nodejs argues about some v8 vulnerabilities not applying to them. One example is that the browser takes untrusted JS as input, but presumably nodejs works with trusted JS code. In that situation having a vulnerability that nefarious JS code can crash the engine and execute arbitrary machine code has different impact.
1a. Actually even if we unbundle, and the reverse dependencies just don't compile against non-vulnerable versions of v8, what would you do? Keeping vulnerable versions in the tree might as well encourage more clients to use them.
2. Yes, I share that concern. The biggest problem would be actually packages modifying their bundled copy, but for now I'm not aware of anyone doing that. v8 is an open project and it's pretty easy to contribute. The shared library configuration is also part of continuous integration system (http://build.chromium.org/p/client.v8/console and specifically http://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20shared), so at least on that end it shouldn't break.
Anyway, with limited time and "cycles" the effort needs to focus on something. For now I'd say it should be getting stable v8 API and maybe ABI at some point.
IMHO ffmpeg API changes are rarely difficult to patch. I use a live ffmpeg ebuild and maintain user patches for all the ffmpeg consumers. It've never been more than 30min work to make compatible, although I don't worry about version checks in the patches since they're only local, so I don't put them up on the Gentoo Bugzilla, I could be persuaded to do so if there is sufficient interest, I'm just spread pretty thin. Obviously, expecting upstream for the various projects to keep up is unrealistic, and it does mean a little extra work for the maintainers of ffmpeg consuming ebuilds, although a little coordination goes a long way, usually changes needed are pretty similar.
sjn, things seem to have indeed improved for ffmpeg since I wrote this post
For some Chromium-related examples, https://groups.google.com/a/chromium.org/d/msg/chromium-dev/XpM-4l8Wrx4/VeyLapOLAgAJ and https://ffmpeg.org/pipermail/ffmpeg-devel/2016-September/199590.html may be an interesting read.
Post a Comment