Monday, February 22, 2010

Gentoo Mozilla Team meeting decisions

On Feb 21st, Sunday, the Gentoo Mozilla Team had an informal meeting to discuss some of the recent changes which have a large-ish impact on users. Below is a list of them and the decisions that were taken. After the list is a description of each decision.

SQLite with Firefox: Firefox will use the bundled sqlite by default. Users can select the system-wide sqlite by setting USE=system-sqlite.

Ebuilds for Extensions in-tree: The Gentoo Mozilla team will not ship ebuilds for extensions such as noscript and weave anymore. We will only have ebuilds for extensions which are linux-specific and compiled; such as enigmail.

Firefox and Thunderbird Alphas and Betas: In addition to the overlay, we are going to start placing alphas and betas of Firefox and Thunderbird in the tree with a package.mask and a big fat ewarn during installation. Adventurous users are encouraged try them out but report bugs only in the Gentoo bugzilla. Do not go to upstream unless we ask you.

NOTE: These ebuilds will not have language packs since upstream does not release packs for these.

NSS/NSPR Changes: This involves various cleanups to the ebuild, and moving of the libraries from ${libdir}/{nss/nspr} to ${libdir} and removal of the /etc/env.d/08{nss,nspr} LDPATH entries alongwith corresponding changes in firefox-bin and thunderbird-bin launchers. These changes should be completely transparent to the user.

Revival of #gentoo-moz @ FreeNode: The mozilla team has expanded in recent times, and with that we have decided to revive the ages-old #gentoo-moz irc channel. Users are welcome to idle, discuss, and ask for help on that channel. Come on over!


Why we made these decisions:

SQLite with Firefox: We've done flip-flops on using the system-installed sqlite for for XULRunner and Firefox. Initially, we used the internal one, but folks reported bugs about that (we prefer not to use bundled libraries), so we switched to the system sqlite.

Then with the next version of Firefox, people started reporting major bugs with using the system sqlite and we temporarily disabled it. Once the problems were resolved, we added USE=+sqlite to use the system-installed sqlite by default.

Recently, another issue cropped up: upstream mozilla was getting bug reports that the sqlite db was insecure, and trivial tools like grep could be used to get (private) deleted information from it. They traced that to distros using the system sqlite which didn't have support for SQLITE_SECURE_DELETE. They then made that a mandatory configure check with 3.6 (and we added a dep on the system sqlite for it). Soon, we started getting bug reports from people who did not want that enabled system-wide (since it zeroes out the data when deleting it, which has a performance penalty). Upstream made it clear that they would not make the check optional.

In the end, we decided to make firefox use the internal sqlite by default, and allow the user to select the system installed sqlite via USE=sytem-sqlite if they are ok with secure-delete being on system-wide.

Ebuilds for Extensions in-tree: We found that a number of extensions were being released at a very swift rate which meant two things:

  1. Bumping ebuilds was very tedious, which led to them becoming stale very quickly
  2. Judging whether an ebuild can go stable was not possible, and most often the extension developers just want the users to use the latest release.
This offset the benefit of users being able to install extensions system-wide and have it managed by portage. Users can still manually install and manage system-wide extensions if they so wish. They are also free to copy the old ebuilds to their local overlays and use them if they wish.

Firefox and Thunderbird Alphas and Betas: Mozilla Upstream has been complaining that their betas and release candidates don't get much testing on Linux since all the distros ship only the final releases. This means that bugs are caught very late, and aren't fixed till the next major version. To help with this, we've decided to revise our release strategy and give a bit more visibility to our alphas and betas (which are generally kept in the overlay) by pushing them to the tree under package.mask, and with large ewarns all over the place. Users are strongly advised NOT to report bugs with these directly to upstream. Please use the Gentoo Bugzilla.

NSS/NSPR Changes: the libraries for nss and nspr were installed in a prefix due to collisions with other packages (libssl.a for instance). Recently, Google decided to use portage for cross-compiling and managing ChromeOS. This combined with some bugs reported with NSS/NSPR led to some investigation, and a voluntary review of the ebuilds.

This led to a number of changes which included disabling static libraries for NSS and NSPR, and moving the rest to ${libdir}. Further details on the changes and their "why" are listed on the review bugs linked above.

10 comments:

Anonymous said...

Many thanks for that very interesting post! It's nice to know the rationales behind the changes.

Anonymous said...

Some stuff that comes up in the ebuild review seems wrong. For example:

>
> * --with-pthreads: It's not necessary to specify --with-pthreads
> because using pthreads is the default. It should be harmless
> to specify --with-pthreads, but I would delete the option.

> I just wanted to make sure you do NOT specify --enable-ipv6. That
> option is not necessary for Linux. On Linux NSPR will automatically
> detect at run time if IPv6 can be used.

According to Gentoo guidelines, "Packages should not configure and link based
upon what is available at compile time — any autodetection must be overridden."

http://devmanual.gentoo.org/general-concepts/use-flags/index.html

So it's wrong to have the package detect the presence of pthreads and ipv6 (or
anything else for that matter.) This should be controlled by USE flags.

David said...

Neat. Yes, I agree -- very interesting. Thanks for posting the explanations.

Anonymous said...

I guess that's why I now get
"The application has been updated, but your version of SQLite is too old and the application cannot run." instead of my Firefox?
[ebuild R ] dev-db/sqlite-3.6.22-r2 USE="-debug -doc -extensions fts3 -icu readline secure-delete -soundex -tcl -test threadsafe" 0 kB
[ebuild R ] www-client/mozilla-firefox-3.6-r2 USE="alsa -bindist custom-optimization dbus gnome -java libnotify startup-notification -wifi" LINGUAS="-af -ar -as -be -bg -bn -bn_BD -bn_IN -ca -cs -cy -da -de -el en en_GB -en_US -eo -es -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi fr -fy -fy_NL -ga -ga_IE -gl -gu -gu_IN -he -hi -hi_IN -hr -hu -id -is -it -ja -ka -kk -kn -ko -ku -lt -lv -mk -ml -mr -nb -nb_NO nl -nn -nn_NO -oc -or -pa -pa_IN -pl -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -sq -sr -sv -sv_SE -ta -ta_LK -te -th -tr -uk -vi -zh_CN -zh_TW" 0 kB

Now, how do I start Firefox ?

Nirbheek said...

@Anon with sqlite problem: See http://bugs.gentoo.org/show_bug.cgi?id=304913#c29

fcool said...

But up to today weave has to be compiled for each architecture that is not x86, isn't it?

Nirbheek said...

@fcool: Yes, that's why it won't be removed till WeaveCrypto has been ported to js -- https://bugzilla.mozilla.org/show_bug.cgi?id=513798

Anonymous said...

Thanks for your work

Could you please add an ebuild of this 'modification' of FF? to portage :)

disi said...

This was really confusing and I went throught all of it. First sqlite install, just for firefox. Then use flag for secure-delete and nor it's gone again :)

Nice to know how dynamic the devs react on the feedback in bug reports

Thanks!

Tim said...

@Anon asking about Swiftweasel. The difference that I can see is that it is built for specific architectures (Intel, AMD, etc) If you set up your make.conf correctly, portage will do exactly that. It will even strip the branding by default if you build it yourself. Swiftweasel on Gentoo is pointless.