Title: Tech WTF: Updating on pip and Cargo Date: 2018-02-04 Modified: 2018-06-24 Category: Blog Slug: tech-wtf-updating-on-pip-and-cargo Tags: tech-wtf Summary: Apparently updating things on pip and Cargo is Nintendo Hard
EDIT 2: Having investigated even further, it seems like Pip is even less awful
than I suspected, although it would be awesome if its
--help option for
specific commands was better-documented. Furthermore,
actually has an
--all option! Yet another rewrite! The (rather
hyperbolic) title will remain, but never let it be said I don't admit my
mistakes (four months later).
EDIT: Having investigated all these things further, they're not quite as bad as I'd thought. In particular, Pip is a bit friendlier than I'd assumed, and both Pip and Cargo have various helpers that can make this easier. Thus, I've rewritten everything to take this into account. Also, to be constructive, I've decided to add some helper scripts of my own for people to use to make their own interactions with these tools easier.
I recently switched to Gentoo. There are a bunch of reasons for this, including, but not limited to:
Thanks to Gentoo using OpenRC and having one of the best communities ever, I don't have these problems anymore, and will continue migrating all my devices to Gentoo over time. As part of that, I finally decided to properly manage my non-system packages. I use a bunch of these, including, but not limited to:
These are written in several languages: Rust, Python and Ruby, to be exact. Each
of these is managed using a different tool:
cargo for Rust,
gem for Ruby. Overall, installing stuff with all of them is
pretty straightforward, and tends to go without issue.
However, when it comes to updating stuff you've installed with these, only
really makes it convenient; you just go
and you're golden.
cargo, however, make this needlessly harder.
This post will explain how to get simple, one-command updates (or as close to
that as possible) for both
After spending some time understanding that
pip install --help is where you
should start looking, you'll find the following useful option:
-U, --upgrade Upgrade all specified packages to the newest available version. The handling of dependencies depends on the upgrade-strategy used.
Unfortunately, you still need to tell
pip precisely which packages to update. If
you have only a few, you can probably just memorize or script them, but this shouldn't be
necessary in this day and age. Luckily, there is a package that can solve this
pip-autoremove. In particular, we're most interested in its
-L, --leaves list leaves (packages which are not used by any others).
This means that if you use
pip for managing executables, you can now use
pip-autoremove -L to find them, since they typically aren't dependencies.
pip-autoremove doesn't take into account whether the Python
package is installed by your package manager or
pip, so you'll end up getting a
list of both. Once again,
grep to the rescue:
[koz@Sebastian ~]$ pip-autoremove -L | grep '\.local'
This will only show those packages which are installed locally. We can
then combine this with an incantation of
pip install --upgrade --user to update everything in one shot:
pip install --user --upgrade $(pip-autoremove -L | grep '\.local' | cut -d' ' -f1 | xargs)
The use of
cut above limits the output to just the name of the package we're
interested in, while
xargs packs it nicely into a horizontal list, suitable
for feeding to
pip install. This is something you can stuff into your
.bashrc, and then call at your leisure.
cargo is not as elaborate as
pip, although it's far from
straightforward there either. Firstly, you're going to need
cargo-update installed. Then, you have to do the only semi-obvious:
cargo install-update --all
which will do exactly what you need. While you could write a function like for Python above, I find it's not really needed.
A lot of people are going to start yelling something to the tune of "Pip and
Cargo aren't package managers!" right about now as a justification for not
including this functionality directly. I don't buy this excuse for
even one minute. Many Python-based and Rust-based executables give you
no other choice to install them - if your distro doesn't package them, then
you've got no other options. For at least
proselint, I know my distro
doesn't, and I dare you to try and find a distro that packages Tectonic!
Whether you like it or not,
cargo are used as package
managers, and thus, easy updates are a requirement. Having to install other
packages just to make this reasonably smooth is not a reasonable thing to expect
on this basis.
Furthermore, these suggestions don't come up in searches - the edit history of
this page should show you that even for someone reasonably literate, this isn't
an easy thing to figure out! Why
aren't included in
cargo by default is strange in itself, but
finding the information necessary to install and use them is also needlessly
hard in my opinion.
cargo need to incorporate the ability to do one-command updates,
gem currently does:
[koz@Sebastian ~]$ gem update Updating installed gems Nothing to update
This doesn't appear hard relative what
cargo can already do.
Not having this built-in and easily-findable in the documentation for either of
them is quite sad in my opinion, and should be fixed (but probably won't be).