Dependencies management in 2019: a review

eLabFTW - Mar 3 '19 - - Dev Community

dependencies

Hello folks,

The aim of this post is to make a (mostly) complete and accurate view of the different solutions available to you when you produce code and want to manage your dependencies like a boss, for different languages. We'll see how languages have different approaches to dependency management.

I'll be writing about these 9 languages:

  • PHP
  • Python
  • Rust
  • Go
  • Javascript
  • Elixir
  • Nim
  • C#
  • Ruby

Let's start with an easy one: PHP

Composer

GitHub logo composer / composer

Dependency Manager for PHP

Composer

Dependency Management for PHP

Composer helps you declare, manage, and install dependencies of PHP projects.

See https://getcomposer.org/ for more information and documentation.

Continuous Integration

Installation / Usage

Download and install Composer by following the official instructions.

For usage, see the documentation.

Packages

Find public packages on Packagist.org.

For private package hosting take a look at Private Packagist.

Community

Follow @packagist or @seldaek on Twitter for announcements, or check the #composerphp hashtag.

For support, Stack Overflow offers a good collection of Composer related questions, or you can use the GitHub discussions.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project and its community you agree to abide by those terms.

Requirements

Latest Composer

PHP 7.2.5 or above for the latest version.

Composer 2.2 LTS (Long Term Support)

PHP versions 5.3.2 - 8.1 are still supported via the…

Composer is the de facto standard for dependencies management in PHP. It's very hard today to get anything done in PHP without it. I recommend reading this article from Théo Fidry if you want to know more about the limitations of composer and the workarounds (like using PHARs).

PEAR

GitHub logo pear / pear-core

This is the definitive source of PEAR's core files.

PEAR - The PEAR Installer

https://travis-ci.org/pear/pear-core.svg?branch=stable

What is the PEAR Installer? What is PEAR?

PEAR is the PHP Extension and Application Repository, found at http://pear.php.net.

The PEAR Installer is this software, which contains executable files and PHP code that is used to download and install PEAR code from pear.php.net.

PEAR contains useful software libraries and applications such as MDB2 (database abstraction), HTML_QuickForm (HTML forms management), PhpDocumentor (auto-documentation generator), DB_DataObject (Data Access Abstraction), and many hundreds more. Browse all available packages at http://pear.php.net, the list is constantly growing and updating to reflect improvements in the PHP language.

Warning

Do not run PEAR without installing it - if you downloaded this tarball manually, you MUST install it. Read the instructions in INSTALL prior to use.

Documentation

Documentation for PEAR can be found at http://pear.php.net/manual/. Installation documentation can be found in the INSTALL file included in this tarball.

Tests

Run the…

This is the old school player. It installs dependencies to the system. Only use it if your distribution is not shipping the extension in a package already. Note that you can use composer to install a PEAR package, see this part of phptherightway for more.

That's it for PHP. I'm pretty sure there are no other PHP dependencies management applications out there, but if I'm wrong please let me know in the comments section.

Second language: Python

Pip

GitHub logo pypa / pip

The Python package installer

pip - The Python Package Installer

PyPI PyPI - Python Version Documentation

pip is the package installer for Python. You can use pip to install packages from the Python Package Index and other indexes.

Please take a look at our documentation for how to install and use pip:

We release updates regularly, with a new version every 3 months. Find more details in our documentation:

If you find bugs, need help, or want to talk to the developers, please use our mailing lists or chat rooms:

If you want to get involved head over to GitHub to get the source code, look at our development documentation and feel free to jump on the developer mailing lists and chat rooms:

Code of Conduct

Everyone interacting in the pip project's codebases, issue trackers, chat rooms, and mailing lists is expected…

pip is the goto package manager for python, it has its drawbacks but is a very popular tool to install Python dependencies. BTW, you should Stop using sudo pip install. Use pip with moderation (and the --user flag!).

Venv

The official module for managing your project's dependencies inside a virtual environment, thus not polluting the rest of the OS.

Virtualenv

GitHub logo pypa / virtualenv

Virtual Python Environment builder




This project started in 2011 and is very useful to isolate dependencies for a project. Part of this project is now venv in the standard python library. So you can think of virtualvenv as a more feature complete venv.

Pipenv

GitHub logo pypa / pipenv

Python Development Workflow for Humans.

Pipenv: Python Development Workflow for Humans

image image CI image


Pipenv is a Python virtualenv management tool that supports a multitude of systems and nicely bridges the gaps between pip, python (using system python, pyenv or asdf) and virtualenv Linux, macOS, and Windows are all first-class citizens in pipenv.

Pipenv automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates a project Pipfile.lock, which is used to produce deterministic builds.

Pipenv is primarily meant to provide users and developers of applications with an easy method to arrive at a consistent working project environment.

The problems that Pipenv seeks to solve are multi-faceted:

  • You no longer need to use pip and virtualenv separately: they work together.
  • Managing a requirements.txt file with package hashes can be problematic. Pipenv uses Pipfile and Pipfile.lock to separate abstract dependency declarations from the last tested…

Pipenv started in early 2017 and was very quickly popular. It's my favorite, too. It works very similarly to composer with a lockfile and all the dependencies are installed in a special folder. To run the script in the environment, you need to prepend pipenv run. See my previous post about it. Note that the official Python doc is recommending pipenv for managing dependencies.

Something interesting to note, is that all of the above-mentioned projects are all hosted in the github.com/pypa namespace, the Python Package Authority, a group dedicated to maintaining the python packagers. I think it's nice to have this kind of "authority" for improving the ecosystem in Python.

The third language we'll talk about is: Rust

Cargo

Well, here it's a nobrainer, because (AFAIK) there is only one solution, and it's Cargo.

GitHub logo rust-lang / cargo

The Rust package manager

Cargo

Cargo downloads your Rust project’s dependencies and compiles your project.

To start using Cargo, learn more at The Cargo Book.

To start developing Cargo itself, read the Cargo Contributor Guide.

The Cargo binary distributed through with Rust is maintained by the Cargo team for use by the wider ecosystem For all other uses of this crate (as a binary or library) this is maintained by the Cargo team, primarily for use by Cargo and not intended for external use (except as a transitive dependency). This crate may make major changes to its APIs.

Code Status

CI

Code documentation: https://doc.rust-lang.org/nightly/nightly-rustc/cargo/

Compiling from Source

Requirements

Cargo requires the following tools and packages to build:

Other requirements:

The following are optional based on your platform and needs.

  • pkg-config — This is used to help locate…

Cargo is great. Cargo is shit. YMMV. Honestly, it's pretty cool to have one official tool for dependencies management, at least it saves you the time you would have spent choosing one! It installs dependencies per-project but can also be used for system-wide packages. Interestingly, it uses the TOML syntax for the configuration file, whereas other projects chose YAML or JSON.

Let's move on to the fourth language: Go

go get is is the way to install a dependency in Go. But the issue is that it'll be installed system-wide (in $GOPATH). Say hello to conflicts and breaking changes because you can only have one version of a lib…

This led to other approaches like git submodules or project specific $GOPATH and a lot of developers being angry at Go.

However, with Go version 1.11 (released in August 2018) there is now something called modules. Check out the wiki page to know more about Go modules. I believe this will be the future of Go dependencies management, because fuck system-wide dependencies.

What do I see? Is that the fifth language? Yep: Javascript

Aaaaahhh, javascript and the dependencies, what a wonderful world! :)

node_modules

Npm

GitHub logo npm / cli

the package manager for JavaScript

npm - a JavaScript package manager

npm version license CI - cli Benchmark Suite

Requirements

One of the following versions of Node.js must be installed to run npm:

  • 18.x.x >= 18.17.0
  • 20.5.0 or higher

Installation

npm comes bundled with node, & most third-party distributions, by default. Officially supported downloads/distributions can be found at: nodejs.org/en/download

Direct Download

You can download & install npm directly from npmjs.com using our custom install.sh script:

curl -qL https://www.npmjs.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Node Version Managers

If you're looking to manage multiple versions of Node.js &/or npm, consider using a node version manager

Usage

npm <command>
Enter fullscreen mode Exit fullscreen mode

Links & Resources

Everyone knows npm. You define your dependencies in a package.json and install them in the node_modules folder. Since version 5, there is a package-lock.json allowing you to have reproducible builds.

Yarn

GitHub logo yarnpkg / yarn

The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry

ℹ️ Important note

This repository holds the sources for Yarn 1.x (latest version at the time of this writing being 1.22). New releases (at this time the 3.2.3, although we're currently working on our next major) are tracked on the yarnpkg/berry repository, this one here being mostly kept for historical purposes and the occasional hotfix we publish to make the migration from 1.x to later releases easier.

If you hit bugs or issues with Yarn 1.x, we strongly suggest you migrate to the latest release - at this point they have been maintained longer than 1.x, and many classes of problems have already been addressed there. By using the nodeLinker setting you'll also have the choice of how you want to install your packages: node_modules like npm, symlinks like pnpm, or manifest files via Yarn PnP.


Fast, reliable, and secure dependency management

Circle Status Appveyor Status Azure Pipelines status Discord Chat Commitizen friendly


Fast: Yarn caches every package it has…




Yarn had reproducible builds well before npm. It also doesn't suffer from the sometimes poor decisions taken by npm's project leaders. It's fast, works well. It's my tool of choice.

Bower

Come on now! We said 2019 in the title! Forget about bower.

I will not talk about Parcel or Webpack as these are bundlers, not package managers.

Note that you can use either yarn or npm with the exact same package.json!

One thing to note about packages in javascript, is that it has become completely crazy, with micro libraries consisting of one line of code more or less, being used by hundred of thousands of other packages. For fun, go into a Javascript project and try this command:

ls node_modules | grep '^is*'

How fun is it to have is-obj, is-object, is-plain-obj, is-plain-object or things like isarray, but also is-arrayish… Do we really need a package is-windows when all it does is:

return process && (process.platform === 'win32' || /^(msys|cygwin)$/.test(process.env.OSTYPE)); ?

And good luck if you want to look at the code of all of these libraries for your project. That lead to the left-pad chaos.

So my advice here is to think hard before installing a javascript dependency, as it might come with hundred of these little library, and one day, one of them will become malicious (see malicious packages on npmjs.com).

You can use Depcheck to try and cleanup a bit your dependencies.

Let's talk about a more exciting language, like: Elixir

Elixir is a bit like Rust when it comes to dependency management: one official tool that gets the job done: Mix (or Hex for Erlang). Nothing more to say here, it works well enough and lets you focus on other things.

Okay so what do we have left? Nim?

Nim is a pretty new and cool language, and the name of the package manager is sooooo cute <3: Nimble. Which reminds me of this guy:

Nibbler

Nothing to see here, just use the one provided and get coding!

The next language is C#

Nuget

GitHub logo NuGet / NuGet.Client

Client Tools for NuGet - including Visual Studio extensions, command line tools, and msbuild support. (Open issues on https://github.com/nuget/home/issues)

NuGet logo


NuGet Client Tools

This repo contains the following clients:

Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Getting Started guide

For how to contribute to this repo follow the Contributing doc.

NuGet/Home repo

The NuGet/Home repo is the starting point for all things NuGet. It has the issue tracker and basic information about all things NuGet. Make sure to consult it before beginning your journey through NuGet code.

Feedback

File NuGet.Client bugs in the NuGet/Home.

License

Unless explicitly stated otherwise all files in this repository are licensed under the License in the root repository




This is the standard package manager that comes with Visual Studio. It exists since 2010 and is developed by the .NET foundation. I don't have much more to say about it because it's really windowsy and that's clearly not my platform of choice when it comes to anything vaguely related to computing.

Last but not least: Ruby

Rubygem

GitHub logo rubygems / rubygems

Library packaging and distribution for Ruby.

RubyGems Maintainability

RubyGems is a package management framework for Ruby.

A package (also known as a library) contains a set of functionality that can be invoked by a Ruby program, such as reading and parsing an XML file We call these packages "gems" and RubyGems is a tool to install, create, manage and load these packages in your Ruby environment.

RubyGems is also a client for RubyGems.org, a public repository of Gems that allows you to publish a Gem that can be shared and used by other developers. See our guide on publishing a Gem at guides.rubygems.org

Getting Started

Installing and managing a Gem is done through the gem command. To install a Gem such as Nokogiri which lets you read and parse XML in Ruby:

$ gem install nokogiri

RubyGems will download the Nokogiri Gem from RubyGems.org and install it into your Ruby environment.

Finally, inside your Ruby program…

This is the standard way to get dependencies for a ruby project. It can be seen as pip for ruby, because it installs packages system-wide. You can install them for your user with the --user-install flag (better: add gem: --user-install to ~/.gemrc to always install gems in your home).

So what about managing dependencies per-project you'll ask?

Bundler: a gem to bundle gems

GitHub logo rubygems / bundler

Manage your Ruby application's gem dependencies

Bundler is now maintained in the rubygems/rubygems repository.


Version     Inline docs Slack

Bundler: a gem to bundle gems

Bundler makes sure Ruby applications run the same code on every machine.

It does this by managing the gems that the application depends on. Given a list of gems, it can automatically download and install those gems, as well as any other gems needed by the gems that are listed. Before installing gems, it checks the versions of every gem to make sure that they are compatible, and can all be loaded at the same time. After the gems have been installed, Bundler can help you update some or all of them when new versions become available. Finally, it records the exact versions that have been installed, so that others can install the exact same gems.

Installation and usage

To install (or update to the latest version):

gem install bundler

To install a prerelease version (if…

You get a Gemfile and Gemfile.lock file, and you're sure the project will run everywhere with the same versions of gems.

But there is something interesting about the ruby ecosystem: it exists a lot more projects to switch between different ruby versions (chruby, rbenv, rvm, uru) than to install dependencies.

Conclusion

As you can see, some languages have it all figured out for you (rust, nim), while for others you need to make a choice (python, javascript), and for some you have system-wide and project-wide solutions (python, ruby). I don't quite understand why Go, a "recent" language didn't do a better job at dependency management. But this fits well with the view that "Go was created like we were still in the 70's".

Take home messages

  • Install your dependencies per project with a reproducible build (lock down versions of packages) to avoid surprises and to get something portable.

  • Try to avoid installing too many dependencies (javascript, I'm looking at you!).

  • Don't ship the dependencies with your project but ship the lockfile ;)

That's all folks, please leave a comment if you think I forgot something important or if I got something wrong ;)

Have fun coding!

. . . . . .