A brief introduction to the pants construction system
Pants’ build system is a polyglot tool with many great features, including full parallelism and support for remote workers. The pants have been around for several years, but since version 2.0 runs on a completely rebuilt Rust engine built for speed.
Pants is on version 2.9 at the time of this writing and now includes alpha support for Java and Scala as well as support for Python and Go. Keep reading to learn about this promising open source build tool.
There are a few cool features that make Pants a build system worth considering as an alternative to traditional picks. Here they are with a description.
- Automated dependency inference. Take a deep breath and breathe a sigh of relief: Pants can analyze your import statements and figure out what to download without a ton of metadata.
- Fine invalidation. The pants work on the smallest unit possible, so failure invalidates the minimum new work.
- Intelligent caching of results. Pants is designed to cache results based on input, so the same runs over time won’t require re-doing the same work.
- Concurrent execution. Pants is written in Rust and designed to take full advantage of multi-core machines, allowing parallelism in your builds.
- Remote workers. Pants can be configured on remote construction machines. Combined with caching and concurrency, Pants allows teams to reuse build results from each other and leverage high-performance execution.
- Polyglot. Multiple languages and platforms are supported through a unified interface.
- Plugin API. An advanced plug-in system allows plug-ins to use all of these same features.
Dependency management in Pants
Traditional build tools like Maven or Bazel require you to maintain a set of dependencies, listed in an external file. Pants offers to change this by analyzing your source files and inferring them.
As you can imagine, doing this on wildly different platforms (Python vs. Java, for example) is no easy task. See this description of the effort in the JVM world.
In some cases, you will still need to provide hints to the system where to find external dependencies for Java. As the team notes when adding alpha support:
[…] third-party dependencies are often in JAR files that Pants won’t even download until he’s sure he needs them. Sometimes the artifact’s group name can give us a clue — an artifact in the com.google.truth group might very well contain com.google.truth class files — but the rest of the time we need more of data. You can provide a list of JVM packages provided by a Maven artifact when defining your dependencies, and for other artifacts we use an internal database of packages at unversioned Maven coordinates.
So you’ll still have some degree of manual dependency tracking, but less than usual, and once the system figures out how to resolve a given dependency at its remote location, Pants will use that information across all your projects within the repository. . You won’t have to repeat this information like you do with Maven.
In Python’s case, you’ll similarly create a central file to map the “universe” of dependencies used by the repository, but Pants knows which dependencies are actually used. (This contrasts with the blanket approach used by other tools.)
As Pants’ docs state, “Pants understands exactly what dependencies each file in your project needs and effectively uses only that subset of dependencies needed for the job.”
In Pants, you define requirement mappings in a file, which for Python looks like Listing 1.
Listing 1. python_requirements file for pants
There are a whole range of options for fine-tuning the dependency management engine. You will find the details here.
Similar to other command line tools, Pants uses the concept of goals as the main commands when running. For example, you can run the simple helper goal by running
Objectives can have sub-objectives (
./pants help goals), can be configured with different options, and can also be chained, as with
./pants fmt lint ::which runs the format goal, followed by the lint goal, configured with
::, which means all files recursively. (You can find more information about file and directory syntax for Pants command line here.)
Pants also supports the ability to run in a “loop” mode, where the engine will watch for changes and re-execute commands in sequence. For example:
./pants --loop check test project/app_test.py
Pants BUILD files
Metadata for your Pants builds will be defined in BUILD files. One of the main things these files define is targets, which are named blocks of information to apply to a goal type. Each target resides in a directory, and the path plus name provide a unique ID to address that target when configuring other targets or setting targets on the command line.
Backs of pants
In the pants, the various abilities that lead to the goals are known as backends. You enable backends on your project in the global pants.toml configuration file. You can see how to enable some Python-related features in Listing 2.
Listing 2. Enabling Pants backends
backend_packages = [
You can see that backend capabilities are hierarchical namespaces. Full list of available backends is seen here.
Source roots pants
Another important aspect that you will need to configure is the source roots in your projects. This is also done in the global pants.toml file. You can see a simple example of identifying Python and Java roots in Listing 3.
Listing 3. Source Roots for Python
root_patterns = [
Project introspection in the pants
Pants includes quite powerful introspection capabilities to gain insights into your plans.
For example, you can find all targets in your projects by running the command
list goal. You can identify all dependencies used by a target with the
dependencies target, or find what files a target is using with
Of course the
count-loc goal is still handy for counting total lines of code and estimating complexity. Goals like these also support fine-tuning of files and directories to mine by passing the file and dir arguments.
Pants and continuous integration
the Papers on pants include quite a bit of information on how best to use the system with CI (continuous integration). You can leverage many of the inherent benefits of the Pants system (parallelism, speed, and caching) for your CI setup, so it’s worth considering these options.
Pants and remote work
There are two aspects to remote working in Pants:
- Remote caching. Pants stores the results of the local process execution in a remote cache and also consumes the results from this remote cache.
- Remote execution. Pants offloads process execution to a remote server (and consumes cached results from that remote server).
Local Pants clients on your development machines then interact with the remote server to take advantage of these features. For large teams, these are potentially huge gains in developer productivity.
Pants language support
The ultimate goal of Pants is to provide a single build experience for all your programming languages. That would make it the ultimate monorepo build system, where you could just throw anything at the tool and it would figure it out.
As of this writing, Pants has Tier 1 support for the following:
- Python – Since version 2.0
- Protobuf and gRPC – Since version 2.2
- Shell – Since version 2.5
- Go – Since version 2.8
- Support for Java – Alpha from version 2.9
- Scala – Alpha support from version 2.9
Pants is an ambitious project. If it continues to add languages, it could offer an incredibly new and easier way to create projects, especially for large projects using multiple languages.
Copyright © 2022 IDG Communications, Inc.