PyDev of the Week: Benjy Weinberger

This week we welcome Benjy Weinberger (@benjy) as our PyDev of the Week! Benjy works on the Pants build system. He is also the co-founder of Toolchain.

You can catch up with Benjy over on GitHub. Let’s take a few moments to get to know Benjy!

Can you tell us a little about yourself (hobbies, education, etc):

My name is Benjy Weinberger, I was born in the UK, grew up in Israel, and have lived in the US since 2002. I studied undergraduate and graduate Maths, Physics and Computer Science at the Hebrew University in Jerusalem. I’ve been a software engineer since 1997, working at companies such as Check Point, Google, Twitter and Foursquare. I’m now the CEO and co-founder of Toolchain, where we’re creating a distributed build system. When I’m not working I love running, SCUBA diving and live performances. In fact I’m on the board of LINES Ballet, a contemporary dance company in San Francisco.

Why did you start using Python?

I joined Google almost 20 years ago, and they were using Python as their primary production scripting language. So I had to learn it in order to deploy my changes. It then quickly replaced C++ and Java in my personal projects, and I’ve never looked back!

What other programming languages do you know and which is your favorite?

C/C++, Java, Scala and some Rust. I love Python for its well-chosen trade-offs between power and simplicity, and it’s my go-to language for most uses. But Rust is intriguing, and can interoperate with Python effectively, so I would love to do more with it.

What projects are you working on now?

As CEO I don’t do engineering work as much any more. But when I do get some coding time, I’m usually contributing to Pants, an ergonomic open-source build system that emphasizes first-class support for Python, and is the client for Toolchain’s distributed build solution.

Which Python libraries are your favorite (core or 3rd party)?

Among the standard libraries, I think dataclasses really pulls its weight, providing a ton of useful functionality via a thoughtful, succinct API. And I have a lot of love for textwrap. It takes care of a small but fiddly problem in a robust way, and I find myself using it more often than you’d think.

Some 3rd party libraries I really enjoy using include the venerable Django, which does so much heavy lifting when building webapps, and Click, which is a neat, well-designed framework for creating command-line applications.

What made you decide to help create the Pants open-source build system?

Historically, to have a really great build experience, you had to work at one of a handful of giant companies, such as Google and Microsoft, that could afford to staff a large developer productivity team. After I left Google I was disappointed with the quality of generally available build systems and practices and wanted to bring that high-quality experience to my new team. Pants was initially an internal experiment at Twitter. But we soon realized that creating yet another org-specific solution in a vacuum was not the best way forward. So we turned Pants into a fully-fledged OSS project, with Foursquare and others involved, which gave us a much wider perspective.

Some of those giant-company systems are now open source, but they are each designed for the specific practices of those organizations, and tend not to generalize well to the needs of diverse organizations of different sizes. We designed and built Pants to support the real-world use cases of a wide variety of small, medium and large engineering teams. In fact, last October we launched “Pants v2”, a complete redesign and reimplementation of the system based on lessons learned from the past decade of working on previous iterations.

Can you tell us why you would use Pants over another build system?

There are several reasons, all of which factored into the design of Pants v2:

• Pants has a strong focus on real-world Python use cases, for which we provide first class support. In fact Pants itself is written in Python and Rust! For example, Pants has out-of-the-box support for running pip, setuptools, pytest, mypy, black, pylint, isort, bandit, flake8, docformatter, protoc, pex, AWS lambda and more. Other systems were designed for C++ and/or Java, with Python support often tacked on as an afterthought, or via third-party plugins.
• Pants is more ergonomic and user-friendly than most systems. For example, users think in terms of files and directories, so Pants adapts to that instead of imposing new conceptual models that the user has to master in order to get anything done.
• Pants natively understands the semantics of a variety of build actions such as “test”, “lint”, “format”, “typecheck”, “package” and so on. And you can even add your own custom actions. Other tools may only understand a generic “build” action and cannot reason about semantics such as “linters may run concurrently but formatters must not”.
• Pants requires a lot less metadata and BUILD file boilerplate than other tools, thanks to dependency inference, sensible defaults and auto-generation.
• The Pants execution engine carefully isolates side effects for accuracy and performance. There is no global state that has to be synchronized on.
• Pants is easy to extend, with a powerful plugin API that uses idiomatic Python 3 async code, so you can use natural control flow and powerful language features (including type hints) in your plugins. Other systems often confine you to a limited, brittle API that does not have the full power of Python.
• Pants has been an OSS-first project for over a decade, with a governance model that allows any individual to participate and any org to play an equal role.

Pants is state-of-the-art, especially for Python. I recommend giving it a whirl, and we’re happy to answer questions and debug issues at https://www.pantsbuild.org/docs/getting-help .

Is there anything else you’d like to say?

It’s been wonderful to see the evolution of Python in the last decade or so from “scripting language used around the edge of ‘real code'” to a full-fledged workhorse that is a practical and sensible solution to a wide range of challenges, from webapps to data science to devops and more. And a lot of this is thanks to the inclusive and dedicated community that has made Python what it is today. I’m excited for the future of the language!

Thanks for doing the interview, Benjy!