PyDev of the Week: Pradeep Kumar Srinivasan

This week, we welcome Pradeep Kumar Srinivasan as our PyDev of the Week! Pradeep works on the Pyre package, which is a performant type-checker for Python as well as the Pysa security tool.

You can sometimes find Pradeep at PyCon giving talks and interacting with the community.

Let’s spend some time getting to know Pradeep better!

Can you tell us a little about yourself (hobbies, education, etc):

From childhood, I have been obsessed with one question: what is the best way to think? Accordingly, I devour books on rationality, cognitive biases, and automated reasoning. My college blog was called “Road to Rationality”, and I’m still trying to travel, hopefully faster, on that long road.

My hobbies include playing badminton, going on long bike rides, and learning about the tectonic plates of economics and culture that shift surface-level phenomena in the world. I’m also a huge fan of P. G. Wodehouse and have spent many a pleasant day poring through his prose and marveling at his similes.

I did my Bachelor’s in India and Master’s in the U.S., both in Computer Science, and moved to the Bay Area for work.

Why did you start using Python?

I first came across Python as an undergrad. Until then, programming for me had been synonymous with C/C++, which I had found pretty painful. When I was doing some course projects, though, I found that I had the option of using Python. I couldn’t believe the professors were letting us get away with using a language as fun as Python for serious assignments. No manual memory management, no pesky pointers, and, what seemed to be, in comparison to C/C++, no rules at all! It seemed almost too good to be true. Even afterwards, I somehow still thought that, while I could use Python for my own hobby projects, professional projects had to be in serious languages such as C++ or Java.

That finally changed when I spent a summer on the Django web server for my university’s massive, student-run technical festival (called Shaastra), where we gave the organizers tools to track attendees and resources. That’s when I dropped the “serious vs fun languages” dichotomy. Just as with Python, I was struck by how clean and pleasant Django was. It was a great education in how a program designer could apply the “Don’t Repeat Yourself” principle beyond just de-duplicating identical lines of code. More generally, it showed me that you don’t need to sacrifice elegance for practicality.

What other programming languages do you know and which is your favorite?

I followed Eric S. Raymond’s advice and learned Lisp as my second language. It really was an enlightening experience to solve the exercise problems from Structure and Interpretation of Computer Programs and start to get The Way of functional programming. I still hack on Emacs Lisp at home for personal productivity.

Over time, though, I’ve been seduced by the wonders of static typechecking. (Very roughly, “static typechecking” means getting so-called “compilation errors” up front, just as we do in C++ or Java, even in a dynamic language such as Python.) These days, I can’t even imagine confidently changing a non-trivial program without typechecking.

I love Haskell for its purity and expressiveness, though I find the compilation time and dependency hell frustrating. When working on the Pyre typechecker, I’ve been using OCaml, so that’s the functional language with which I have the most professional experience, and I have enjoyed how it lets you express complicated code without sacrificing speed or type-safety, especially using modules and module functors. I have also spent a few months writing Rust code on the Mozilla Servo project.

When it comes to everyday coding, though, I think it’s hard to beat Python. It has pretty much every library I want and it just works. Python’s biggest superpower may, weirdly enough, be its syntax. In almost every other language, I miss the significant indentation (and, thus, the lack of ugly braces) as well as the sweet syntax for comprehensions/decorators/with-statements. Keyword-arguments, for example, are prosaic and easy to underestimate, but I miss them sorely when I need to modify a years-old PHP or JavaScript function call and can’t puzzle out which of the arguments correspond to which parameters. (I’ve seen people try to work around this by using comments to indicate the keyword-argument name in the function call; that’s just a bug waiting to happen when the function changes.) The more the years go by, the more I appreciate how, despite not being the most expressive language around, Python stays competitive with its syntax, ecosystem, and pleasantness.

Which Python libraries are your favorite (core or 3rd party)?

I’ve found libcst to be indispensable for quickly analyzing or modifying Python code. For example, I’ve used it to collect detailed stats on how people are using type annotations in the wild. I couldn’t have done that so easily without libcst.

Other favorite libraries are pytest for testing, itertools/more_itertools for common transformations, and Beautiful Soup for extracting information from web pages in my personal scripts.

How did you get involved with the Pyre project?

I joined Meta (which was then Facebook) after grad school, following a summer internship there. I had done a lot of projects on static analysis and I was looking specifically for language-level teams. I spoke to a few teams and found Pyre, which was a great fit for me, as I got to do work I like (static analysis) on a fun language (Python) using another fun language (OCaml).

One great benefit of having a large internal user base was that I got to field hundreds of questions and bug reports. This made me dive into Python’s docs and PEPs to understand both its runtime behavior and, of course, its static type system. There’s nothing quite like an urgent bug report to motivate you to discover the obscure clause of some old PEP that specifies the correct behavior. I also got to see exactly how existing features were being used in our code base, where the most pressing gaps were, and thus which new Pyre features would most help our users.

Another benefit was getting to work closely with the open-source Python typing community and actually contributing back to the typing standard. Thanks to the encouragement of the Pyre team members, the typing community, and my collaborators, I was able to contribute PEP 646, PEP 673, and PEP 675. I was also able to give talks about my work at PyCon 2022 and PyCon 2023, as well as a PyCon Tutorial on “Python Types for Fun and Profit” (with Pyre teammates Shannon Zhu and Jia Chen).

As Pyre has matured, I’ve started branching out beyond catching “compilation errors” (aka type errors), and have been trying to use the typechecker as a platform for other analyses, including privacy enforcement and code-navigation (such as go-to-definition).

What projects are you working on now?

As part of using the typechecker as a platform, I’ve been working on a feature in Pyre called ReadOnly. You can roughly think of it as declaring a Python variable to be “deep const“. This means that not only can you not mutate any of an object’s direct attributes, you cannot mutate any of the attributes’ attributes or call any methods that may mutate the object. Using this feature, you can guarantee that code will not write data to places where it shouldn’t. For example, if you have a function that accesses sensitive data and you don’t want to leak that sensitive data to any of the function’s parameters by mistake, you can mark the parameters as ReadOnly so that they can never be written to.

Besides programming, I also run a monthly Typing Meetup. When working on some of my PEPs, I saw that there were many ideas that excited people but that no one was actively working on, or that were getting bogged down in long mailing-list threads. Often, one email criticizing minor design choices could derail the entire discussion. So, in 2021, I started a meetup where members of the typing community – some Python core developers (Guido van Rossum, Jelle Zijlstra), representatives from Mypy, Pyright, Pyre, etc. – all come together to discuss cutting-edge features. I’ve invited dozens of speakers on various topics and we’ve had some spirited but civil discussions. By having most of the stakeholders present, it has been easier to either get consensus or realize that something is not feasible in its current form. Some of the PEPs that have emerged after discussions are PEP 695 by Eric Traut for type variable syntax, PEP 692 for more precise **kwargs types by Franek Magiera (whom you have interviewed), and PEP 677 for nicer Callable syntax by Steven Troxler and me (though many of us were disappointed to see this one rejected :|).

What are your top three reasons for using Pyre over Mypy?

Ah, now we’re getting controversial! My personal reasons are:

• Analysis beyond typechecking: Pyre is not just for typechecking. On top of Pyre, Meta has built a security-focused static analysis tool for Python, called Pysa, which is used extensively to detect security issues in Instagram’s code base. Pyre also has a powerful Query API that you can use to understand your code beyond just type errors: getting the function-call graph, class hierarchy, attributes, etc.
• Speed on large code bases: This was the key reason we built Pyre and why Instagram had switched from Mypy to Pyre back in 2018, though Mypy has made quite some strides in the meantime.
• First-class support for Meta’s infra: Overall, though, Pyre’s biggest strength is its support for Meta’s infra at scale – operating seamlessly with Meta’s internal build system (Buck), customizing features for a variety of projects, handling files with custom non-Python syntax, and powering code navigation. Scaling to such large, complex projects is where it really shines.

Please note that this is not meant to diminish Mypy’s genuine strengths in the open-source world. If you’re a smallish (< 5k LOC) open-source project thinking of adopting types, you really couldn’t go wrong using any of the major type checkers (Mypy, Pyre, Pyright, etc.).

Is there anything else you’d like to say?

If people are curious about the latest typing features in Python, want to get help adopting types in their projects, or want to contribute something back, please join the Python typing community on the Typing Discourse forum. We are a friendly community and welcome contributions. We also have a dedicated Typing Summit at PyCon each year (Jelle Zijlstra and I organized it last year), so if you are at PyCon just drop by.

Thanks so much for doing the interview, Pradeep!