PyDev of the Week: Reuven Lerner

This week we welcome Reuven Lerner (@reuvenmlerner) as our PyDev of the Week. Reuven is a trainer who teaches Python and data science all over the world. You can find out more on his website. Reuven also has a newsletter on becoming a better developer that you might enjoy.

Reuven also has the following resources freely available:

• His free, 6-hour “Ace Python Interviews” course
• His free, 15-hour “Python for non-programmers” course
• Weekly Python Exercise, a set of courses to improve your Python fluency
• Reuven’s YouTube Channel

Let’s take some time getting to know Reuven better!

Can you tell us a little about yourself (hobbies, education, etc):

I grew up in the Northeastern United States, and studied computer science at MIT, graduating in 1992. After working for Hewlett Packard and Time Warner, I moved to Israel in December 1995, opening my own consulting company. I had neither consulted nor run a business at that point, but I was single and optimistic, so I gave it a shot.

I’ve been in business for myself since then, pausing along the way to get a PhD in learning sciences from Northwestern University. My dissertation involved the creation of the Modeling Commons, which allows people to collaborate in the creation of agent-based models.

For years, I did a little bit of everything: I wrote software, did system administration, tuned databases, consulted with companies, and did training. About a decade ago, I realized that training was more fun and more lucrative than development — and that it was a good business practice to specialize in one thing. I’ve been a full-time Python trainer since then. Most days, I teach between 4-10 hours for companies around the world, teaching everything from “Python for non-programmers” all the way up to advanced Python workshops.

I’m married, with three children (20, 18, and 15), and live in Modi’in, a small city halfway between Tel Aviv and Jerusalem.

As for hobbies, my big obsession over the last few years has been studying Chinese. I find it fun and interesting, and also practical, given that I normally travel to China a few times each year to do corporate training there. (That has obviously been put on hold, thanks to the pandemic.)

Aside from Chinese, I read a lot, especially about current events. I also enjoy doing crosswords, and am steadily getting better at them. Everyone in my family, including me, also enjoys cooking, although I don’t often have a chance to do it as much as I’d like. And as of the start of the pandemic, I’ve been taking very long, very early walks — about 15 km/day, starting at 4 a.m. I have found it a nice, refreshing way to get out in this time of staying at home.

Why did you start using Python?

I was introduced to Python back in early 1993, when the Web was young and we were looking for languages with which we could write server-side scripts, aka “Web applications.” (I actually objected to having the term “application developer” on my business card, because I thought it was laughable that you could call what we wrote “applications.” Whoops.)

At the time, I did some Perl and some Python. At the time, Perl was more popular and had a much larger library of third-party modules. So while I knew Python and recommended it to anyone I knew who wanted to start programming, I personally used Perl for a while, continuing to use Python here and there, but not doing much with it.

I saw that Perl wasn’t doing well as a language or community, and tried to figure out in which direction I could move. I tried Python, but the Web application frameworks at the time were too weak or too weird. (I even did a big project using Zope, with its object database.) That’s when Ruby on Rails was released; because Ruby is basically Perl with Smalltalk objects, I was delighted to use the language.

But I couldn’t escape noticing that Ruby was largely trapped in the Web world, whereas Python was growing in scope and mindshare. The number of third-party packages on PyPI was growing rapidly, and when I decided to exclusively do training (rather than doing it alongside development and consulting), I found that there was far more demand for Python than for anything else.

I’ve been deeply steeped in the Python world ever since, and I couldn’t be happier.

What other programming languages do you know and which is your favorite?

I learned Lisp back in college, and I still use Emacs for editing — so I continue to have affection for Lisp as a language, and often refer to the concepts, I learned in it when working with Python.

As I wrote above, I loved working with Ruby. Everything is an object in Python, but that’s even more the case in Ruby. I loved the freedom and creativity of the Ruby world, but the object model is hard for people to grasp — and in Ruby, if you don’t eat objects for breakfast, you’ll have a hard time with it.

My research group in graduate school developed NetLogo, an agent-based modeling language. That’s a completely different way of writing code and expressing ideas, one which more developers should try.

I’m not sure if any of these would count as my favorite; I’ve now been using Python for long enough that I find that I can most easily express myself using its idioms.

I keep hoping to find time to learn Rust, because the idea of a systems language that doesn’t require me to use pointers seems really attractive, and I’ve heard such great things about it. But I keep struggling to find time to learn it.

What projects are you working on now?

I’m always doing far too many things! Here are a few of them:

• I keep publishing my “Better developers” newsletter, and the number of subscribers continues to grow rapidly. It’s an amazing feeling, being able to publish to a large, worldwide audience, and know that I’m able to help them understand Python better.
• I’m recording a bunch of new Python courses, including about (a) working with files, (b) modules and packages, and (c) decorators. I also recognize that my online store now has far too many things in it, and that it’s hard for people to figure out where to start. So I’m starting to take the initial plunge into redoing it.
• I’m working on a new book for Manning about Pandas, which I hope to write at a faster pace than the previous book — not that this will be hard!
• I’m actively updating my YouTube channel on a regular basis. I try to upload 1-2 videos a week, typically answering a question that I got from my students in class, or even from people contacting me online. I’m still learning how to use the medium, but it’s fun to be able to reach so many people with screencasts.
• I’m starting to find new ways to offer training. Obviously because of the pandemic, I’m no longer teaching in person, so it’s a good thing that I’ve gotten lots of experience (even before this year) using WebEx and Zoom to teach. However, I’m now offering all sorts of mix-and-match options to companies — including my recorded courses with live Q&A follow-up, half-day sessions, and even half-day, single-topic courses My clients are experimenting, and so am I.
• I’m setting up a company in China to handle any future training I’ll be doing there. This is big and new adventure, and will likely involve me hiring someone with whom I speak mostly in Chinese. I think that this is likely to improve my fluency quite a bit, as well as drive said employee bananas.

Which Python libraries are your favorite (core or 3rd party)?

I use Jupyter a ton in my training, and even in my day-to-day work, if I want to experiment with things. I continue to be impressed by the functionality that the core developers have put in there.

But for sheer depth, I have to say that Pandas continues to amaze me. Every time I use it, I discover not just a little bit of new functionality, but dozens of methods and options that I didn’t previously know existed. You could spend your entire career just working with Pandas, and you would probably will not know all it has to offer.

I’ve become a big fan of pytest, also. All of my “Weekly Python Exercise” courses now use pytest for checking code, and I’d say that the courses have improved dramatically as a result.

Finally, the team that developers PyPI and pip deserves everyone’s thanks and credit. I did a tiny bit of work on it at the sprints, during PyCon 2019, and I discovered that it’s a tiny group of smart and hard-working people who are supporting an infrastructure rivaling many enterprise operations.

What do you enjoy the most as a Python trainer?

I love helping people to overcome conceptual hurdles. So many people have been using Python for months or years without truly understanding the logic of the language. If I can help them break down these ideas so that they can then understand how they work and are applied across the board, it really makes me happy.

It can be something as simple as how dictionaries work, or how a “for” loop works behind the scenes. Or it can be something more complex, such as inner functions, variable scoping, or decorators. Once people understand the simple rules that we can apply in Python, consistently and repeatedly, their work is easier, and I’m satisfied that they’ll be able to do much more, in less time and with less code, than before.

Which concepts in Python are the hardest for your students to learn?

Comprehensions are extremely hard for people to understand. The syntax is already hard enough, but then understanding *when* you want to use a comprehension vs. a traditional “for” loop is really difficult. People often ask me if they should use comprehensions because they’re more efficient. They’re often surprised when I say that comprehensions aren’t about efficiency. Rather, they’re about what you’re trying to accomplish — if you’re trying to execute something multiple times (a regular loop) or if you’re trying to create a new list based on an existing iterable. The distinction is very hard at first, but it becomes clearer over time.

Decorators are a topic that people don’t get at first, but which I try to break down and explain. (I gave a talk on the subject, “Practical Decorators,” at PyCon 2019.) At first, people don’t understand how to write decorators. Then they don’t understand what to do with them. And then they’re turned on, and find all sorts of uses — and discover that they have been seeing and using decorators already, just without knowing it or understanding what they were doing.

In my advanced courses, I talk about descriptors — and it’s really hard for people to grasp. We think that we know what’s going on when we access a.b, but if b is a class attribute *and* is an instance of a class that has a __get__ method, then all sorts of magic things happen. People are confused by why we even have them, until I show them that you need descriptors in order for methods to work.

Finally, it’s hard for people — especially those coming from backgrounds in C and C++ — to understand that in Python, efficiency isn’t the most important thing. Rather, readability and maintainability are. I always tell people that Python is perfect for an age in which people are expensive, but computers are cheap. This doesn’t mean that efficiency is bad, but it’s not the thing to worry about before anything else. We put people first!

Is there anything else you’d like to say?

I’m generally impressed with the Python community, in that it’s welcoming to newcomers and patient with their questions. There are so many people learning Python, and for them it’s not a passion or the latest language on a long list — it’s something they have to do for work, and they’re a bit confused by the terminology, the ecosystem, and even the syntax. I love working with newcomers to the language, and I encourage everyone to do what they can to help the huge influx of programming immigrants (for lack of a better term), to help this all make sense to them.\

Thanks for doing the interview, Reuven!