I’ve been wanting to publish my favorite Python tools annually for a long time now. It’s practical for a lot of people, and it allows to see how they evolve over time and the projects I contribute to.
So here’s the first edition of this list.
Development tools
poetry
Poetry is my package manager of choice.
ruff
Ruff is a linter and code formatter. It runs at lightning speed and contains an impressive number of features found in flake8 plugins. I don’t know anyone who, after trying ruff, preferred to stick with another linter.
For me, in 2023, it replaced flake8 and many flake8 plugins. As for black, the integration is not quite to my taste yet so I will hold until I switch.
My current ruff configuration:
select = ["B", "D", "E", "F", "I", "T20", "W", "N", "YTT", "ASYNC", "S", "A", "C4", "DTZ", "T10", "PIE", "PT", "RSE", "SLF", "SIM", "TCH", "PTH", "PGH", "PLC", "RUF", "RET"]
# D1 - Don't enforce docstring, everywhere, all the time.
# D401 - Don't enforce imperative mood. (Actually fine with me, but to strict to some colleagues.)
# D413 - I don't get why blank lines at the end of Docstrings are a big deal.
# PTH123 - Use Pathlib.Path.open instead of open directly. We're fine with open as it is.
# RUF001 Can fix false positives like "L'individu" -> "Lindividu" (French word)
ignore = ["D1", "D401", "D413", "PTH123", "RUF001"]
# F401 - Strips unused imports. Quite annoying when your file editor runs the fix command on every save.
# F841 - Strips unused variables. Same excuse as for F401.
# PIE804 - Replaces kwargs with actual arguments. Annoying when working with pydantic and wanting to test the kwargs interface.
unfixable = ["F401", "F841", "PIE804"]
pydocstyle = {convention = "google"}
black
A code formatter. Very few settings to keep team discussions and debates to a minimum.
mypy
A static type checker for python. It’s hard to believe that there was a time when complex python applications were coded without such a tool. Today, it’s an essential. Why mypy vs Pyre or pyright? I don’t have a good reason. I started with mypy, it met my needs and I’ve never tried or compared the alternatives.
pytest
pytest is a library for testing python applications. Yes, python comes with the unittest module, but pytest has made a name for itself with its advanced features and plugins. I’ve been using it for years now. I haven’t heard of a strong rival to replace it, nor do I feel the need to.
hypothesis
It’s a bit of a lie for me to claim that hypothesis is in my python toolbox. It’s a great tool, a way of making tests much more robust, but I’m nowhere near using it as much as I should. I simply haven’t mastered it enough to use it effectively. That’s why I’m putting it on the list anyway.
bandit
Bandit is a tool that statically scans code for potential sources of security problems. It is technically integrated into ruff, but I’m not yet sure whether it replaces it completely or whether bandit updates make their way into ruff quickly enough or not. Since I have doubt, I’m keeping both. When we’re talking about security, I’d rather not take the risk.
Production Tools
structlog
Structlog is a logging library. I use it mainly for structured logging. Once correctly configured, locally it prints in the console with colors. In the cloud, I configure it to send logs in JSON, which is very useful for searching with tools like ElasticSearch or AWS Log Insights.
pydantic and pydantic-settings
A good development practice is to validate all incoming data. Pydantic enables you to do this at various levels, and pydantic-settings does it for configuration loading. The latter supports configuration files, .ini, .env, environment variables and more, making it very flexible.
On the Talk Python To Me podcast, I recently heard about msgspec, which is even faster than pydantic. (link to episode) I haven’t tried or compared it to see if it can really replace it or if it’s made for other use cases. To be continued.
fastapi
FastAPI is a web application framework. It supports asyncio, performs validation of incoming data with pydantic and uses these same validation models to produce an OpenAPI specification. For me, it has replaced flask.
sentry-sdk
This item betrays which error handler I use the most. It’s on this year’s list, but may disappear by next year. I work a lot with AWS Lambdas and sentry-sdk takes a long time to initialize: 1 second for some of our production functions. sentry-sdk automatically detects the libraries used, so if it discovers FastAPI in the environment, even if it’s not used, it takes another 200ms to initialize. If you’re familiar with the concept of “cold start”, you’ll understand that this is unacceptable for serving responses quickly in an API.
sqlalchemy and alembic
sqlalchemy is a library of tools for interacting with databases. It also lets you map python models to database tables.
Alembic lets you create different versions of the database. It can then be used to perform updates or roll back if things don’t go as planned.
If you have any must-have tools to suggest, please leave them in the comments.