A year ago, I wrote about the sad state of Python packaging. The large number of tools in the space, the emphasis on writing vague standards instead of rallying around the One True Tool, and the complicated venv-based ecosystem instead of a solution similar to node_modules. What has changed in the past year? Has anything improved, is everything the same, or are things worse than they were before?
There is an area of Python that many developers have problems with. This is an area that has seen many different solutions pop up over the years, with many different opinions, wars, and attempts to solve it. Many have complained about the packaging ecosystem and tools making their lives harder. Many beginners are confused about virtual environments. But does it have to be this way? Are the current solutions to packaging problems any good? And is the organization behind most of the packaging tools and standards part of the problem itself?
Join me on a journey through packaging in Python and elsewhere. We’ll start by describing the classic packaging stack (involving setuptools and friends), the scientific stack (with conda), and some of the modern/alternate tools, such as Pipenv, Poetry, Hatch, or PDM. We’ll also look at some examples of packaging and dependency-related workflows seen elsewhere (Node.js and .NET). We’ll also take a glimpse at a possible future (with a venv-less workflow with PDM), and see if the PyPA agrees with the vision and insights of eight thousand users.
Today’s blog post is going to contain fairly advanced Python hackery. We’ll
take two functions — one is a wrapper for the other, but also adds some
positional arguments. And we’ll change the signature displayed everywhere from
the uninformative f(new_arg, *args, **kwargs) to something more
appropriate.
In Python, virtual environments are used to isolate projects from each other (if they require different versions of the same library, for example). They let you install and manage packages without administrative privileges, and without conflicting with the system package manager. They also allow to quickly create an environment somewhere else with the same dependencies.
Virtual environments are a crucial tool for any Python developer. And at that, a very simple tool to work with.
Pipenv is a Python packaging tool that does one thing reasonably well — application dependency management. However, it is also plagued by issues, limitations and a break-neck development process. In the past, Pipenv’s promotional material was highly misleading as to its purpose and backers.
In this post, I will explore the problems with Pipenv. Was it really recommended by Python.org? Can everyone — or at least, the vast majority of people — benefit from it?
(This post has been updated in February 2020 and May 2020 to reflect the current state of Pipenv.)
As part of your code, you may be inclined to call a command to do something. But is it always a good idea? How to do it safely? What happens behind the scenes?
Gynvael Coldwind jest badaczem bezpieczeństwa pracującym w Google, który organizuje cotygodniowe livestreamy na tematy bezpieczeństwa i programowania po polsku i po angielsku). Częścią streamów są misje — w skrócie, zadania w stylu CTF-owym dotyczące inżynierii wstecznej. Wczorajsza misja była o elfickim — znaczy o Paint’cie — znaczy o programowaniu w Pythonie i jego bajtkodzie.
Setting up Python is usually simple, but there are some places where newcomers (and experienced users) need to be careful. What versions are there? What’s the difference between Python, CPython, Anaconda, PyPy? Those and many other questions may stump new developers, or people wanting to use Python.
You might get unusual errors about Unicode and inability to convert to ASCII. Programs might just crash at random. Those are often simple to fix — all you need is correct locale configuration.
To create a project that other people can use and contribute to, you need to follow a specific directory structure. Moreover, releasing a new version should be as simple and painless as possible. For my projects, I use a template that has the structure already in place, and comes with automation for almost every part of a release.