I like to use pythonz in this case; it’s a tool to manage Python installs, and it puts the installs in a directory under your home directory, not affecting anything in the system.

It does build each version from source, which introduces some quirks; I’ve found compilation for some Python versions works better with clang, and sometimes, you need to enable build options.

Still, I think this is a good way to do things; just start whichever Python version you want, and then create a venv with it.

other commenters have hinted at this, but the main point of most of the good advice is this: don’t use the system Python install (ie the one from apt) for development. uv is my go to, but the idea behind *conda, pyenv, asdf, etc is the same. the underlying OS shouldn’t be an issue; you should be able to ship the code between OSs and build just fine, ideally.

But I couldn’t install a specific Python version? System python is 3.13 but I needed 3.10.

The others have covered virtual environments, which is what you need if you really want a 3.10 interpreter.

But… the thing I’m here to tell you is:

they recommend virtual box to not mess with your default installation of the program and the databases it uses.

for many projects this doesn’t actually matter. You can just ignore the warnings, use the most recent version and install whatever you need.

You’re already sandboxing this stuff in a virtual box, which you should be able to reset or bootstrap again when you need to. You’re not interfering with your actual systems’ python, you’re messing with your virtual box’s system python.

I find the whole venv stuff to be very annoying, I never need it, because I use libraries that don’t interfere with system operations and I don’t downgrade to interact with projects. And even if you’re not installing “correct” versions, most of the time newer versions fix bugs and expand functionality. It is extremely rare that functions get removed and it will actually break by you not using their exact version. Or like, version conflicts.

And besides, they would need some kind of CI / testing that would check for compatibility anyway.

tldr: ignore venvs, try it bare metal, see if something breaks. If not, there you go, if yes, you can still invest the time and effort of learning venvs.

As others have said [uv] (https://github.com/astral-sh/uv) is likely a good option but since you’ve mentioned being a data scientist you might also check out [pixi] (https://prefix.dev/tools/pixi).

It’s built on top of conda so will likely have all the packages you might need.

It’s got quite a nice workflow, keeps things contained in the project directory, and adds a few conveniences over standard conda.

use something like uv to install whatever python version you want and set up a venv with it. that way you can have project-specific python versions.

Here’s my 2¢. Debian is a reasonable OS to develop on, since it provides a super stable and reasonably secure base platform. I’ve used it quite extensively for the same purpose without any issues whatsoever.

However, it won’t satisfy the needs of the modern style of development. For that, you need a reproducible development environment so that every developer gets the same results. (That means avoiding the ‘it works for me’ type of bugs). That means you need the same version of runtimes (python), same version of libraries/dependencies and same developer tools on every development system, irrespective of the distro it’s running on.

The problem with Debian is that it often ships older versions of software to keep it stable. It will likely not match the version of python and tools you need. For that matter, no distributions including the frequently updated Arch Linux are likely to meet those requirements. (There are exceptions - NixOS and Guix.) So the widely adopted solution is to create a dev environment independent of your core system, from your regular non-root account. That means separately installing a python runtime that’s different from your distro repo, etc. They don’t touch your core system and keep it clean and pristine.

The way we achieve this is by using 4 tools:

1. Runtime version manager: Manages the version of runtimes.

• Eg: python: pyenv, node: nvm, rust toolchains: rustup

2. Environment manager: Creates and maintains isolated environments where the correct runtimes, project dependencies and tools are available.

• Eg: python: venv, node: node project, rust: rust project

3. Dependency manager: Resolves and installs the correct version of all project dependencies into the environment.

• Eg: python: pip, node: npm, pnpm, rust: cargo add

4. Tool manager: Installs additional development tools for the project, like formatter, linter, etc.

• Eg: python: pip, pipx, node: npx, rust: cargo install

Often, many of these are combined and you may get less than 4 different tools. The current situation is extremely complicated and there are many different tool combinations you can use. So let’s address your specific requirements.

If your project uses only Python

In this case, the choice is pretty straightforward. Use uv.

The package management situation in the Python ecosystem was an absolute mess until UV appeared on the scene. UV combines all the 4 functions I mentioned above. It replaces venv, pyenv, pip and pipx in a single fast binary. You’ll be surprised by its speed if you’re used to the speed of pip. It’s easy to use and very well integrated. It also integrates additional functionalities like:

• Formatter (replaces black)
• Project setup (helps with poetry, flit, hatch, etc)
• Project publishing (replaces twine)

If you will use more than Python

If you will use tools or languages other than Python in a single project or in other future projects, you might want to use a ‘language-agnostic runtime and tool manager’. They can manage runtimes and tools of multiple languages. Dependencies have to be managed using language package managers (like UV, pip, cargo, npm, etc).

The most well known tool manager is asdf. Others include aqua, vfox, etc. But the upcoming star is mise. I use mise for multiple projects including for Python projects. Mise uses UV behind the scenes for Python. So, mise projects play well with UV projects and with others who use UV.

The only disadvantage with multitool managers like asdf and mise is that they tend to be more complex compared to single language tools like UV. They obviously handle more and provide more features. However, investing time in tools like mise pays in the long run when you’re handling multiple languages.

Servers like databases

The tools I mentioned above don’t handle daemons like postgresql, redis, etc. This is why your colleagues recommend VMs. I will talk about VMs in a while. But I want to show you some simpler solutions here.

Another commenter has already mentioned the use of docker-compose files to set up such servers. It’s the easiest solution possible.

Another more refined solution specifically for development containers is testcontainers. It’s essentially the same as the docker compose solution, but with more dials and switches to help with automated tests like unit tests during CI. You’ll have to learn a bit more than docker compose, to use it. However, those test servers are also readily available online and require little configuration.

Do you need virtualbox?

The methods explained above don’t ruin your base Debian install. So a dedicated VM is not really required. However, I’m leaving this information here for completeness.

Use of VMs was widespread in the past. But they didn’t run virtualbox, VMware or Qemu directly. Instead, a CLI frontend tool was used to set up those VMs for development. The most common tool was Hashicorp’s Vagrant. Another tool available today is Lima. These tools mount your project directory into the VM, set up its network, install required tools, start required services (like DBs), attach a shell for you to work on, etc. These VMs are complete development environments and you don’t need to do anything on the host system other than starting them up.

Since the advent of containers, the same idea has been implemented using containers instead of VMs. These are obviously less resource intensive than VMs. Most of them follow the devcontainers standard. So a devcontainer configuration works on multiple platforms, including GitHub’s famous codespaces. Local tools for it include devpod, ona, devbox and devenv.

Conclusion

There are a lot more solutions. But these are the ones you’re most likely to settle on. So I leave it at that. Please let me know if you have any questions about this reply. Hope you find your favorite setup soon.

Especially on Debian, I would not be messing with the base / host system so much. This is why a VM is being recommended.

My solution would be to use Distrobox to run another Linux distro in a container. This could a distro that defaults to a version of Python you need like Arch, Fedora, or Ubuntu.

This is a good idea even if you use something else to get the Python version you want. Especially so.

The big advances of this solution are that you get native performance, can see all your normal files, and have an environment that you can customize that does not mess up your host system. You can have other Distroboxes with other tools. You can delete your environment and start over.

Why not use Miniconda or a similar virtual environment solution? I never ever ever work with the system Python. I always pick my Python version and packages per-project to work best with the other dependencies.

Gnome feels really unusual to me and I hate it, I guess I can replace it with KDE.

Yeah I use Debian with KDE on all my systems. Shit’s cash 😎

Also unrelated, but you ought to bookmark the following article that shows you how to use Jupyter in a multi-environment setup:

https://bitsofanalytics.org/posts/jupyter-conda-envs/jupyter_conda_envs

You’ll run into that on many distros,
not only Debian, since most of the distros use a specific python version for OS packages, its recommended to leave the python for your OS unchanged.

But VirtualBox is not a good idea either.

Instead use a venv (virtual environment),
which is the python way to develop/run under different python versions.

https://docs.python.org/3/library/venv.html

Python is a clusterfuck because so many operating systems rely on it generally, so they may require a version minimum.

Which version of Debian are you trying to do this with and what version of python?

I have a repo (why-py3) where I’ve cobtainerized python of various versions with various versions of Debian for “questionable reasons”

Debian is a great choice for developing, it’s rock solid and stable BUT if you want to be working with bleeding edge packages you may need to use the rolling release Debian which is less stable.

Or fedora might be more your speed as it has newer packages typically.

In addition to others’ answers here, I suggest reading over this document whenever you get frustrated by Debian: Don’t Beak Debian

Debian has a particular philosophy for its design choices. Using non-standard sources.list entries, for instance, can break your system in the future. You’ll have a much better time in the long run if you come to understand why Debian is the way it is

You want to look into uv, it can set you up a specific python version fairly easily.

I’ve successfully used pyenv in the past, although uv claims that it includes all pyenv functions and more.

I install python into my user directory on debian and it works great. But if you need to run a particular version of python as root I’d go the container (podman/docker/etc) instead of virtualbox. Massive performance benefit over vbox as containers share your host OS kernel and generally integrates better into tooling. You just specify an image by name like 3.13-bookworm and you get a container with python 3.13 on a debian bookworm equivalent userland

What is this project, and why does it require specific Python 3?