Deploying Python Web Applications with nginx and uWSGI Emperor
You’ve just written a great Python web application. Now, you want to share it with the world. In order to do that, you need a server, and some software to do that for you.
The following is a comprehensive guide on how to accomplish that, on multiple Linux-based operating systems, using nginx and uWSGI Emperor. It doesn’t force you to use any specific web framework — Flask, Django, Pyramid, Bottle will all work. Written for Ubuntu, Debian, Fedora, CentOS 7, Alma Linux, Rocky Linux and Arch Linux (should be helpful for other systems, too). Now with an Ansible Playbook.
Revision 8 (2022-02-20): works with Fedora 35, AlmaLinux 8, RockyLinux 8
CI status for the associated Ansible Playbook:
For easy linking, I set up some aliases: https://go.chriswarrick.com/pyweb and https://go.chriswarrick.com/uwsgi-tut (powered by a Django web application, deployed with nginx and uWSGI!).
Prerequisites
In order to deploy your web application, you need a server that gives you root and ssh access — in other words, a VPS (or a dedicated server, or a datacenter lease…). If you’re looking for a great VPS service for a low price, I recommend Hetzner Cloud (reflink [1]), which offers a pretty good entry-level VPS for €3.49 + VAT / month (with higher plans available for equally good prices). If you want to play along at home, without buying a VPS, you can create a virtual machine on your own, or use Vagrant with a Vagrant box for Fedora 35 (fedora/35-cloud-base
).
Your server should also run a modern Linux-based operating system. This guide was written and tested on:
Ubuntu 18.04 LTS, 20.04 LTS or newer
Debian 10 (buster), 11 (bullseye) or newer
Fedora 33 or newer (with SELinux enabled and disabled)
CentOS 7 (with SELinux enabled and disabled) — manual guide should also work on RHEL 7.
AlmaLinux 8 (with SELinux enabled and disabled) — manual guide should also work on RHEL 8. Referred to as “EL8” collectively with Rocky Linux.
Rocky Linux 8 (with SELinux enabled and disabled) — manual guide should also work on RHEL 8. Referred to as “EL8” collectively with AlmaLinux.
Arch Linux
Debian 8 (jessie) 9 (stretch), Ubuntu 16.04 LTS, and Fedora 24 through 32 are not officially supported, even though they still probably work.
What if you’re using Docker? The story is a bit complicated, and this guide does not apply, but do check the Can I use Docker? at the end of this post for some hints on how to approach it.
Users of other Linux distributions (and perhaps other Unix flavors) can also follow this tutorial. This guide assumes systemd
as your init system; if you are not using systemd, you will have to get your own daemon files somewhere else. In places where the instructions are split three-way, try coming up with your own, reading documentation and config files; the Arch Linux instructions are probably the closest to upstream (but not always). Unfortunately, all Linux distributions have their own ideas when it comes to running and managing nginx and uWSGI.
nginx and uWSGI are considered best practices by most people. nginx is a fast, modern web server, with uWSGI support built in (without resorting to reverse proxying). uWSGI is similarly aimed at speed. The Emperor mode of uWSGI is recommended for init system integration by the uWSGI team, and it’s especially useful for multi-app deployments. (This guide is opinionated.)
Automate everything: Ansible Playbook
A Playbook that automates everything in this tutorial is available.
How to use
Install Ansible on your control computer (not necessarily the destination server).
Clone the Playbook from GitHub.
Read
README.md
. You should also understand how Ansible works.Configure (change three files:
hosts
,group_vars/all
, andgroup_vars/os_<destination OS>
Make sure all the dependencies are installed on your destination server
Run
ansible-playbook -v nginx-uwsgi.yml -i hosts
and watch magic happen.Skip over to End result and test your site.
The manual guide
Even though I personally recommend the Playbook as a much less error-prone way to set up your app, it might not be compatible with everyone’s system, or otherwise be the wrong solution. The original manual configuration guide is still maintained.
Even if you are using the Playbook, you should still read this to find out what happens under the hood, and to find out about other caveats/required configuration changes.
Getting started
Start by installing Python 3 (with venv), nginx and uWSGI. I recommend using your operating system’s packages. Make sure you are downloading the latest versions available for your OS (update the package cache first). For uWSGI, we need the logfile
and python3
plugins. (Arch Linux names the python3
plugin python
; the logfile
plugin may be built-in — check with your system repositories!). I’ll also install Git to clone the tutorial app, but it’s optional if your workflow does not involve Git.
Ubuntu, Debian:
Fedora:
CentOS 7:
yum install epel-release yum install python36 uwsgi uwsgi-plugin-python36 uwsgi-logger-file nginx git wget
EL8 (AlmaLinux 8, Rocky Linux 8):
dnf install epel-release dnf install python36 uwsgi uwsgi-plugin-python3 uwsgi-logger-file nginx git wget
Arch Linux:
Preparing your application
This tutorial will work for any web framework. I will use a really basic Flask app that has just one route (/
), a static hello.png
file and a favicon.ico
for demonstration purposes. The app is pretty basic, but all the usual advanced features (templates, user logins, database access, etc.) would work without any other web server-related config. Note that the app does not use app.run()
. While you could add it, it would be used for local development and debugging only, and would have to be prepended by if __name__ == '__main__':
(if it wasn’t, that server would run instead of uWSGI, which is bad)
The app will be installed somewhere under the /srv
directory, which is a great place to store things like this. I’ll choose /srv/myapp
for this tutorial, but for real deployments, you should use something more distinguishable — the domain name is a great idea.
If you don’t use Flask, this tutorial also has instructions for other web frameworks (Django, Pyramid, Bottle) in the configuration files; it should be adjustable to any other WSGI-compliant framework/script nevertheless.
We’ll start by creating a virtual environment, which is very easy with Python 3:
(The --prompt
option is not supported on some old versions of Python, but you can just skip it if that’s the case, it’s just to make the prompt after source bin/activate
more informative.)
Now, we need to put our app there and install requirements. An example for the tutorial demo app:
cd /srv/myapp git clone https://github.com/Kwpolska/flask-demo-app appdata venv/bin/pip install -r appdata/requirements.txt
I’m storing my application data in the appdata
subdirectory so that it doesn’t clutter the virtual environment (or vice versa). You may also install the uwsgi
package in the virtual environment, but it’s optional.
What this directory should be depends on your web framework. For example, for a Django app, you should have an appdata/manage.py
file (in other words, appdata
is where your app structure starts). I also assumed that the appdata
folder should have a static
subdirectory with all static files, including favicon.ico
if you have one (we will add support for both in nginx).
At this point, you should chown this directory to the user and group your server is going to run as. This is especially important if uwsgi and nginx run as different users (as they do on Fedora). Run one of the following commands:
Ubuntu, Debian:
Fedora, CentOS, EL8 (AlmaLinux, Rocky Linux):
Arch Linux:
Configuring uWSGI and nginx
We need to write a configuration file for uWSGI and nginx.
uWSGI configuration
Start with this, but read the notes below and change the values accordingly:
Save this file as:
Ubuntu, Debian:
/etc/uwsgi-emperor/vassals/myapp.ini
Fedora, CentOS, EL8 (AlmaLinux, Rocky Linux):
/etc/uwsgi.d/myapp.ini
Arch Linux:
/etc/uwsgi/vassals/myapp.ini
(create the directory first and chown it to http:mkdir -p /etc/uwsgi/vassals; chown -R http:http /etc/uwsgi/vassals
)
The options are:
socket
— the socket file that will be used by your application. It’s usually a file path (Unix domain socket). You could use a local TCP socket, but it’s not recommended.chdir
— the app directory.binary-path
— the uWSGI executable to use. Remove if you didn’t install the (optional)uwsgi
package in your virtual environment.virtualenv
— the virtual environment for your application.-
module
— the name of the module that houses your application, and the object that speaks the WSGI interface, separated by colons. This depends on your web framework:Framework Flask, Bottle Django Pyramid Package module where app
is definedproject.wsgi
(project
is the package withsettings.py
)module where app
is definedCallable Flask: app
instance
Bottle:app = bottle.default_app()
application
app = config.make_wsgi_app()
Module package:app
project.wsgi:application
package:app
Caveats Make sure app
is not in anif __name__ == '__main__':
blockAdd environment variable for settings: env = DJANGO_SETTINGS_MODULE=project.settings
Make sure app
is not in anif __name__ == '__main__':
block (the demo quickstart does that!) uid
andgid
— the names of the user account to use for your server. Use the same values as in thechown
command above.processes
andthreads
— control the resources devoted to this application. Because this is a simple hello app, I used one process with one thread, but for a real app, you will probably need more (you need to see what works the best; there is no algorithm to decide). Also, remember that if you use multiple processes, they don’t share memory (you need a database to share data between them).plugins
— the list of uWSGI plugins to use. For Arch Linux, useplugins = python
(thelogfile
plugin is always active). For CentOS 7 only (i.e. not for EL8), useplugins = python36
.logger
— the path to your app-specific logfile. (Other logging facilities are available, but this one is the easiest, especially for multiple applications on the same server)env
— environment variables to pass to your app. Useful for configuration, may be specified multiple times. Example for Django:env = DJANGO_SETTINGS_MODULE=project.settings
You can test your configuration by running uwsgi --ini /path/to/myapp.ini
(disable the logger for stderr output or run tail -f /srv/myapp/uwsgi.log
in another window).
If you’re using Fedora, CentOS, or EL8, there are two configuration changes you need to make globally: in /etc/uwsgi.ini
, disable the emperor-tyrant
option (which we don’t need, as it sets uid/gid for every process based on the owner of the related .ini
config file — we use one global setup) and set gid = nginx
. We’ll need this so that nginx can talk to your socket.
nginx configuration
We need to configure our web server. Here’s a basic configuration that will get us started:
Save this file as:
Ubuntu, Debian:
/etc/nginx/sites-enabled/myapp.conf
Fedora, CentOS, EL8 (AlmaLinux, Rocky Linux):
/etc/nginx/conf.d/myapp.conf
Arch Linux: add
include /etc/nginx/conf.d/*.conf;
to yourhttp
directive in/etc/nginx/nginx.conf
and use/etc/nginx/conf.d/myapp.conf
Note that this file is a very basic and rudimentary configuration. This configuration is fine for local testing, but for a real deployment, you will need to adjust it:
set
listen
to443 ssl
and create a http→https redirect on port 80 (you can get a free SSL certificate from Let’s Encrypt; make sure to configure SSL properly).set
server_name
to your real domain nameyou might also want to add custom error pages, log files, or change anything else that relates to your web server — consult other nginx guides for details
nginx usually has some server already enabled by default — edit
/etc/nginx/nginx.conf
or remove their configuration files from your sites directory to disable it
Service setup
After you’ve configured uWSGI and nginx, you need to enable and start the system services.
For Arch Linux
All you need is:
Verify the service is running with systemctl status emperor.uwsgi
For Fedora, CentOS, EL8
Make sure you followed the extra note about editing /etc/uwsgi.ini
earlier and run:
Verify the service is running with systemctl status uwsgi
If you disabled SELinux, this is enough to get an app working and you can skip over to the next section.
If you want to use SELinux, you need to do the following to allow nginx to read static files:
We now need to install a SELinux policy (that I created for this project; updated 2020-05-02) to allow nginx and uWSGI to communicate. Download nginx-uwsgi.pp and run:
Hopefully, this is enough (you can delete the file). In case it isn’t, please read SELinux documentation, check audit logs, and look into audit2allow
.
For Ubuntu and Debian
Ubuntu and Debian (still!) use LSB services for uWSGI. Because LSB services are awful, we’re going to set up our own systemd-based (native) service.
Start by disabling the LSB service that comes with Ubuntu and Debian:
Copy the .service
file from the uWSGI systemd documentation to /etc/systemd/system/emperor.uwsgi.service
. Change the ExecStart line to:
You can now reload systemd daemons and enable the services:
systemctl daemon-reload systemctl enable nginx emperor.uwsgi systemctl reload nginx systemctl start emperor.uwsgi
Verify the service is running with systemctl status emperor.uwsgi
. (Ignore
the warning about no request plugin)
End result
Your web service should now be running at http://localhost/ (or wherever you set up server to listen).
If you used the demo application, you should see something like this (complete with the favicon and image greeting):
If you want to test with cURL:
curl -v http://localhost/ curl -I http://localhost/favicon.ico curl -I http://localhost/static/hello.png
Troubleshooting
Hopefully, everything works. If it doesn’t:
Check your nginx, system (
journalctl
,systemctl status SERVICE
) and uwsgi (/srv/myapp/uwsgi.log
) logs.Make sure you followed all instructions.
If you get a default site, disable that site in nginx config (
/etc/nginx/nginx.conf
or your sites directory).If you have a firewall installed, make sure to open the ports your web server runs on (typically 80/443). For
firewalld
(Fedora, CentOS, EL8):
If it still does not work, feel free to ask in the comments, mentioning your distribution, installation method, and what doesn’t work.
Can I use Docker?
This blog post is written for systems running standalone. But Docker is a bit special, in that it offers a limited subset of OS features this workflow expects. The main issue is with user accounts, which generally work weird in Docker, and I had issues with setuid
/setgid
as used by uWSGI. Another issue is the lack of systemd, which means that another part of the tutorial fails to apply.
This tutorial uses uWSGI Emperor, which can run multiple sites at once, and offers other management features (such as seamless code restarts with touch /etc/uwsgi/vassals/myapp.ini
) that may not be useful or easy to use in a Docker environment. You’d probably also run uWSGI and nginx in separate containers in a typical Docker deployment.
Regardless, many parts of this tutorial can be used with Docker, although with the aforementioned adjustments. I have done some work on this topic. This tutorial has an Ansible Playbook attached, and the tutorial/playbook are compatible with five Linux distros in multiple versions. How do I know that there were no unexpected bugs in an older version? I could grab a Vagrant image or set up a VM. I do that when I need specific testing, but doing it for each of the distros on each update would take at least half an hour, probably even more. Yeah, that needs automating. I decided to use GitHub Actions for the CI, which can run anything, as long as you provide a Dockerfile.
The Docker images were designed to support running the Playbook and testing it. But the changes, setups and patches could be a good starting point if you wanted to make your own Docker containers that could run in production. You can take a look at the Docker files for CI The images support all 5 distros using their base images, but you could probably use Alpine images, or the python
docker images; be careful not to mix Python versions in the latter case.
That said, I still prefer to run without Docker, directly on the system. Less resources wasted and less indirection. Which is why this guide does it the traditional way.