Setup black, isort, flake8 in VSCode
After you completed this article, the Python code will be auto formatting and linting whenever you save the file in VSCode.
VSCode Python Extension
Please make sure you have installed the python extension in the VSCode. You can install it through the UI.
Or type the “Ctrl + p” or “Cmd + p” (Based on you are using the Windows or MacOS). Paste the below line and hit enter to install.
Black
First, install the black into your Python environment by pip.
Type the “Ctrl + ,” or “Cmd + ,” to open the settings page in the VSCode. (You can also find it through UI “Preferences -> Settings”)
Search “format on save” and check the checkbox.
Search “python formatting provider” and select the “black”.
Now, if you open the python file and save, it will be auto formatted by the black!
For example, if your file is like this before:
After save, it will be like this:
isort
Type “Ctrl + Shift + P” or “Cmd + Shift + P”. Search the “Preferences: Configure Language Specific Settings” and press enter. And search the “Python” and press enter. It will open the “settings.json”.
Please add the following settings:
Therefore, it may look like this:
(VSCode already has the built-in function to use isort so that we don’t need to install it by pip)
Then isort will reorder your import order when you save the file. For example, if the file you have is like this:
After save, it will be like this:
flake8
First, install the flake8 into your Python environment by pip.
Type “Ctrl + Shift + P” or “Cmd + Shift + P”. Search the “Python: Select Linter” and press enter. Select the “flake8”
How to Protect Your Python Code Health ����
Code can be long and complex. Let’s configure a few good tools to help keep our Python codebase healthy. Flake8, Black, and isort.
Linting and Formatting
Flake8 is a linter. It’s designed to find mistakes in your code that can later become bugs when your code is run.
A formatter arranges our code so that it’s more readable on the screen, but does not change what our code does. Black and isort are formatters.
Installation
Let’s install our linting and formatting tools. For a healthy coding environment use a virtual environment like venv or virtualenv ( venv comes with Python).
Create a virtual environment and activate it.
Exit fullscreen mode
Install the packages.
Exit fullscreen mode
Configuration
You’ll want to configure these tools a little to make them play nicely together.
Flake8 ��
There’s a lot you can configure with Flake8, but you don’t have to. Add the bare minimum first to get Black and Flake8 to cooperate (also we’ll exclude a few directories from linting). In the root of your project create a file called .flake8 .
Exit fullscreen mode
Black ��️
Black is famous as a highly opinionated «uncompromising code formatter». While there are a few settings you can configure, nearly all of the formatting choices are made for you. That’s actually half of its appeal. The other half is getting back all the time you previously used to tweak where this comma was or that spacing. Embrace the way Black will do it all for you and you’ll thank me later.
In the root of your project create a file called pyproject.toml .
Exit fullscreen mode
isort ��️
Short for «import sort» or just «I Sort», this utility will arrange your module’s imports in a consistent opinionated way, much like Black does for the rest of your code.
Let’s configure isort to play nicely with Black. Add the following to our pyproject.toml .
Exit fullscreen mode
Checking Your Code’s Health ��
Let’s run our tools individually first.
Exit fullscreen mode
Now let’s make this a little easier for ourselves by setting up a pre-commit hook. In your project create a file called .git/hooks/pre-commit .
Exit fullscreen mode
When you run git commit this script will run and check your code health before a commit is made. It will show you what issues Flake8 has found and it will format your code. You’ll then get to go back and fix the issues. Finally, you’ll run git commit again.
Here’s what it looks like with errors.
Exit fullscreen mode
Here’s what it will look like when you’re code is healthy.
Exit fullscreen mode
Automation ��
With code health, the more often you can check it the better. You’ve seen how to setup a Git hook. Here’s a few other ways to run Flake8, Black, and isort.
Your code editor likely integrates with these tools using only some light configuration. You’ll benefit from spellcheck-like behavior from Flake8, and format on save from Black and isort.
Some info on setting up VSCode for linting and formatting.
Pre-Commit
The Pre-Commit Framework is Git hooks with super powers. Its easy setup and configuration give you access to an extensive list of shared Git hooks.
CI/CD Pipelines
While a Git hook is great for catching errors before they go to GitHub/GitLab you still want the safety of your CI/CD Pipeline running these tools again. That’s because sometimes people forget to configure their Git hooks ��.
For our formatters, there is a special CI/CD mode that will throw an error when the formatting is incorrect.
Exit fullscreen mode
Source Code Example ��
You’re all set with the tools you’ll need to keep your code healthy. Remember the goal is to make our code bug free, and easy to maintain. Flake8, Black, and isort can help you get there.
Check out packagecake for an example of what we’ve covered here.
sirfuzzalot / packagecake
Turn your Python package into a delicious cake ��
Find out what type of cake your Python package is by running the following:
How to use black, flake8, isort, and pre-commit framework to format Python codes
With black you can format Python code from 2.7 all the way to 3.8 (as of version 20.8b1), which makes for a great replacement for YAPF which can only format code depending on the Python version being used to run it.
My preference is using PEP 8 as my style guide, and so, 79-characters per line of code is what I use. So it’s as simple as running the following code at the root of my project and all non-compliant files will be reformatted:
Let’s explain each option.
- -l or —line-length : How many characters per line to allow. [default: 88]
- -t or —target-version : Python versions that should be supported by Black’s output. [default: per-file auto-detection]
Fairly simple. Allow 79 characters per line, and use py27 as the targetted version.
isort: A Python library to sort imports.
And just as their slogan states: “isort your imports, so you don’t have to.”
The options used are mainly to be compatible with black (see here):
- —multi-line : Multiline output (0-grid, 1-vertical, 2-hanging, 3-vert-hanging, 4-vert-grid, 5-vert-grid-grouped, 6-vert-grid-grouped-no-comma, 7-noqa, 8-vertical-hanging-indent-bracket, 9-vertical-prefix-from-module-import, 10-hanging-indent-with-parentheses).
- 3-vert-hanging
- black
- 27 for Python 2.7
But there’s still something missing. black does not care about comments or docstrings, and isort cares even less, for obvious reasons; enter flake8 .
flake8: A python tool that glues together pep8, pyflakes, mccabe, and third-party plugins to check the style and quality of some python code
Anthony Sottile (@asottile) has mentioned that he plans to drop support for Python 2.7 in future releases, maybe in version 3.9 or 4.0.
Fortunately, I can still use it for Python 2 by running the following command:
PEP 8 recommends limiting docstrings or comments to 72 characters, which is exactly what I’m using for flake8.
So let’s explain each option used.
- —max-doc-length : Maximum allowed doc line length for the entirety of this run. (Default: None)
- —ignore : Comma-separated list of errors and warnings to ignore (or skip). For example, —ignore=E4,E51,W234 . (Default: [‘E226’, ‘E123’, ‘W504’, ‘E121’, ‘W503’, ‘E126’, ‘E704’, ‘E24’])
In my case, I am using 72 as the maximum allowed characters for my docstrings, in accordance with PEP 8, and ignoring the following errors:
- E211 : whitespace before ‘(‘
- Since in Python 2 print is not a function, black adds a space between the print statement from Python 2, and the opening parenthesis
- In my case, this occurs again with the print statement where I am printing just one argument like this print arg
- I do import some modules in order to get “Intellisense” when I peek into the details in PyCharm
- In one of my libraries I am checking if an argument is a string, and in order to cover my bases with plain strings (str) and Unicode, I found that using basestring would work for all characters, including non-Latin characters
- It doesn’t like when binary operators are broken into multi-line statements
pre-commit: A framework for managing and maintaining multi-language pre-commit hooks.
Finally, let’s put it all together with pre-commit.
So in order to use flake8 you’ll have to create a .flake8 file. Mine looks like this:
A pyproject.toml file that in my case looks like this:
And finally my .pre-commit-config.yaml file:
After you’ve configured all of this for the first time, first run the install command for pre-commit and to run tests I use run with the —all-files option, just like this:
So every time you try to commit something to your Git repo, all tests should be marked as Passed , otherwise, the commit will fail.
At the moment of writing this post both black and isort do support the use of pyproject.toml , something that flake8 still hasn’t been implemented unlike flake9 or FlakeHell, which I have not integrated into my workflow; I’m still using flake8 because I’ve installed it via Homebrew.
While you have the option to “ pip -install” all of these tools, currently, I decided to use Homebrew because I don’t usually check if my packages are outdated, something that Homebrew contributors actually do with each new release. See: black , flake8 , and isort , which will install python@3.9 as they all depend on it.
But if you do use pip , I recommend adding an alias for updating all of your outdated packages that should run the following command:
Automate Python workflow using pre-commits: black and flake8
Before I commit my staged Python files, black formats my code and flake8 checks my compliance to PEP8. If everything passes, the commit is made. If not, then I the perform necessary edits and commit again. Less time is spent on code formatting so I can focus more on code logic.
Code reviews are fun! They enable me to learn from others’ codes while providing an opportunity to teach what I know. However, there are still some things I wish to improve when facilitating reviews in my open-source projects:
- Less time commenting on code format, and more time discussing code logic
- Less hassle spotting format errors (“can you really see that trailing whitespace on Line 76?”)
- Stop sounding nitpicky (“Please put two blank lines between function definitions”)
If I could automate the processes above and remove the human-in-the-loop, we can focus more on code logic and implementation. Good thing, I learned about Git hooks, specifically pre-commit hooks. It enables you to automatically run a short script before commit ting. This script can be a checking tool or a formatter. If the script passes, then the commit is made, else, the commit is denied.
In this section, I’ll describe how I created a pre-commit pipeline in PySwarms using the black code formatter, flake8 checker, and the pre-commit Python framework. The entire pipeline looks like this:
Figure: Pre-commit pipeline with black and flake8 for checking my .py files
I’ll first discuss the pre-commit framework, then add components one-by-one: first is black , and then flake8 . I will show the dotfiles present in my project, so feel free to adapt them into your own!
The pre-commit Python framework
We can run shell files all we want to dictate how our pre-commit process goes, but this pre-commit framework written in Python got us covered. It even comes with a set of pre-commit hooks out of the box (batteries included!). To adopt pre-commit it into our system, we simply perform the following actions:
- Install pre-commit: pip install pre-commit
- Add pre-commit to requirements.txt (or requirements-dev.txt )
- Define .pre-commit-config.yaml with the hooks you want to include.
- Execute pre-commit install to install git hooks in your .git/ directory.
The YAML file configures the sources which the hooks will be taken from. In our case, flake8’s already been included in this framework so we just need to specify its id. On the other hand, we need to define where to source black using a few lines of code. Below is a sample .pre-commit-config.yaml file that I use in my project:
Update (03-04-2020) You can also add flake8 from its own repo like so:
In the next section, I will discuss my code formatter ( black ) and checker ( flake8 ). As usual, I will provide my config files for each component.
Code Formatter: black
The black code formatter in Python is an opinionated tool that formats your code in the best way possible. You can check its design decisions in the repository itself. Some notable formatting decisions, in my opinion:
- Unlike in PEP8, code length is 88 characters, not 79.
- Use of double-quotes than single-quotes in strings.
- If there are many function args, each arg will be wrapped per line.
I’d rather maintain the recommended 79-character length. Good thing, they have an option to do so. I just need to configure my pyproject.toml to line-length=79 and everything is all set. Here’s my .toml file for configuring black :
If you are not a fan of black, there’s always autopep8 — a formatter more faithful to PEP8. Good thing, the pre-commit framework already has a hook on this tool, so there’s no need to source from another repository.
Flake8 checker
Flake8 is a powerful tool that checks our code’s compliance with PEP8. In order for black to work nicely with flake8 (or prevent it from spewing out various errors and warnings), we need to list down some error codes to ignore. You can check my .flake8 configuration below:
Results
So what we have is a pipeline that safeguards my project against wrongly-formatted code. On CONTRIBUTING page, I explicitly mentioned using pre-commits (or run flake8 and black on their code manually) before submitting a Pull Request.
Figure: Pre-commit pipeline with black and flake8 for checking my .py files
Now that we have a pre-commit framework set up with black and flake8, let’s see it in action! Here we’ll see how black formats a Python file automagically:
Figure: Short demo on pre-commit hooks
pre-commit
pre-commit is a framework for managing pre-commit hooks in Git. Oh, but what is Git Hook?
Git Hook is a script that is run automatically every time a specific event occurs in the Git repository.
In this case, the event here is the commit code. We will use the pre-commit hook to check the changes in the code of each style convention automatically before commit and integrating it into the system. If something goes wrong, the commit will fail and we’ll get the associated error messages to fix. Commit is only successful when no errors occur.
So far we have just introduced and run the above tools manually. Now it’s time to combine them to run automatically!
WORKFLOW WITH PRE-COMMIT HOOKS
overview
Workflow with pre-commit (Image taken from the end of post link)
Before implementing Git commit, I will use isort and black to format the code automatically, then use flake8 to check again with standard PEP8 (all configured by pre-commit ). The commit will be successful if there is no error. If an error occurs, we will go back and fix it where necessary and commit again. This workflow helps to reduce the time to reformat the code manually, thereby focusing more on the logic. Team working together is also easier and more efficient.
Set-up step by step
(These are the configuration files that I am using. You can customize them to suit your own style or needs!)
Create virtualvenv if needed and install pre-commit:
Don’t forget to add it to requirements.txt or Pipfile after installation.
Configure pre-commit by creating a .pre-commit-config.yaml in the root directory with the following content (you can adjust to the latest version at the time of reading this article):
Configure isort by creating a .isort.cfg file in the root directory with the following content:
If you have noticed in the pre-commit configuration file, along with isort , we use add seed-isort-config to automatically add packages to known_third_party in the isort configuration (instead of doing it manually). Configure black by creating a pyproject.toml file in the root directory with the following content:
Configure flake8 by creating .flake8 file with the following content:
After configuration is complete, run the following command to complete the installation:
Finally, before executing Git commit, run the following command:
As you can see, installing the above workflow is very easy for the team because all the configuration files are already in the project, members just need to run the pre-commit install command.
Name already in use
If nothing happens, download GitHub Desktop and try again.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching Xcode
If nothing happens, download Xcode and try again.
Launching Visual Studio Code
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
Latest commit
Git stats
Files
Failed to load latest commit information.
README.rst
Flake8 meet isort
Use isort to check if the imports on your python files are sorted the way you expect.
Add an .isort.cfg to define how you want your imports sorted and run flake8 as you usually do.
See isort documentation for .isort.cfg available options.
Install with pip:
Install with conda:
If using the select option from flake8 be sure to enable the I category as well, see below for the specific error codes reported by flake8-isort.
See flake8 —help for available flake8-isort options.
Error code Description I001 isort found an import in the wrong position I002 no configuration found (.isort.cfg or [isort] in configs) I003 isort expected 1 blank line in imports, found 0 I004 isort found an unexpected blank line in imports I005 isort found an unexpected missing import - Python 3.7, 3.8, 3.9, 3.10, 3.11 and pypy3
- flake8
- isort
Relation to flake8-import-order
As an alternative to this flake8 plugin, there’s flake8-import-order that could be worth checking out. In contrast to this plugin that defers all logic to isort, the flake8-import-order comes bundled with it’s own logic.
flake8-import-order comes with a few predefined set of styles meanwhile this plugin can be customized a bit more. But the biggest difference could lie in that flake8-isort actually has the corresponding sorting engine isort that can sort the import orders of your existing python files. Meanwhile flake8-import-order has no such corresponding tool, hence big existing projects who want to adopt either would get a more automized experience choosing flake8-isort.