How to Auto Generate requirements.txt (Dependencies) in Python — 3 examples
In this tutorial, we'll see how to automatically create requirements.txt in Python.
We'll cover 3 different ways of auto generation of dependencies:
- pip freeze > requirements.txt — most generic ones. No need for additional installation.
- dephell deps add —from=Pipfil — additional library is needed. It can generate dependencies for a single Python script
- pipreqs /path/to/project — additional library is needed. Allow customization in order to extract requirements related to given folder
Example 1: Generate requirements.txt by pip freeze
Creating requirements.txt for a Python project with virtual environments:
Before running the command be sure that:
- the virtual environments is activated
- the command will be executed in the same folder as the project
A file requirements.txt with Python dependencies will be generated in the same folder as the execution.
If you use this command all requirements will be collected from the virtual environment. If you need to get requirements used only in the current project scope then you need to check next options.
Example 2: Auto Generate requirements.txt by dephell — for a single Python script
Dephell is good option to auto generate requirements.txt if:
- your project is not using virtual environments
- to generate them only for a single script for a bigger Python project
More information for this package:
The dephell is installed by two options:
How to generate requirements for a single Python script? It can be done by a command:
- requirements.txt is the output file
- my_script.py is the file for which dependency list will be build
The structure of the project could be: ../project/src/my_script.py — you need to run the command from folder: src .
Another option is to use imports like:
—from – path or format for reading requirements. If it is format then dephell will scan the current directory to find out a file that can be parsed by this converter. If it is filename then dephell will automatically determine file format.
Example 3: Auto Generation of requirements.txt with pipreqs
Another alternative is pipreqs . More information is available on this link: github pipreqs.
First the package needs to be installed by:
Generating all dependencies with:
Note Why to use pipreqs ? Because pip freeze will collect all dependencies from the environments. While pipreqs will collect requirements used only in the current project!
- pip freeze only saves the packages that are installed with pip install
- pip freeze saves all packages in the environment
- if you like to generate requirements.txt without installing the modules
By using SoftHints — Python, Linux, Pandas , you agree to our Cookie Policy.
Automatically create requirements.txt
Sometimes I download the python source code from github and don’t know how to install all the dependencies. If there is no requirements.txt file I have to create it by hands.
The question is:
Given the python source code directory is it possible to create requirements.txt automatically from the import section?
25 Answers 25
You can use the following code to generate a requirements.txt file:
more info related to pipreqs can be found here.
Sometimes you come across pip freeze , but this saves all packages in the environment including those that you don’t use in your current project.
Use Pipenv or other tools is recommended for improving your development flow.
If you do not use a virtual environment, pigar will be a good choice for you.
For python3: (I have both python 2 and 3 on my machine, where python2 is the default)
To check your python version:
In my case, I use Anaconda, so running the following command from conda terminal inside my environment solved it, and created this requirements.txt file for me automatically:
This was taken from this Github link pratos/condaenv.txt
If an error been seen, and you are using anaconda, try to use the .yml option:
For other person to use the environment or if you are creating a new enviroment on another machine:
Kinda mind-blowing how this simple task is so complicated in Python. Here is what I think is the best way to do it automatically.
You need two tools:
pip3 install pipreqs
pipreqs will go through your project and only install the packages that your project use. Instead of all the packages in your python environment as pip freeze would do.
But there’s a problem with this approach. It does not install the sub-packages.
For example, your project uses pandas==1.3.2 . pandas itself uses numpy==1.21.2 among other packages. But pipreqs itself does not write the sub-packages (i.e. numpy) in requirments.txt
This is where you need to combine pipreqs with the second tool.
pip3 install pip-tools
pip-tools will take the packages in requirements.in and generate the requirements.txt with all the sub-packages. For example, if you have pandas==1.3.2 in requirements.in , pip-tools would generate
numpy==1.21.2 # via pandas in requirements.txt .
But you need to manually add the package in requirements.in . Which is prone to mistake and you might forget to do this once in a while.
This is where you can use the first tool.
But both the tools write to requirements.txt . So how do you fix it?
Use the —savepath for pipreqs to write in requirements.in instead of the default requirements.txt .
To do it in one command; just do
pipreqs —savepath=requirements.in && pip-compile
There you go. Now you don’t need to worry about manually maintaining the packages and you’re requirements.txt will have all the sub-packages so that your build is deterministic.
TL;DR
- pip3 install pipreqs
- pip3 install pip-tools
Use the following to build a deterministic requirements.txt
pipreqs —savepath=requirements.in && pip-compile
As most of the answers using pipreqs didn’t work for me. Here, is my answer.
To generate the requirements.txt file:
I prefer using pipreqs more than pip freeze , as pip freeze saves all packages in the environment including those that you don’t use in your current project. However, pipreqs only save the ones you are using in your project.
To install the requirements use:
I blindly followed the accepted answer of using pip3 freeze > requirements.txt
It generated a huge file that listed all the dependencies of the entire solution, which is not what I wanted.
So you need to figure out what sort of requirements.txt you are trying to generate.
If you need a requirements.txt file that has ALL the dependencies, then use the pip3
However, if you want to generate a minimal requirements.txt that only lists the dependencies you need, then use the pipreqs package. Especially helpful if you have numerous requirements.txt files in per component level in the project and not a single file on the solution wide level.
Firstly, your project file must be a py file which is direct python file. If your file is in ipynb format, you can convert it to py type by using the line of code below:
Then, you need to install pipreqs library from cmd (terminal for mac).
Now we can create txt file by using the code below. If you are in the same path with your file, you can just write ./ . Otherwise you need to give path of your file.
That will create a requirements.txt file for your project.
Make sure to run pip3 for python3.7.
Before executing the above command make sure you have created a virtual environment.
python3:
python2:
After that put your source code in the directory. If you run the python file now, probably it won’t launch if you are using non-native modules. You can install those modules by running pip3 install <module> or pip install <module> .
This will not affect you entire module list except the environment you are in.
Now you can execute the command at the top and now you have a requirements file which contains only the modules you installed in the virtual environment. Now you can run the command at the top.
I advise everyone to use environments as it makes things easier when it comes to stuff like this.
Simple Pythonic Way
To get a list of all the REQUIREMENTS in a standard requirements.txt file, you can use the following command.
Now, this should automatically create a standard requirements file with all of the packages installed alongside their corresponding versions.
Pretty Print on Terminal
If you just want to get a pretty print on the terminal you can use the following approach.
This lists all of the installed packages, in a pretty print format.
Custom Dependency
If you have a project folder like say, a Github Repo, and you want to get a custom requirements.txt for project You can use the following Package. https://pypi.org/project/pipreqs/ pipreqs
Usage
Contents of requirements.txt
If you have installed many dependencies in your system and you need requirements.txt for a specific project, you can install first pipreqs:
and execute the below command under the project folder.
This command will generate requirements.txt file for the particular project.
Automatic requirements.txt updating approach
While developing a python application with requirements.txt we have several choices:
- Generate requirements.txt after development, when we want to deploy it. It is performed by pip freeze > requirements.txt or pipreqs for less messy result.
- Add every module to requirements.txt manually after each install.
- Install manager that will handle requirements.txt updates for us.
There are many answers for the 1-st option, the 2-d option is self-explanatory, so I would like to describe the 3-d approach. There is a library called to-requirements.txt. To install it type this:
If you read the whole command at once you would see, what it does. After installing you should setup it. Run:
Use requirements.txt
PyCharm provides integration with the major means of requirements management and makes it possible to track the unsatisfied requirements in your projects and create a virtual environment based on the requirements.txt file .
Define requirements
From the Tools menu, select Sync Python Requirements .
In the opened dialog, specify the name of the requirements file. The recommended name for the requirements file is requirements.txt . When a file with this name is added to the root project directory, it is automatically detected by Python Integrated tools.
Select the method of handling versions of the required libraries. The version numbers can be defined:
Greater or equal
Define the requirements management policy:
Remove unused requirements
Deletes records that correspond to unused libraries and packages.
Modify base files
Allows modifications in the base requirements files (if any is referenced in the requirements.txt file).
Keep existing version specifier if it matches the current version
Leaves the version number unchanged if it satisfied the selected method versions handling.
Click OK and inspect the generated file.
You can also run pip freeze > requirements.txt in the command line to generate a requirements.txt file for your project. See https://pip.pypa.io/en/stable/reference/pip_freeze/ for more details.
If the name of the requirements file differs from requirements.txt or when you have several requirements files in one project, you have to notify PyCharm about the requirements file you want to apply.
Configure the default requirements file
Press Ctrl+Alt+S to open the IDE settings and select Tools | Python Integrated Tools .
In the Package requirements file field, type the name of the requirements file or click the browse button and locate the desired file.
Click OK to save the changes.
Though you can always run the Sync Python Requirements to update the requirements file, PyCharm provides quick fixes that enable populating this file.
Update a requirements file
In an import statement of a Python file, click a package which is not yet imported. PyCharm suggests a quick-fix:
Select and apply the suggested quick-fix. The package is added to the dependency management file.
PyCharm provides quick fixes and notifications related to the unsatisfied dependencies.
Install the required packages
The notification bar is displayed when the Unsatisfied package requirements inspection is enabled. You can enable it in the Settings | Editor | Inspections dialog.
Open a project with the requirements file specified, a notification bar is displayed on top of any Python or requirements file opened in Editor :
Click one of the provided links to satisfy or ignore requirements.
If you have selected the Ignore option, you can always change your mind, and remove the package from the list of the ignored packages.
Add the ignored dependencies
Press Ctrl+Alt+S to open project Settings .
Go to Editor | Inspections .
In the list of the inspections, select Unsatisfied package requirements .
Preview the list of the ignored requirements and click the Add icon () to add them.
How to Create Requirements.txt Python?
A requirement.txt python file is a type of file that usually stores information about all the libraries, modules, and packages which are specific to project that are used while developing a particular project.
This article will guide us in how to create requirements.txt file python and installing dependencies from the requirements.txt file. We would be looking into various ways to creating requirements.txt Python.
Need for Requirements.txt File
While working on Python Projects, we have probably noticed a file called requirements.txt. This requirements.txt file is used for specifying what python packages are required to run the project. It stores the information of what packages with specified version is needed for running the particular project.With a requirement.txt file we can get started with the application and install all the required dependencies in a single command . This plays a crucial role as we start developing our python application , we would be using specific version of packages . Any change in the version might break your code. Globally we might have installed a large number of packages but in particular project we might need fewer.
Problems with the Traditional Requirements.txt Files
Traditionally, we create a requirements.txt file by creating a virtual environment . After activating a virtual environment, we have to run the command as follows:
It works fine, but the problem with this approach is that it includes all the packages that are installed via pip install <package_name> and the sub-dependency packages .Going with this approach adds up a lot of sub packages which are not required to install manually and comes with main packages installation. Listing these all makes the file long.
Let’s consider the following scenario: We are working on Django based application, and the only package we have installed is Fastapi through the command :
However, when we tried to put all the installed packages into the requirements.txt file apart from Django, additional packages were installed internally by pip for Fastapi . To avoid this, we can use the grep command to filter for only required packages. The command to pass the standard output from pip freeze to grep is as follows :
The | command is called a pipe . It is used to pipe, or transfer , the standard output from the command on its left into the standard input of the command on its right .
The shell command grep is used to search files for lines that match a pattern and returns the results. Various options can be specified along with the grep command to specify the search.
What is a Virtual Environment?
In basic, a virtual environment is an isolated environment for python projects. It allows you to create an isolated environment for each python project. This makes it easier for us to manage packages and dependencies throughout projects, especially where they share the same dependencies. Various ways exist to create a virtual environment and make a requirements.txt file. Some of them are as follows :
- virtualenv — Virtualenv is a library that allows us to execute a virtual environment .
- pipenv — Pipenv is a dependency manager for Python projects.
- Pipreqs — Pipreqs is another alternative that works without creating a virtual environment in first place.
Once the virtual environment is created for our project, let’s see how we can use different packages to generate a requirements.txt file. Let’s first explore how to use the Virtualenv packages.
Working with Virtualenv
Virtualenv is a library that allows us to execute a virtual environment. To install and work with it, you can install it through the following pip command:
After installation and setting up the environment, we need to activate the environment using the source :
source is a shell built-in command which is used to read and execute the content of a file(generally set of commands), passed as an argument in the current shell script.
Once the virtual environment is activated, the name of your virtual environment will appear on the left side of the terminal. This will let you know that the virtual environment is currently active.
Now you can install dependencies related to the project in this virtual environment. For example, if you use Fastapi 0.77.1 for a project, you can install it like other packages.
How to Get the Requirements.txt File: Using Virtualenv?
Creation of a requirements.txt through Virtualenv could be done through a pip freeze command following Redirecting command to requirements.txt file.
Firstly , what pip freeze would generate is a list of packages that are required or installed through pip commands , following with the > symbol which takes all the generated text or names as output and gives to requirements.txt file as input. If requirements.txt file is absent before the command execution , a new file would be created itself by the same command.
The > symbol is used to redirect output by taking the output from the command on the left and passing as input to the file on the right.