The Python tempfile Module
Introduction
Temporary files, or “tempfiles”, are mainly used to store intermediate information on disk for an application. These files are normally created for different purposes such as temporary backup or if the application is dealing with a large dataset bigger than the system’s memory, etc. Ideally, these files are located in a separate directory, which varies on different operating systems, and the name of these files are unique. The data stored in temporary files is not always required after the application quits, so you may want these files to be deleted after use.
Python provides a module known as tempfile, which makes creating and handling temporary files easier. This module provides a few methods to create temporary files and directories in different ways. tempfile
comes in handy whenever you want to use temporary files to store data in a Python program. Let’s take a look at a couple of different examples on how the tempfile
module can be used.
Creating a Temporary File
Suppose your application needs a temporary file for use within the program, i.e. it will create one file, use it to store some data, and then delete it after use. To achieve this, we can use the TemporaryFile()
function.
This function will create one temporary file to the default tempfile
location. This location may be different between operating systems. The best part is that the temporary file created by TemporaryFile()
will be removed automatically whenever it the file is closed. Also, it does not create any reference to this file in the system’s filesystem table. This makes it private to the current application i.e. no other program will be able to open the file.
Let’s take a look at the below Python program to see how it works:
import tempfile #1
print("Creating one temporary file...")
temp = tempfile.TemporaryFile() #2
try:
print("Created file is:", temp) #3
print("Name of the file is:", temp.name) #4
finally:
print("Closing the temp file")
temp.close() #5
It will print the below output:
$ python3 temp-file.py
Creating one temporary file...
Created file is: <_io.BufferedRandom name=4>
Name of the file is: 4
Closing the temp file
- To create one temporary file in Python, you need to import the
tempfile
module. - As explained above, we have created the temporary file using the
TemporaryFile()
function. - From the output, you can see that the created object is actually not a file, it is a file-like object. And, the
mode
parameter (not shown in our example) of the created file isw+b
, i.e. you can read and write without being closed. - The temporary file created has no name.
- Finally, we are closing the file using the
close()
method. It will be destroyed after it is closed.
One thing we should point out is that the file created using the TemporaryFile()
function may or may not have a visible name in the file system. On Unix, the directory entry for the file is removed automatically after it is created, although this is not supported on other platforms. Normally TemporaryFile()
is the ideal way to create one temporary storage area for any program in Python.
Create a Named Temporary File
In our previous example, we have seen that the temporary file created using the TemporaryFile()
function is actually a file-like object without an actual file name. Python also provides a different method, NamedTemporaryFile()
, to create a file with a visible name in the file system. Other than providing a name to the tempfile, NamedTemporaryFile()
works the same as TemporaryFile()
. Now let’s use the same above example to create a named temporary file:
import tempfile
print("Creating one named temporary file...")
temp = tempfile.NamedTemporaryFile()
try:
print("Created file is:", temp)
print("Name of the file is:", temp.name)
finally:
print("Closing the temp file")
temp.close()
Running this code will print output similar to the following:
$ python3 named-temp-file.py
Creating one named temporary file...
Created file is:
Name of the file is: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpa3rq8lon
Closing the temp file
So, the created file actually has a name this time. The advantage of NamedTemporaryFile()
is that we can save the name of the created temp files and use them later before closing or destroying it. If the delete
parameter is set to False
, then we can close the file without it being destroyed, allowing us to re-open it later on.
Providing a Suffix or Prefix to the Name
Sometimes we need to add a prefix or suffix to a temp-file’s name. It will help us to identify all temp files created by our program.
To achieve this, we can use the same NamedTemporaryFile
function defined above. The only thing we need to add is two extra parameters while calling this function: suffix
and prefix
import tempfile
temp = tempfile.NamedTemporaryFile(prefix="dummyPrefix_", suffix="_dummySuffix")
try:
print("Created file is:", temp)
print("Name of the file is:", temp.name)
finally:
temp.close()
Running this code will print the following output:
$ python3 prefix-suffix-temp-file.py
Created file is:
Name of the file is: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/dummyPrefix_uz63brcp_dummySuffix
So, if we will pass the two extra arguments suffix
and prefix
to the NamedTemporaryFile()
function, it will automatically add those in the start and end of the file name.
Finding the Default Location of Temp Files
The tempfile.tempdir
variable holds the default location for all temporary files. If the value of tempdir
is None
or unset, Python will search a standard list of directories and sets tempdir
to the first directory value, but only if the calling program can create a file in it. The following are the list of directories it will scan, in this order:
- The directory named by the TMPDIR environment variable.
- The directory named by the TEMP environment variable.
- The directory named by the TMP environment variable
- Platform-specific directories:
- On windows, C:TEMP, C:TMP, TEMP, and TMP, in the same order.
- On other platforms, /tmp, /var/tmp, and /usr/tmp, in the same order.
- The current working directory.
To find out the default location of temporary files, we can call tempfile.gettempdir()
method. It will return the value of tempdir
if it is not None
. Otherwise it will first search for the directory location using the steps mentioned above and then return the location.
import tempfile
print("Current temp directory:", tempfile.gettempdir())
tempfile.tempdir = "/temp"
print("Temp directory after change:", tempfile.gettempdir())
If you will run the above program, it will print an output similar to the following:
$ python3 dir-loc-temp-file.py
Current temp directory: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T
Temp directory after change: /temp
You can see that the first temp directory location is the system-provided directory location and the second temp directory is the same value as the one that we have defined.
Reading and Writing Data from Temp Files
We have learned how to create a temporary file, create a temporary file with a name, and how to create a temporary file with a suffix and/or prefix. Now, let’s try to understand how to actually read and write data from a temporary file in Python.
Reading and writing data from a temporary file in Python is pretty straightforward. For writing, you can use the write()
method and for reading, you can use the read()
method. For example:
import tempfile
temp = tempfile.TemporaryFile()
try:
temp.write(b'Hello world!')
temp.seek(0)
print(temp.read())
finally:
temp.close()
This will print the output as b'Hello world!'
since the write()
method takes input data in bytes (hence the b
prefix on the string).
If you want to write text data into a temp file, you can use the writelines()
method instead. For using this method, we need to create the tempfile using w+t
mode instead of the default w+b
mode. To do this, a mode
param can be passed to TemporaryFile()
to change the mode of the created temp file.
import tempfile
temp = tempfile.TemporaryFile(mode='w+t')
try:
temp.writelines("Hello world!")
temp.seek(0)
print(temp.read())
finally:
temp.close()
Unlike the previous example, this will print “Hello World” as the output.
Create a Temporary Directory
If your program has several temporary files, it may be more convenient to create one temporary directory and put all of your temp files inside of it. To create a temporary directory, we can use the TemporaryDirectory()
function. After all temp files are closed, we need to delete the directory manually.
import tempfile
with tempfile.TemporaryDirectory() as tmpdirname:
print('Created temporary directory:', tmpdirname)
# Both the directory and its contents have been deleted
It will print the below output:
$ python3 mk-dir-temp-file.py
Created temporary directory: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpn_ke7_rk
Create a Secure Temporary File and Directory
By using mkstemp()
, we can create a temporary file in the most secure manner possible. The temporary file created using this method is readable and writable only by the creating user ID. We can pass prefix
and suffix
arguments to add prefix and suffix to the created file name. By default, it opens the file in binary mode. To open it in text mode, we can pass text=True
as an argument to the function. Unlike TemporaryFile()
, the file created by mkstemp()
doesn’t get deleted automatically after closing it.
As you can see in the example below, the user is responsible for deleting the file.
import tempfile
import os
temp_directory = tempfile.TemporaryDirectory()
print("Directory name:", temp_directory)
os.removedirs(temp_directory)
$ python3 mk-secure-dir-temp-file.py
Directory name: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/tmpf8f6xc53
Similar to mkstemp()
, we can create a temporary directory in the most secure manner possible using mkdtemp()
method. And again, like mkstemp()
, it also supports prefix
and suffix
arguments for adding a prefix and suffix to the directory name.
Conclusion
In this article we have learned different ways to create temporary files and directories in Python. You can use temp files in any Python program you want. But just make sure to delete it if the particular method used doesn’t automatically delete it on its own. Also keep in mind that behavior may different between operating systems, like the output directory names and file names.
All of these functions we have explained above works with many different arguments, although we have not covered in detail what type of arguments each function takes. If you want to learn more about the tempfile
module, you should check out the Python 3 official documentation.