How to make temporary memory files in shell scripts

Recently, we have had many discussions about shell scripts that create a temporary memory file that is only available to the invoked shell but not visible to any other user or process except the root user. If we use the/tmp directory to create a file, then it will be available to everyone. Next comes the idea of the “HERE” document feature. For example, the following simple shell program uses the “HERE” document feature and other techniques.

Advertisement

What are temporary files in Linux and Unix shells?

Temporary files, a key tool in data management, store data that programs or scripts need for a short period. These files serve various purposes, such as holding data temporarily before writing it to a disk or sending it over a network, acting as intermediate storage, and preventing excessive writes to storage when working with data that doesn’t need to be permanently saved. This efficient system ensures smooth data flow and optimized storage usage.

Where are temporary files stored by default?

For convenience and system compatibility, the standard temporary directory in most Unix-like systems (such as Linux, *BSD family, and macOS) is /tmp/. You can see the current /tmp/ dir using the ls command:
$ ls -l /tmp/
$ sudo ls -l /tmp/
$ sudo ls -ld /tmp/

How to make temporary memory files in shell scripts

Click to enlarge

Outputs from the last command:

drwxrwxrwt 20 root root 12288 Mar 13 12:34 /tmp/

Some Unix/Linux sysadmin can create temporary files directly in memory using tmpfs for faster access. I usually use tempfs for storing nginx buffering or caching files.

Uses for temporary files in your shell scripts

Shell scripts in Bash, Zsh, Tcsh, Sh are text files that automate tasks in Linux or Unix. I often use temporary files in my shell scripts for data processing and output redirection. Unique temporary file names help prevent conflicts with existing files. Let us see tools and command for working with temporary files when using with your shell scripts.

# 1 : Temporary memory files and shell scripts using the HERE document

Try the following examples at the CLI

#!/bin/bash
cat <<STOP
DATA1=""
DATA2=""
echo "$DATA1"
STOP

OR use something as follows in your shell sciprt to make index.html file:

#!/bin/bash
INDEX="/var/www/html/index.html"
cat <<EOF > "${INDEX}"
<!DOCTYPE html>
<title>Direct IP access not allowed</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
</head>
<body>
<center>
<h1>🚫 Direct IP access not allowed 🚫</h1>
<p>Hello, <strong><!--#echo var="REMOTE_ADDR" --></strong>!</p>
You've requested an IP address that is part of the shared web-hosting network. A valid Host header must be supplied to reach the desired website.
<hr>
<small>Nginx server</small> 
</center>
</body>
</html>
EOF

However, I soon discovered that “here” documents are implemented as temporary files by the bash shell. For example, if we run strace on the script, we can easily see it is opening the file in /tmp/:
$ strace -f script.bash
Bottom line if data critical using /tmp/ in shell scripts are useless, it is better go with private directory for scripts or program in C, Python, Rust, Bash, Go etc. There are other tools mktemp command or [nixmd name=”tempfile”] too.

#2 Creating a private TMP directory

Say you are running a script as a jobs user from /home/jobs/scripts/ directory. Then log in as the wwwjobs user:
admin@nixcraft-server1:$ su - wwwjobs
Make a new directory using the mkdir command wwwjobs@nixcraft-server1:$ mkdir /home/wwwjobs/.tmp_dir1
Set correct permissions for the /home/wwwjobs/.tmp_dir1 using the chown command or chmod command wwwjobs@nixcraft-server1:$ chown wwwjobs:wwwjobs /home/wwwjobs/.tmp_dir1/
wwwjobs@nixcraft-server1:$ chmod 0600 /home/wwwjobs/.tmp_dir1

Finally, update your script to use /home/wwwjobs/.tmp_dir1/ as a temporary location.

#3: Using the mktemp command

To make or create a temporary file or directory try the mktemp command. Just type:
$ mktemp
Outputs:
/tmp/tmp.O3XciIvKs3

Here is how to use in your script:

#!/bin/bash
_tmp_file_1="$(mktemp /tmp/my_coool_script.XXXXXXXXX)" || exit 1
echo "This script using '$_tmp_file_1' as a temp file..."
# rest of the script ..

Creating an empty temporary file with a given suffix and print the absolute path to file

The syntax is:
$ mktemp --suffix ".backup-script"
Outputs:
/tmp/tmp.uhtaOcoCwe.backup-script

How to make a temporary directory and print the absolute path to it

Try:
$ mktemp -d
Outputs:
/tmp/tmp.nxukGATJgf

Verify it using the ls command or stat command $ ls -ld /tmp/tmp.nxukGATJgf
Outputs:

drwx------ 2 vivek vivek 4096 Mar 13 12:57 /tmp/tmp.nxukGATJgf

OR
$ stat /tmp/tmp.nxukGATJgf
Outputs:

  File: /tmp/tmp.nxukGATJgf
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: fd01h/64769d	Inode: 55181421    Links: 2
Access: (0700/drwx------)  Uid: ( 1000/   vivek)   Gid: ( 1000/   vivek)
Access: 2024-03-13 12:57:02.251762853 +0530
Modify: 2024-03-13 12:57:02.251762853 +0530
Change: 2024-03-13 12:57:02.251762853 +0530
 Birth: -

Best practices for making temporary memory files in shell scripts

  1. It is crucial to permanently remove temporary files from shell scripts once they are no longer needed. Please do so to save disk space and potential security risks.
  2. Always use system-defined directories or /tmp for temporary files. As explained earlier, create temporary directories specific to the script or app for sensitive data.
  3. Use the “mktemp” or “tempfile” command to prevent conflicts and possible data overwrites. This avoids race conditions.

Putting it all together

Here is a sample shell script:

#!/bin/bash
# Name: demo.sh
# Purpose: Make a temp file
# Author: Vivek Gite <www.cyberciti.biz> under GPL v2.x+
#------------------------------------------------------------
 
# Create a temporary file with a unique name
my_temp_file="$(mktemp)"
 
# Process the df command and store the result in the temporary file
df -HT | grep -E -v '^(/dev/loop|tmpfs|udev|Filesystem)' > "$my_temp_file"
 
# Now print or do something with the contents of the temporary file 
echo "Processing device ..."
cat "$my_temp_file"  
#grep "nvme" "$my_temp_file" 
 
# Remove the temporary file 
rm -f "$my_temp_file"

Run the the shell script:
$ chmod +x -v demo.sh
$ ./demo.sh

🥺 Was this helpful? Please add a comment to show your appreciation or feedback.

nixCrat Tux Pixel Penguin
Hi! 🤠
I'm Vivek Gite, and I write about Linux, macOS, Unix, IT, programming, infosec, and open source. Subscribe to my RSS feed or email newsletter for updates.

0 comments… add one

Leave a Reply

Your email address will not be published. Required fields are marked *

Use HTML <pre>...</pre> for code samples. Your comment will appear only after approval by the site admin.