OpenSSH offers RSA and DSA authentication to remote systems without supplying a password. keychain is a special bash script designed to make key-based authentication incredibly convenient and flexible. It offers various security benefits over passphrase-free keys.
How does keychain make it better than a keyless passphrase?
If an attacker manages to log into the server with passphrase-free keys, all other your servers/workstation on which keys used are also the security risk. With keychain or ssh-agent attacker will not be able to touch your remote systems without breaking your passphrase. Another example, if your laptop or hard disk stolen, an attacker can simply copy your key and use it anywhere as a passphrase does not protect it.
The keychain act as a manager for ssh-agent, typically run from ~/.bash_profile. It allows your shells and cron jobs to share a single ssh-agent process. By default, the ssh-agent started by keychain is long-running and will continue to run, even after you have logged out from the system. If you want to change this behavior, take a look at the --clear and --timeout options, described below. Our sample setup is as follows:
peerbox.nixcraft.net.in => Remote Backup Server. Works in pull only mode. It will backup server1.nixcraft.net.in and server2.nixcraft.net.in. vivek-desktop.nixcraft.net.in => My desktop computer. server1.nixcraft.net.in => General purpose remote server. server2.nixcraft.net.in => General purpose remote web / mail / proxy server.
You need to install keychain software on peerbox.nixcraft.net.in so that you or scripts can log in securely to other two servers for backup.
Install keychain on CentOS / RHEL / Fedora Linux
RHEL/CentOS Linux user type the following command to first enable psychotic repo and install keychain package on CentOS 7.x:
##[*** Install psychotic repo **]##
$ sudo rpm --import http://wiki.psychotic.ninja/RPM-GPG-KEY-psychotic
$ sudo rpm -ivh http://packages.psychotic.ninja/6/base/i386/RPMS/psychotic-release-1.0.0-1.el6.psychotic.noarch.rpm
##[*** install keychain from psychotic repo **]##
$ sudo yum --enablerepo=psychotic install keychain
Sample outputs:
Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.fibergrid.in * epel: epel.mirror.angkasa.id * extras: mirror.fibergrid.in * updates: mirror.fibergrid.in Resolving Dependencies --> Running transaction check ---> Package keychain.noarch 0:2.8.0-3.el7.psychotic will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================= Package Arch Version Repository Size ============================================================================= Installing: keychain noarch 2.8.0-3.el7.psychotic psychotic 44 k Transaction Summary ============================================================================= Install 1 Package Total download size: 44 k Installed size: 97 k Is this ok [y/d/N]: y Downloading packages: keychain-2.8.0-3.el7.psychotic.noarch.rpm | 44 kB 00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : keychain-2.8.0-3.el7.psychotic.noarch 1/1 Verifying : keychain-2.8.0-3.el7.psychotic.noarch 1/1 Installed: keychain.noarch 0:2.8.0-3.el7.psychotic Complete!
Fedora Linux user type:
$ sudo dnf install keychain
Install keychain on Debian / Ubuntu Linux
To add the package:
$ sudo apt-get update
$ sudo apt-get install keychain
Sample outputs:
Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: gnupg-agent ssh-askpass The following NEW packages will be installed: keychain 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 27.4 kB of archives. After this operation, 81.9 kB of additional disk space will be used. Get:1 http://mirrors.service.networklayer.com/ubuntu xenial/universe amd64 keychain all 2.8.1-0.1 [27.4 kB] Fetched 27.4 kB in 0s (0 B/s) Selecting previously unselected package keychain. (Reading database ... 81414 files and directories currently installed.) Preparing to unpack .../keychain_2.8.1-0.1_all.deb ... Unpacking keychain (2.8.1-0.1) ... Processing triggers for man-db (2.7.5-1) ... Setting up keychain (2.8.1-0.1) ...
Install keychain on FreeBSD
To install the port:
# cd /usr/ports/security/keychain/ && make install clean
To add the package use pkg as follows:
# pkg install keychain
Install keychain on OpenBSD
To add the package use pkg_add as follows:
# pkg_add -v keychain
How do I setup SSH keys with passphrase?
Simply type the following commands:
$ ssh-keygen -t rsa
OR
$ ssh-keygen -t dsa
Assign the pass phrase when prompted. See the following step-by-step guide for detailed information:
- Howto Linux / UNIX setup SSH with DSA public key authentication (password less login)
- Howto use multiple SSH keys for password less login
How do I Use keychain?
Once OpenSSH keys are configured with a pass phrase, update your $HOME/.bash_profile file which is your personal initialization file, executed for login BASH shells:
$ vi $HOME/.bash_profile
Append the following code:
### START-Keychain ### # Let re-use ssh-agent and/or gpg-agent between logins /usr/bin/keychain $HOME/.ssh/id_dsa source $HOME/.keychain/$HOSTNAME-sh ### End-Keychain ###
Now you’ve keychanin configured to call keychain tool every login. Just log out and log back in to server from your desktop to test your setup:
$ ssh root@www03.nixcraft.net.in
Sample Output:
Fig.01 - Keychain in Action
# scp $HOME/.ssh/id_dsa.pub server1.nixcraft.net.in:~/pubkey
# scp $HOME/.ssh/id_dsa.pub server2.nixcraft.net.in:~/pubkey
# ssh server1.nixcraft.net.in cat ~/pubkey >> ~/.ssh/authorized_keys2; rm ~/pubkey
# ssh server2.nixcraft.net.in cat ~/pubkey >> ~/.ssh/authorized_keys2; rm ~/pubkey
# ssh root@server1.nixcraft.net.in
# ssh user@server2.nixcraft.net.in
Task: Clear or delete all of ssh-agent’s keys
# keychain --clear
Security Task: Make sure intruder cannot use your existing ssh-agent’s keys (only allow cron jobs to use password less login)
The idea is pretty simply only allow backup shell scripts and other cron jobs to allow password-less login but all users including an intruder must provide a passphrase key for interactive login. It is done by deleting all of ssh-agent’s keys. This option will increase security, and it still allows your cron jobs to use your ssh keys when you are logged out. Update your ~/.bash_profile as follows:
/usr/bin/keychain --clear $HOME/.ssh/id_dsa
If you are using RSA, use:
/usr/bin/keychain --clear $HOME/.ssh/id_rsa
Now, just log in to remote server box once :
$ ssh root@peerbox.nixcraft.net.in
Log out (only grant access to cron jobs such as backup)
# logout
Task: Use keychain with backup scripts for password-less login via cron job
Add the following before your rsync, tar over ssh, or any other network backup command:
source $HOME/.keychain/$HOSTNAME-sh
Here is a sample rsync script:
#!/bin/bash # Remote Server Rsync backup Replication Shell Script # Local dir location LOCALBAKPOINT=/iscsi LOCALBAKDIR=/backups/server1.nixcraft.net.in/wwwroot # Remote ssh server setup SSHUER=root SSHSERVER=server1.nixcraft.net.in SSHBACKUPROOT=/wwwroot # Make sure you can log in to remote server without a password source $HOME/.keychain/$HOSTNAME-sh # Make sure local backup dir exists [ ! -d ${LOCALBAKPOINT}${LOCALBAKDIR} ] && mkdir -p ${LOCALBAKPOINT}${LOCALBAKDIR} # Start backup /usr/bin/rsync --exclude '*access.log*' --exclude '*error.log*' -avz -e 'ssh ' ${SSHUER}@${SSHSERVER}:${SSHBACKUPROOT} ${LOCALBAKPOINT}${LOCALBAKDIR} # See if backup failed or not to /var/log/messages file [ $? -eq 0 ] && logger 'RSYNC BACKUP : Done' || logger 'RSYNC BACKUP : FAILED!'
If you are using rsnaphot backup server (see how to setup RHEL / CentOS / Debian rsnapshot backup server) add the following to your /etc/rsnapshot.conf file
# Get ssh login info via keychain cmd_preexec source /root/.keychain/hostname.example.com-sh
A note about keychain and security
- Cracker with an advanced attacking with deadly coding skills can still get key from memory. However, keychain makes it pretty difficult for normal users and attackers to steal your keys and use it.
- OpenSSH sshd server offers two additional options to protect abuse of keys. First, make sure root login disabled (PermitRootLogin yes). Second, specify which user accounts on the server are allowed to be used for authentication by adding AuthorizedKeysFile %h/.ssh/authorized_keys_FileName. See sshd_config man page for further details.
Suggested readings:
- man pages sshd, sshd_config, keychain
- rsnapshot MySQL backup script.
- The OpenSSH project official website.
- The Keychain project official website.
- FreeBSD Install Rsnapshot Backup Utility
- Debian / Ubuntu Linux Install Rsnapshot Backup Utility
- How To Install Rsnapshot on a Red hat / CentOS Linux
- UNIX / Linux: Rsnapshot Restore Backups
- Rsync: Preserve / Copy Hard Links ( Backup Rsnapshot Directory Tree )
- Rsnapshot WARNING: Could not lchown() symlink "/path/to/file" Error and Solution
- keychain: Set Up Secure Passwordless SSH Access For Backup Scripts
🐧 Get the latest tutorials on Linux, Open Source & DevOps via:
- RSS feed or Weekly email newsletter
- Share on Twitter • Facebook • 15 comments... add one ↓
| Category | List of Unix and Linux commands |
|---|---|
| File Management | cat |
| Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
| Network Utilities | dig • host • ip • nmap |
| OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
| Package Manager | apk • apt |
| Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
| Searching | grep • whereis • which |
| User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
| WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
I think I found a typo.
“OpenSSH sshd server offers two additional option to protect abuse of keys. First, make sure root login disabled (PermitRootLogin yes).”
another interesting way to protect ssh, is chroot them, but it depends of the particulary needs of each one
I have a problem with my rsnapshot configuration. If I enter your command into rsnapshot.conf file, I get an error:
ERROR: cmd_preexec source /home/lexsys/.keychain/dev-server-sh - "source" is not executable or can't be found. Please use an absolute path.I created an executable 1.sh, placed the command into this file and write in rsnapshot.conf:
cmd_preexec /root/1.shEverything works fine.
ssh-copy-id command is an easier way to copy your public key to a server:
ssh-copy-id -i ~/.ssh/id_dsa.pub user@host
Hi, I have been trying to get rsnapshot to run with keychain under cron for root when logged out.
For me adding
source /root/.keychain/-sh
to cmd_preexec in the rsnapshot.conf did not work
What has finally worked for me which works remotely and locally is:
under cron run a command pointing to shell scripts for hourly daily weekly and monthly rsnapshots
my script is for hourly backups is hourly.sh
#!/bin/bash
ENV=/root/.bashrc
source /root/.keychain/-sh
rsnapshot hourly
the reason why this was needed is because cron for ssh doesn’t enter a shell to perform it’s function, so before rsnapshot begins you must point the process into a shell or you get an annoying and failing error 255 stating rsync couldn’t ssh(or something like that). Then just re comment the cmd_preexec line in the rsnapshot.conf
hahaha root@pee … thats hot
An alternative approach is to lock down passphraseless keys so they do exactly and only what they need, so that an attacker doesn’t actually get anything useful even if they do manage to steal the key.
The thing about needing to be an uber-hacker to get at the keys in memory is a resort to security through obscurity, which will encourage sloppy thinking about the real issue, which is that you in effect have passwordless keys on the system, so you should make sure that those keys only get to do what you want and nothing more. As shown in the above link, it’s possible to lock it down to the point that the keys only open up the tiny crack of read-only access from the right IP address, so an attacker really gets nothing from having such keys. I seriously doubt anyone using this keychain approach will bother with the
command=bit in theirauthorised_keysfile, which means that they’re giving an attacker much more if there is a break-in.Oh, and you should be setting
PermitRootLogintowithout-password, orno(rather than yes)I still receive :
Error:Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(454) [sender=2.6.9]
Hi
good post .i was wondering,what if i lost password in this case.
> With keychain or ssh-agent attacker won’t able to touch your remote
> systems without breaking your passphrase.
This is not true; this is false (at least in the case of ssh-agent): If I have file level access to the ssh-agent socket on a host where you are running ssh-agent, I can use (but not save) all of the keys that have been loaded into ssh-agent (unless you are locking the ssh-agent, which you say nothing about and which would defeat the nature of automated authentication).
Nice Article.. !!
keychain still asks me for my passphrase even after adding they the first time. Do this scenario covers passphrase protected ssh keys? If it is it’s not working for me, can you tell me how to do it?
Hi,
Thanks for the great article. I have a small issue, While trying to install in Centos 7 it shows “No package keychain available.” even after adding the RPMforge Repo. Please help me!
Today, the only way i was able to do this in a bash script via crontab was like that:
eval $(keychain --eval --agents ssh id_rsa id_dsa id_ed25519)source $HOME/.keychain/$HOSTNAME-sh
Need to reset for my key chain