Run scp or sftp Without a Password

To automate a job for exchanging data files between two servers, it is asked to run it without entering a password by human being.  Assume there are two servers local2d (with users batchdev and batchmgr) and remote1d (with user user01).  Below steps worked on RHEL5. 

A. if a passphase was not entered for "ssh-keygen"
1. on the local server local2d

batchdev@local2d: /u06/app
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter>   <== option: another path/name
Enter passphrase (empty for no passphrase): <Hit enter>           <== leave it empty
Enter same passphrase again: <Hit enter>
Your identification has been saved in /u06/app/.ssh/id_rsa.
Your public key has been saved in /u06/app/.ssh/id_rsa.pub.
The key fingerprint is:
66:38:f8:65:74:5d:10:88:f9:9a:3e:89:4a:d5:0c:49 batchdev@local2d.domain.com
The key's randomart image is:
+--[ RSA 2048]----+
|      E  o .oo.  |
|     . .o .. .   |
|      o ... .    |
|     . * ..      |
|    . + So       |
|     o *o        |
|    . .o .       |
|   .  . +        |
|    ..   .       |
+-----------------+
Note: If run "$ ssh-keygen -t dsa", it will generate two files with names id_dsa and id_dsa.phb

batchdev@locald2d: /u06/app
$ cd .ssh
$ ls -al
-rw-------  1 batchdev users 1675 Oct 13 11:46 id_rsa
-rw-r--r--  1 batchdev users  410  Oct 13 11:46 id_rsa.pub
-rw-r--r--  1 batchdev users 1622 Aug  6 10:50 known_hosts

2. On the remote server remote1d
$ hostname
remote1d.domain.com
$ whoami
user01

## copy the pub key to remote1d (from server local2d).
$ cd /u04/apps/.ssh
$ scp -p batchdev@local2d:/u06/app/.ssh/id_rsa.pub local2d.key
batchdev@local2d's password: xxxxx

$ mv local2d.key authorized_keys                <== do not overwrite the file!
## Note if the file exists, run "$ cat local2d.key >> authorized_keys"
$ chmod 600 authorized_keys

$ ls -al /u04/app/.ssh
-rw-------  1 user01 users  410    Oct 13 11:46 authorized_keys  
-rw-r--r--  1 user01 users  2087  Mar 30  2015 known_hosts

3. Test SFTP and SSH on local server local2d

batchdev@local2d: /u06/app/  

$ sftp remote1d                             ## batchdev can not get onto it. sftp asks the password.
Connecting to remote1d...
Password:

$ sftp user01@remote1d      ## But login to remote1d as user01 works without a password!
Connecting to remote1d...
sftp> dir -all
-rw-r--r--    1 user01  users     481 Jun 22  2010 .profile
-rw-------    1 user01  users     933 Oct 13 12:35 .sh_history
drwx------   2 user01  users     096 Oct 13 12:27 .ssh
sftp> quit

batchdev@local2d: /u06/app  
$ ssh user01@remote1d            ## ssh to remote1d as user01 also works!
$ hostname
remote1d.domain.com
$ exit

batchdev@local2d: /u06/app
$ scp user01@remote1d:/u04/app/cert.txt .   ## even SCP does not ask for the password
cert.txt                                                   100% 1383     1.4KB/s   00:00 

TROUBLESHOOTING: permission on /u06/app and /u04/app should be 755 or 700. 777 will make passwordless authentication not work. Needs more restrictive.

If it gives below warnings:

$ ssh user01@remote1d

The authenticity of host 'remote1d (167.xx.xx.xxx)' can't be established.

ECDSA key fingerprint is SHA256:r4ofQUP1F8ebW5hWCcCsK7ah...A.

ECDSA key fingerprint is MD5:49:86:cd:21:a3:a4:22:05:68:4a:0e:...:35.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'remote1d' (ECDSA) to the list of known hosts.

Warning: the ECDSA host key for 'remote1d' differs from the key for the IP address '167.xx.xx.xxx'

Offending key for IP in /u06/app/.ssh/known_hosts:20

Are you sure you want to continue connecting (yes/no)? yes

First, answer Yes, and delete line 20 from file /u06/app/.ssh/known_hosts (after doing a backup) on local2d. 

Then, try ssh to login twice. The warning will go away in the 2nd try.

$ ssh user01@remote1d

Warning: Permanently added the ECDSA host key for IP address '167.xx.xx.xxx' to the list of known hosts.

$ ssh user01@remote1d

Now, no more warnings

B. If there is a 2nd user batchMgr on server local2d wants to run sftp into remote1d, additional steps are necessary:
1. Login to local2d server with batchMgr account
2. Create a new directory apps_sftp_key and do a chmod 700 on this directory
3. Copy files id_rsa and id_rsa.pub from ~batchdev/.ssh to apps_sftp_key directory and do a chmod 600 on id_rsa

$ hostname
local2d
$ whoami
batchmgr
$ pwd
/home/batchmgr/apps_sftp_key
$ ls -al
total 16
drwx------  2 batchmgr 1211 4096 Sep 16 14:52 .
drwxrwxr-x 12 batchmgr 1211 4096 Sep 19 12:23 ..
-rw-------  1 batchmgr users 1675 Oct 13 11:46 id_rsa
-rw-r--r--  1 batchmgr users  410  Oct 13 11:46 id_rsa.pub 

One line to run SFTP without interactive:

batchmgr@locald2d: $ sftp -oport=22 -b sftp.ctl -oIdentityFile=~batchmgr/apps_sftp_key/id_rsa user01@remote1d:/apps/local/ftp/in 
Changing to: /apps/local/ftp/in
sftp> put /home/batchmgr/temp/test.del test.del
Uploading /home/batchmgr/temp/test.del to /apps/local/ftp/in/test.del
sftp> version
SFTP protocol 3
sftp> quit

Here, sftp.ctl is a command file in the current directory:
$ more sftp.ctl
put /home/batchmgr/temp/test.del test.del
version
quit 

C. If a passphrase was entered for "ssh-keygen"

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter>
Enter passphrase (empty for no passphrase):    <== "welcome01" works, but "welcome" not
Enter same passphrase again:<Enter your passphrase again>   <== welcome01

Additional steps on ssh agent are needed on the local2d to make it work:

batchdev@local2d: /u06/app
$ echo $SHELL
/usr/bin/ksh
$ ssh-agent $SHELL
$ ps -ef | grep ssh-agent
  batchdev 118621 174327   0 17:32:17       -   0:00 ssh-agent /usr/bin/ksh
  batchdev 161219 174327   0 17:32:28  pts/0  0:00 grep ssh-agent

$ ssh-add
Enter passphrase for /u06/app/.ssh/id_rsa: welcome01
Identity added: /u06/app/.ssh/id_rsa (/u06/app/.ssh/id_rsa)

Now, sftp, ssh and scp shall work without a password or a passphase.