How to Set Up SSH Keys on Linux VPS

Logging into your Linux Virtual Private Server (VPS) with standard passwords leaves your system exposed to automated dictionary scans and brute-force password attacks. SSH Keys provide a modern, cryptographically secure alternative using asymmetric public-key cryptography. With SSH keys, you can log in instantly without typing a password while ensuring that only authorized devices possessing the private key can access your server.

Whether your server is running Ubuntu, Debian, AlmaLinux, Rocky Linux, RHEL, or CentOS, setting up SSH key authentication is one of the most critical security hardening steps you can take on your Aveshost VPS.

In this step-by-step tutorial, you will learn how to generate modern ED25519 and RSA SSH key pairs, copy your public key to any Linux distribution, adjust SELinux contexts on RHEL/AlmaLinux, and safely disable password authentication.

Prerequisites

• An active Linux VPS running Ubuntu, Debian, AlmaLinux, Rocky Linux, RHEL, or CentOS.
• SSH access to your server (see How to Log in and Access Linux VPS via SSH).
• A local computer running Windows 10/11, macOS, or Linux.

Step-by-Step Guide: How to Set Up SSH Keys on Linux VPS

Step 1: Generate an SSH Key Pair on Your Local Computer

Open your local terminal (PowerShell/Command Prompt on Windows, Terminal on macOS/Linux) and generate an SSH key pair. We recommend using the modern ED25519 algorithm for superior performance and cryptographic security:

# Recommended: ED25519 Algorithm
ssh-keygen -t ed25519 -C "[email protected]"

# Alternative: 4096-bit RSA Algorithm (for legacy environments)
ssh-keygen -t rsa -b 4096 -C "[email protected]"

Press ENTER to accept the default file location (~/.ssh/id_ed25519) and optionally enter a passphrase to encrypt your private key locally.

Step 2: Copy the Public Key to Your Linux VPS

Method A: Using ssh-copy-id (Recommended for macOS, Linux, and Git Bash)

ssh-copy-id username@YOUR_SERVER_IP
# Example for root:
ssh-copy-id root@YOUR_SERVER_IP

Enter your server password when prompted. The utility will automatically append your public key to the remote server’s ~/.ssh/authorized_keys file with the correct permissions.

Method B: Manual Installation (Windows PowerShell / Command Prompt)

1. Display your public key content on your local computer:

   cat ~/.ssh/id_ed25519.pub

2. Copy the entire output starting with ssh-ed25519 AAAAC3NzaC1lZDI1NTE5....
3. Connect to your Linux server via SSH and create the .ssh directory:

   mkdir -p ~/.ssh
   chmod 700 ~/.ssh
   nano ~/.ssh/authorized_keys

4. Paste your public key on a new line, save, and exit (CTRL + O, ENTER, CTRL + X).
5. Lock down file permissions:

   chmod 600 ~/.ssh/authorized_keys

Step 3: Fix SELinux Context (AlmaLinux, Rocky Linux & RHEL)

If your VPS runs an enterprise RHEL-based distribution with SELinux in Enforcing mode, ensure the SSH directory has the proper SELinux security context:

restorecon -Rv ~/.ssh

Step 4: Test SSH Key Authentication

Open a NEW terminal window on your local computer without closing your existing session and test logging in:

ssh username@YOUR_SERVER_IP

You should connect directly without being prompted for your Linux account password!

Step 5: Disable Password Authentication on the Server

Once you have verified key authentication, disable password login across the SSH daemon:

1. Open the SSH configuration file:

   sudo nano /etc/ssh/sshd_config

2. Set PasswordAuthentication to no and ensure PubkeyAuthentication is yes:

   PasswordAuthentication no
   PubkeyAuthentication yes

3. Test syntax and restart the SSH service:

   # Validate syntax:
   sudo sshd -t
   
   # On Ubuntu / Debian:
   sudo systemctl restart ssh
   
   # On AlmaLinux / Rocky Linux / RHEL / CentOS:
   sudo systemctl restart sshd

Frequently Asked Questions & Troubleshooting

Q: What if I lose my private key?
A: Use the VNC / Web Console in your Aveshost Client Area to access the emergency terminal and upload a new public key.

Q: Why does the server still ask for a password after copying the key?
A: Incorrect directory or file permissions are the most common cause. Verify that ~/.ssh is set to 700 and authorized_keys is set to 600. On AlmaLinux/Rocky Linux, run restorecon -Rv ~/.ssh.

Need Further Assistance?

If you have questions about configuring SSH keys or troubleshooting access permissions on your Linux VPS, our technical support team is available 24/7. Feel free to submit a support ticket for prompt assistance.