Setting up SSH securely: the first tasks on a new VPS

A new VPS arrives in a state nobody should leave it in for long: a root account, reachable over the internet, usually with a password rather than a key. Most providers hand it to you this way because it is the simplest thing to automate, not because it is a sensible place to stop. The first hour with a new server is for closing that gap, before anything else gets installed on it.

This post covers three things in the order we recommend doing them: creating a proper sudo user, setting a strong password for it, and locking SSH down so root cannot log in with a password at all. None of it is complicated. All of it matters.

Why this comes before anything else

Internet-facing SSH on port 22 gets probed constantly, usually within minutes of a new IP address going live. Most of that traffic is automated scanning for exactly the default state described above: root, password authentication, no rate limiting. Reducing that exposure is one of the highest-value changes you can make to a new server, and it takes less time than installing the first application you actually wanted the VPS for.

Step 1: create a sudo user

Working as root day to day is worth avoiding even once SSH is locked down, since a single mistyped command has no safety net. Start by creating a named account and adding it to the sudo group. At this point you are still logged in as root, since it is the only account that exists, so run these directly rather than with sudo:

adduser yourusername
usermod -aG sudo yourusername

Log out and back in as that user, then confirm sudo actually works before you rely on it:

su - yourusername
sudo whoami

That last command should return root. If it asks for a password you never set, or fails outright, sort that out now rather than after root login has been disabled.

Step 2: set a strong password

The new account still needs a password even if you intend to log in with a key almost immediately, since sudo will ask for it and some recovery paths depend on it. Length matters more than complexity here. A long, randomly generated passphrase from a password manager beats a short password with substituted characters every time, and it is easier to type correctly under pressure.

On Linux you can generate a reasonable random passphrase locally without relying on a web tool:

openssl rand -base64 24

Set it with:

passwd yourusername

Step 3: move to SSH key authentication

Passwords over SSH, even strong ones, are still vulnerable to being guessed given enough attempts, and every password sent is one more thing that can be intercepted or phished. A key pair removes that risk almost entirely, since the private key never leaves your machine.

Generate a key on your local machine if you do not already have one:

ssh-keygen -t ed25519 -C "your-email@example.com"

Copy the public key across to the server:

ssh-copy-id yourusername@your-server-ip

Then test it in a fresh terminal window, without closing your existing session:

ssh yourusername@your-server-ip

Step 4: lock down sshd_config

Once key-based login for your sudo user is confirmed working, the remaining step is telling SSH itself to stop accepting the things you no longer need. Edit /etc/ssh/sshd_config (or a drop-in file under /etc/ssh/sshd_config.d/ on newer systems) and set:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

It is also worth being explicit about who is allowed to connect at all, rather than leaving it open to any account on the system. Add a line naming the specific users permitted to log in over SSH:

AllowUsers yourusername anotherusername

Replace those placeholders with real account names on your server. For example, if your username is bobward, the line becomes:

AllowUsers bobward

Add more names separated by spaces if several people need access, for example AllowUsers bobward alicetan. Anyone not on that list is refused at the SSH level, before a password or key is even checked. If you manage access by group rather than by naming individuals, AllowGroups does the same job:

AllowGroups sshusers

That requires the group to exist and your accounts to be members of it:

sudo groupadd sshusers
sudo usermod -aG sshusers yourusername

Restart the SSH service to apply the change:

sudo systemctl restart ssh

Test again from a new terminal before closing anything. If a new connection succeeds with your key and root or password login is refused, the change has taken effect. Try connecting as an account not on the allow list too, to confirm it is actually rejected.

Worth doing soon after

The three steps above are the minimum for a first session on a new server. A few related changes are worth scheduling for shortly afterwards rather than treating as optional:

• A firewall with a default-deny policy on inbound traffic, allowing only the ports you actually use
• An intrusion prevention tool that blocks IPs showing repeated failed login attempts
• Moving SSH off port 22 if you want to cut down on automated scan noise, understanding this is obscurity rather than a security control on its own

These reduce risk further, but they are refinements. The account, password, and key changes above are the ones that close the obvious gap.

Where this fits into ongoing security

None of this makes a server secure in any permanent sense. Security is not a state you reach and leave; it is an ongoing process of reducing risk as circumstances, software, and threats change. Locking down SSH on day one removes the most obvious weakness, not every weakness, and it is worth revisiting your configuration periodically rather than assuming a good first setup holds indefinitely.

If you would rather have someone experienced walk through this with you, or review a server you have already set up, that is exactly the kind of thing we advise on at Techster Consulting.