Reasoning

I've been doing some development where I need to connect with a different username on the same host, depending on what I'm developing.

Rather than using the same key for all the users (which will work fine), I created a key for each user/host combo.

It also makes it more practical to revoke a single lost key, whether the key is lost or the project is cancelled.

Creating the Key

Use the ordinary ssh-keygen command to generate the key, but set the filename and comment, as needed.

I've used planetary references, rather than my actual project and hostnames.

This tells me the first key is used to login as mars, on the host solar, from my local user mimas. The second key is used to login to mimas on the host saturn, from my local user mimas.

ssh-keygen -f id_mars@solar -C "mimas->mars@solar"
ssh-keygen -f id_mimas@saturn -C "mimas->mimas@saturn

SSH Config File

Next, setup the .ssh/config file. The config file should set the SSH Host alias and SSH User values correctly.

Host solar
  Hostname solar
  User mars

Host saturn
  Hostname saturn.solar
  User mimas

Host *
  IdentityFile ~/.ssh/id_%r@%k

The use of %r (remote username) and %k (host key alias) is what makes SSH use the correct key, based on the remote user and host alias.

The keys must still be deployed on the remote system, either with a configuration management tool (Ansible, Chef, Puppet, etc.) or manually.

Other configurations can also be used in the file -- and if necessary the IdentityFile can be set for each host alias, arther than globally.

Rembember, in .ssh/config, the first match wins.