RSA is safe. You should keep using it. Modern crypto is starting to use ECDSA because smaller key size means faster computation, but you wouldn't notice the difference in SSH anyway.
> If you're using SSH to move gigabytes of binary data (eg: rsync) you might.
No.
RSA or other asymmetric crypto is only used during connection establishment for authentication. Then symmetric key is established using DH and you use symmetric encryption to encrypt the data.
That makes perfect sense. Apologies for spreading incorrect information, and thanks for correcting me.
I remember changing around my ssh config a while back and seeing lower speeds when rsync'ing from an older laptop. But that makes sense, I recall cranking up my DH group to 16, 4096-bit, and that was probably why it was struggling.
What you do notice is using ciphers that are apt for the machine. If both ends are modern x86, for example, then you'd want to use aes256-gcm@openssh.com or aes128-gcm@openssh.com for best performance. Otherwise stick to chacha20-poly1305@openssh.com, which is the default of recent versions.
More info: https://wiki.mozilla.org/Security/Guidelines/OpenSSH