Remote server setup: Check the following lines exists and uncommented in the /etc/ssh/sshd_config, if not you must add it and restart ssh server at the remote side:
Local setup: You must generate an public/private key pair for your own if it's not generated before:
ssh-keygen -t rsa
You can select defaults just hitting enter a few times. After that your public key will be in the ~/.ssh/id_rsa.pub file, you must add this file content's to ~remote-user/.ssh/authorized_keys file. You can scp to copy id_rsa.pub at the remote side and after that you can append id_rsa.pub contents into authorized_keys file at the remote side.
This is a trivial process and there is a special script utility which basically makes this copy and append task called ssh-copy-id:
Now, you can login without entering password