Do not do this in a rush - take a cafe and some time. Prepare your backup systems / routines in advanced.
Make sure have finished the setup until: https://docs.oxen.io/using-the-oxen-blockchain/oxen-service-node-guides/full-service-node-setup-guide#step-2-preparing-your-server (you need a new user with admin permission)
Remarks:
The deb installer will automatically create a _loki and a _lokinet user with a shared _loki group. They will be listed as unprivileged accounts that exists only to run loki/oxen services.
Maybe these names will be changed in further releases due to the renaming of LOKI to OXEN. You can briefly check the names in the terminal with compgen -u.
There is also a non-debian installer available: https://github.com/hesiod-project/oxen-rancher. Hereafter, the guide is tested with the deb installer only. Nevertheless, it should work with both versions just verify carefully the firewall settings (UFW).
Content
Setup some easy and important security settings on your Virtual Private Server (VPS)
This chapter is not mandatory to run a oxen service node but it is recommended to setup some fundamental security settings for your oxen service node. Keep in mind that the traffic between the oxen service nodes is encrypted (secure) but your oxen service node IP is and must be visible to operate.
Since this is made for beginners we highly recommend to follow strictly the process and use the latest Debian or Ubuntu server and use the latest Debian (deb) package. Furthermore, the local computer should also be a Debian distribution it should also work with other Linux OS but it has not been tested yet.
Needless to say, if you are familiar with “server hardening” you can add additional software and settings to secure your oxen service node.
Remark: Regardless of your security implementation it is always highly recommended to backup your VPS before doing any major changes to your system. Normally, this should be done with two clicks in the VPS control panel. To setup such a VPS recovery point just look for ‘backup’ or ‘snapshot’.
If you want a quick go-through you can directly jump to the Express Guide (time: approx. XX min).
Hereafter,
- username stands for your username on your oxen service node
- YourServerIP is your public oxen service node IP address which you got from your VPS provider (it is not your oxen service node public key address)
1. System update and auto-update
When you have your VPS ready and backup’ed login (again) with your ssh connection. This very simple and basic step secures your VPS to all known Ubuntu OS related vulnerabilities.
sudo apt update && apt upgrade && apt autoremove -y
sudo apt autoclean
Normally, you do not use your oxen service node on a daily bases like you do with your PC or mobile phone. Therefore, it is recommended to perform security updates automatically.
sudo apt-get install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
Open and check the auto-upgrade configuration file with :
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
It should look like this:
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
// Extended Security Maintenance; doesn't necessarily exist for
// every release and this system may not have it installed, but if
// available, the policy for updates is such that unattended-upgrades
// should also install from here by default.
"${distro_id}ESMApps:${distro_codename}-apps-security";
"${distro_id}ESM:${distro_codename}-infra-security";
// "${distro_id}:${distro_codename}-updates";
// "${distro_id}:${distro_codename}-proposed";
// "${distro_id}:${distro_codename}-backports";
};
Exit the file with ‘ctrl+x’.
Remark: It is recommended to update your oxen deb package manually and check directly that all is running as expected. For this you can do the default all update routine (sudo apt update && sudo apt upgrade -y).
2. System scan
Like an anti-virus software for Windows you should install and check your Linux system on so called rootkits. This can be easily done with rkhunter (alternatively with chkrootkit)
Install with :
sudo apt install rkhunter -y
A package configuration window will pop up. Select “No configuration” and move to "Ok". Back in the terminal to a checksum update :
sudo rkhunter --propupd
Run a system scan :
sudo rkhunter --check
During the scan you have to click 'enter' to pass the scan sections.
You will notice a Warning saying:
Checking if SSH root access is allowed [ Warning]
We will fix this security hole in a few minutes.
As a summary it should output something like this :
System checks summary
=====================
File properties checks...
Files checked: 142
Suspect files: 0
Rootkit checks...
Rootkits checked : 496
Possible rootkits: 0
Applications checks...
All checks skipped
The system checks took: 2 minutes and 14 seconds
All results have been written to the log file: /var/log/rkhunter.log
Regularly, repeat this scan (i.e. after a system update routine)
3. Firewall
By default, Ubuntu comes with the Uncomplicated FireWall (UFW). It’s default setting is inactive. Before you activate UFW you have to ensure that ssh connection is allowed.
Allow ssh connection to your VPS :
sudo ufw allow ssh
Launch the UFW by entering :
sudo ufw enable
UFW will show you a ssh-warning which you can now accept by typing a ‘yes’ and hit Enter to continue.
You should close all ports and explicitly open all ports which oxen service nodes requires to operate with the oxen service node network. Luckily, the deb package is doing the job for you.
To ensure that all required ports are open you can command the following:
sudo ufw allow 1090/udp
sudo ufw allow 22020/tcp
sudo ufw allow 22021/tcp
sudo ufw allow 22022/tcp
sudo ufw allow 22023/tcp
sudo ufw allow 22025/tcp
Remark:
As long as you are using the standard ports. This port rules are defined in /etc/ufw/applications.d (for the deb package only).
Further explanations: Inbound traffic should allow following ports:
- 1090/udp for the lokinet-router
- 22020/tcp for storage server: storage server talking to other storage server.
- 22021/tcp for storage server: for remote clients to talk to storage server
- 22022/tcp for oxend p2p
- 22025/tcp for quorum/blink communication
Only for localhost (does not need to be externally accessible) 22023/tcp for lokid RPC
Outgoing traffic should be allowed for all ports
If you want to run a testnet node, following ports should be open for inbound traffic:
- ports 38156, 38159/tcp for oxen-daemon
- ports 38154, 38155/tcp for oxen-storage-server
Optionally, if you do not need ipv6 connection you can delete this rules by modifying the general UFW settings.
sudo nano /etc/default/ufw
and set:
IPV6=no
Afterwards save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’. To make the changes effective you need to restart the UFW by entering:
sudo ufw reload
Verify and double-check that UFW is working properly and all ports are set correctly:
sudo ufw status
3.1 Firewall - Switch SSH port
Optionally: ssh default port is 22 so if you start from a freshly installed Ubuntu you can just proceed. If you want to switch the ssh port you have to explicitly allow this port to get your ssh connection established. If you want to use the default port you can directly move to 4. SSH Keys.
Following the guidance of ICAAN (Internet Corporation for Assigned Names and Numbers) we should pick a number between 49152 and 65535.
sudo nano /etc/ssh/sshd_config
Look for
# Port 22
Delete the comment sign "#" and replace the 22 with your new port number i.e. 55555
Port 55555
Afterwards save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’.
Important: To avoid completely locking yourself from your system, you should whitelist the port that you have specified above on your firewall.
Remark: Replace the port number 55555 with your number you have defined in the setting below.
sudo ufw allow 55555/tcp
Important: You have to restart your sshd to make it active.
sudo service sshd restart
Important: Double check that the changes work as expected. To do this do not log out from your current SSH session (do not close the terminal). Open a new Terminal window, use the new SSH port that you have specified above and open a new ssh session to your VPS. This ensures that you don’t completely lock out yourself from your VPS in case of typo, wrong setting or additional firewall settings.
Type this in your new Terminal window:
ssh -p <yourNewSSHportNumber> <username>@<yourVPSip>
If this works you can purge the UFW settings. Firstly, close the old Terminal (the one which you logged in first) by command simply :
exit
and keep the new Terminal window (which you logged in with your new SSH port number).
Delete the default (inactive) SSH port 22:
sudo ufw delete allow 22/tcp
and afterwards check the UFW status with:
sudo ufw status verbose
It should provide you this status report:
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
55555/tcp ALLOW IN Anywhere
55555/tcp (v6) ALLOW IN Anywhere (v6)
4. SSH Keys
Launch a new terminal on your local computer and type the following to generate your oxen service node ssh keys.
ssh-keygen -t ed25519 -a 500 -f ~/.ssh/oxensn_key
Explanation:
-t: Specifies the type of key. Here we select Ed25519 which is a new and modern algorithm. A key alternative is RSA (4096-bit is recommended.) DSA and ESDSA are not recommended.
-a: Defines the Key Derivation Function (KDF) rounds. A higher number increases the resistance to brute-force attacks but is will also slow down the algorithm.
-f: Specify the filename of the generated key file. When you work with several ssh keys it is recommended to define a unique (self-explaining) file name instead of the default name.
You will be asked to type a secure passphrase.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
This is again optionally, you can leave it blank but to increase security it is recommended to add a strong passphrase to set an additional protection layer. When you use additional Authentifcation methodes like One-Time-Password (OTP) you can keep it blank.
Remark: You can easily change this afterwards by command
sudo ssh-keygen -p -f ~/.ssh/oxensn_key -P <oldpassword> -N <newpassword>
Your ssh keys are stored in a hidden folder (on your local computer). Move to the ssh key folder by command:
cd ~/.ssh/
List the content:
ls -l
You should see your private ssh key, named oxensn_key and your public ssh key oxensn_key.pub. The private key should never be shared. Keep it save and backup'ed. The public key is free to share. Since we want to use the keys to login into our VPS we need to copy the public key to our server.
ssh-copy-id -i ~/.ssh/oxensn_key.pub -p <yourNewSSHportNumber> <username>@<YourServerIP>
Remark: If you use a ssh-agent like Special: Qubes OS - split SSH you need to set an additional -f before -i.
Afterwards you will ask to enter your username password (not the key passphrase).
The terminal should reponses with:
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/<username>/.ssh/oxensn_key.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
<username>@<YourServerIP>'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -p '<yourNewSSHportNumber>' '<username>@<YourServerIP>'"
and check to make sure that only the key(s) you wanted were added.
Now that the public key is transferred to your VPS, the connection can be tested from your local computer. In this case, it is important that the server does not ask for the user password, but of course the passphrase of the protected key is required.
ssh -p <yourNewSSHportNumber> <username>@<YourServerIP>
After entering the key passphrase you are authenticated on your service node. Next, let's limit the VPS access to the ssh key:
Open the main ssh configuration file :
sudo nano /etc/ssh/sshd_config
Search (ctrl.+w) for the following line in the file.
#PermitRootLogin yes
Remove the ‘#‘ from the beginning of the line. Make the line look like similar to this.
PermitRootLogin no
Important: You need to have a user with admin permission. See oxen guide: https://docs.loki.network/ServiceNodes/SNFullGuide/#21-create-a-non-root-user.
Next, set ssh key login only
search (ctrl.+w) for 'PubkeyAuthentication' in the ssh configuration file (/etc/ssh/sshd_config) and set the following:
PubkeyAuthentication yes
and
PasswordAuthentication no
Afterwards save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’. To make the changes effective you need to restart the SSH daemon by entering:
sudo /etc/init.d/ssh restart
Since no root login is possible anymore and you need to login as normal user (with your 'username' and then use ‘su’ command to switch to root user).
Optionally: When you have several nodes you may want to easy the login to your VPS. For this you can briefly make some setting in your ssh configuration.
...
On your local computer open:
sudo nano ~/.ssh/config
and copy&paste the following
Host oxensn01
HostName <YourServerIP>
User <username>
Port <yourSSHportNumber>
IdentityFile ~/.ssh/oxensn_key
IdentitiesOnly yes
As usual, modify the content with your 'username' and 'YourServerIP'.
Save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’.
Now, you can setup your ssh connection by simply typing:
ssh oxensn01
Backup time!
5. Multi-Factor-Authentication (MFA)
This chapter will show you how to set up a MFA in combination with your ssh key.
Remark: libpam-google-authenticator is open-source server software. The Google Authenticator mobile app is not open-source. So, do not use the mobile app. You can use open-source alternatives like FreeOTP or if you already have a Yubikey you can use Yubico Authenticator.
Log into your VPS and run the following command to install Google Authenticator :
sudo apt install libpam-google-authenticator -y
When it is done, simply type
google-authenticator
to launch your personalized 2FA / MFA setup.
You can answer all questions with "yes". Afterwards you will get your unique QR code displayed in your terminal window. You may have to increase the terminal size to see it completely. Next, pick your OTP software (i.e. FreeOTP) and scan with your camera the QR code.
Important: As usual, take your time to backup properly all info below your QR code. Additionally, you can make a snapshot and store the image securely. With this you can easily add the 2FA to a new devices.
Next, we need to configure ssh daemon to use the Google Authenticator.
Open your ssh server configuration file :
sudo nano /etc/ssh/sshd_config
Search for 'ChallengeResponseAuthentication' and 'UsePAM' and set both to 'yes' :
ChallengeResponseAuthentication yes
UsePAM yes
at the end of the file :
#One-time password authentication via Google Authenticator
AuthenticationMethods publickey,keyboard-interactive:pam
Switch to the PAM rule file and activate the 2FA:
sudo nano /etc/pam.d/sshd
Add a "#" at the line:
#@include common-auth
to remove one layer of authentification - no user password.
Or
@include common-auth
to set user password as mandatory - one more security layer.
And add this to the end of the file :
#One-time password authentication via Google Authenticator
auth required pam_google_authenticator.so
Afterwards save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’ and restart the SSH daemon.
sudo systemctl restart ssh
Conclusion and recommentation:
This will give you a Multi-Factor-Authentification (MFA) of:
- ssh-key (1FA)
- optionally SSH-key passphrase (2FA)
- optionally user password (3FA)
- OTP (4FA)
If you trust your OTP (no Google, no Microsoft authentification app) you should be fine with ssh-key (without passphrase) and OTP. Final thoughts, using a Yubikey with Yubico-Authenticator provides a very secure and convenient solution for OTP and can be also use securely on Windows, macOS, Linux, iOS and Android systems.
If you own a Yubikey you can also follow this guides to use your Yubikey as SSH-key. Compare to the Yubico-Authenticator setup this is more complex to setup.
6. Fail2ban
Fail2ban is a very powerful tool to protect your oxen service node against brute force authentication attempts. For this beginner tutorial will keep it simple and keep all default settings.
sudo apt install fail2ban
Fail2ban launch automatically you can check the status by entering :
sudo systemctl status fail2ban
(7. TBD: Secure Shared Memory)
Shared memory can be used in an attack against a running service. It is recommended to secure this part of system memory. The protection is quickly done with some modification in the /etc/fstab file:
Open the fstab file by command :
sudo nano /etc/fstab
Add the following line to the end :
tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0
Save the file with ‘ctrl+o’ hit enter and exit with ‘ctrl+x’.
Afterwards you need to reboot your system:
sudo reboot
8. Backup securely all keys, passwords and passphrases !
Beside the backup of all your passwords and ssh key you should also backup your oxen service node key.
sudo oxen-sn-keys show /var/lib/loki/key_ed25519
Copy secret key into a safe place!
When you have no access to your VPS or to your oxen service node you can move to a new VPS or system by restoring your oxen service node with your oxen service node key (ed25519).
Restore key: Rename existing ed25519-key before restoring:
sudo mv /var/lib/loki/key_ed25519 /var/lib/loki/key_ed25519.old
sudo oxen-sn-keys restore /var/lib/loki/key_ed25519
Give the secret key when asked. Afterwards make sure that all services work properly.
9. Express Guide (not ready yet)
VPS login
System updates
System scan
Firewall
Firewall - Switch SSH port
SSH Keys
10. RPi, Tails, Qubes OS - split SSH qube
As writting here security is not just a one click operation - it is all about security layers. Furthermore, you should not ignore other weak spots. Having a super secure VPS and a rootkit or any malicious software on your admin computer can bypass all security you put into your VPS. The best setup you can use to securely login and admin your harded VPS is a seperated computer like Raspberry Pi (RPi) or any other Single Board Computer (SBC) - based on open source software and limited usage for only one porpose. Tip: A good software source can be found here: https://dietpi.com/. Another alternative you can think of is https://tails.boum.org/
A more secure and elegant approach offers https://www.qubes-os.org/. When you follow this chat / guide you can simply setup a split SSH qube which can A) more secure than a smard card (depends on your risk scenario) B) combine this with KeePassXC (runs only within a vault virtual machine, without network access).
https://github.com/Qubes-Community/Contents/blob/master/docs/configuration/split-ssh.md
11. Mobile SSH access to Oxen Service Node
... ...
Respect privacy - Have fun and stay safe !
Table of Contents
- Content
- Setup some easy and important security settings on your Virtual Private Server (VPS)
- 1. System update and auto-update
- 2. System scan
- 3. Firewall
- 4. SSH Keys
- 5. Multi-Factor-Authentication (MFA)
- 6. Fail2ban
- (7. TBD: Secure Shared Memory)
- 8. Backup securely all keys, passwords and passphrases !
- 9. Express Guide (not ready yet)
- 10. RPi, Tails, Qubes OS - split SSH qube
- 11. Mobile SSH access to Oxen Service Node