Linux


General Info

General Info

Directory Structure

 

 

Directory Purpose
/ Root of the filesystem
/bin Essential command binaries
/sbin Essential system binaries
/etc Configuration files
/var Variable data (logs, cache, etc.)
/var/log System and application logs
/usr User applications and binaries
/home User home directories
/root Root user’s home directory
/boot Boot files and kernel
/opt Optional third-party software
/tmp Temporary files
/dev Device files (disks, terminals)
/proc Process and system info
/sys Kernel-related info
/lib, /lib64 Shared libraries
/media Auto-mounted removable media
/mnt Temporary mount points
/srv Data for services (web, FTP)

Web server management

Web server management

Apache setup & Virtual servers

Initial setup

Install packages sudo apt install apache2 mariadb-server libapache2-mod-php php-mysql -y

Open ports at  cd /etc/apache2/ports.conf and add the line Listen NEW_PORT_NUMBER

Creating virtual servers

Create a file named NAME.conf and add something like this

<VirtualHost *:80>
        ServerName pdb.conorbriggs.com.au
        DocumentRoot /mnt/drives/10tb/10tb/web_servers/pdb/

<Directory /mnt/drives/10tb/10tb/web_servers/pdb/>
        Options -Indexes -FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>
ErrorLog /var/log/apache2/pdb-error.log
CustomLog /var/log/apache2/pdb-access.log combined

RewriteEngine on
</VirtualHost>

Enable the site & restart apache with sudo a2ensite site.conf && systemctl restart apache2


SSL Certificate management

Install certbot and run setup

sudo apt install certbot python3-certbot-apache
a2enmod ssl
sudo certbot --apache
List all certificates with certbot certificates
Delete a certificaite certbot delete --cert-name CERT_NAME_HERE

Managing the apache2 service

Start, stop & restart sudo service apache2 start/restart/stop

Enable a config sudo a2ensite SITE_CONF_FILE.conf
Disable a config sudo a2dissite SITE_CONF_FILE.conf
Web server management

SQL

After install

ensure secure access sudo mysql_secure_installation

 Logging in 

mysql -u USERNAME -p

add -h HOSTNAME IP when logging in remotely

Creating a user

CREATE USER 'NEW_USERNAME'@'localhost' IDENTIFIED BY 'NEW_PASSWORD';

Creating a database

CREATE DATABASE DATABASE_NAME

Allow user to modify Database

GRANT ALL PRIVILEGES ON DATABASE_NAME.* TO 'USERNAME'@'localhost';

Update permissions

FLUSH PRIVILEGES;

Open Database to the world

  1. Open file at /etc/mysql/maridadb.conf.d/50-server.cnf
  2. Edit line bind-address = 127.0.0.1 to -> bind-address = 0.0.0.0
  3.  Restart db service systemctl restart mariadb
  4. Allow port through firewall with ufw allow 3306

Automated backup script

#!/bin/bash

# === Config ===
BACKUP_DIR="/var/backups/mysql"
DATE=$(date +"%Y-%m-%d_%H-%M")
DB_USER="backupuser"
DB_PASS="yourpassword"

# Create backup dir if not exist
mkdir -p "$BACKUP_DIR"

# Dump all databases
mysqldump -u "$DB_USER" -p"$DB_PASS" --all-databases --single-transaction > "$BACKUP_DIR/mariadb-$DATE.sql"

# Optional: Compress the backup
gzip "$BACKUP_DIR/mariadb-$DATE.sql"

# Delete backups older than 7 days
find "$BACKUP_DIR" -name "*.sql.gz" -type f -mtime +7 -delete

 

 

 

 

 

 

 

 

Web server management

SSL & Certificate management

Install certbot and run setup

sudo apt install certbot python3-certbot-apache
a2enmod ssl
sudo certbot --apache

List all certificates

certbot certificates

Delete a certificaite

certbot delete --cert-name CERT_NAME_HERE

Web server management

PostgreSQL

Install

Install postgresql and postgis for coordinates data

sudo apt install postgresql postgis -y

 

Login

sudo -u postgres psql

 

Change password to default account

ALTER USER postgres WITH PASSWORD 'new password';

 

Restrict remote access to localhost only

Open the config file

nano /etc/postgresql/15/main/postgresql.conf

uncomment this line

listen_addresses = 'localhost'

if you need remote access

listen_addresses = '0.0.0.0'

and Edit /etc/postgresql/15/main/pg_hba.conf and add a line like:

host    all             all             192.168.1.0/24          scram-sha-256

 

Web server management

SQL Reference

Create a table that auto deletes on other table deletion

CREATE TABLE tradie_postcode_covered (
    tradie_id INT,
    postcode VARCHAR(10),
    PRIMARY KEY (tradie_id, postcode), -- primary key ensures no tradie can be in the same postcode twice
    FOREIGN KEY (tradie_id) REFERENCES tradies(id) ON DELETE CASCADE
);

Web server management

Mail Server

Initial setup

1 - Updates
sudo apt update && sudo apt upgrade -y

2 - Install docker and docker compose
# Install required packages
sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Set up the stable repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Add your user to docker group
sudo usermod -aG docker $USER

3 - Create DNS Records

4 - Install and configure mailserver
# Create directory for mail server
mkdir -p ~/mailserver
cd ~/mailserver

# Download docker-compose.yml and .env template
wget https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master/compose.yaml
wget https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master/mailserver.env

# Rename for easier use
mv compose.yaml docker-compose.yml
mv mailserver.env .env

5 - Edit .env variables

6 - Edit docker-compose.yml
services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest
    container_name: mailserver
    # Provide the FQDN of your mail server here (Your DNS MX record should point to this value)
    hostname: mx.home.conorbriggs.com.au
    env_file: .env
    # More information about the mail-server ports:
    # https://docker-mailserver.github.io/docker-mailserver/latest/config/security/understanding-the-ports/
    ports:
      - "25:25"    # SMTP  (explicit TLS => STARTTLS, Authentication is DISABLED => use port 465/587 instead)
      - "143:143"  # IMAP4 (explicit TLS => STARTTLS)
      - "465:465"  # ESMTP (implicit TLS)
      - "587:587"  # ESMTP (explicit TLS => STARTTLS)
      - "993:993"  # IMAP4 (implicit TLS)
    volumes:
      - ./mail-data/:/var/mail/
      - ./mail-state/:/var/mail-state/
      - ./mail-logs/:/var/log/mail/
      - ./config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
      - /etc/letsencrypt:/etc/letsencrypt:ro
    restart: always
    stop_grace_period: 1m
    # Uncomment if using `ENABLE_FAIL2BAN=1`:
    cap_add:
      - NET_ADMIN
    healthcheck:
      test: "ss --listening --ipv4 --tcp | grep --silent ':smtp' || exit 1"
      timeout: 3s
      retries: 0
    networks:
      - mailserver-network
networks:
  mailserver-network:
    driver: bridge

Connecting

IMAP (Incoming):

SMTP (Outgoing):

Container Management

Basic Container Operations


Start the mailserver 
docker compose up -d

 

Stop the mailserver
docker compose down


Restart the mailserver
docker compose restart


View logs (live)
docker compose logs -f


View logs (last 100 lines)

docker compose logs --tail=100


Check container status
docker ps -a


Check container health
docker inspect mailserver | grep -A 10 Health


 

Email Account Management

Create Email Accounts

 

Create a new email account (will prompt for password)
docker exec -it mailserver setup email add user@home.conorbriggs.com.au

 

Create account with password in command
docker exec -it mailserver setup email add user@home.conorbriggs.com.au password123

 

Create account with quota (e.g., 500MB)
docker exec -it mailserver setup email add user@home.conorbriggs.com.au password123 500M

 

List Email Accounts

 

List all email accounts
docker exec mailserver setup email list

 

View the accounts file directly
docker exec mailserver cat /tmp/docker-mailserver/postfix-accounts.cf

 

Update/Change Passwords

 

Update password for existing account
docker exec -it mailserver setup email update user@home.conorbriggs.com.au new_password

 

Change password (alternative method - will prompt)
docker exec -it mailserver setup email update user@home.conorbriggs.com.au

 

Delete Email Accounts

 

Delete an email account
docker exec -it mailserver setup email del user@home.conorbriggs.com.au

 

Delete account and remove mailbox data
docker exec mailserver setup email del user@home.conorbriggs.com.au
rm -rf ./mail-data/home.conorbriggs.com.au/user

 

 

Alias Management

Create Aliases

 

Create an alias (forward emails from alias to recipient)
docker exec mailserver setup alias add alias@home.conorbriggs.com.au recipient@home.conorbriggs.com.au

 

Create alias with multiple recipients
docker exec mailserver setup alias add sales@home.conorbriggs.com.au "user1@home.conorbriggs.com.au,user2@home.conorbriggs.com.au"

 

List Aliases

 

List all aliases
docker exec mailserver setup alias list

 

View aliases file
docker exec mailserver cat /tmp/docker-mailserver/postfix-virtual.cf

 

Delete Aliases

 

Delete an alias
docker exec mailserver setup alias del alias@home.conorbriggs.com.au recipient@home.conorbriggs.com.au

 

 

Quota Management

Set Quotas

 

Set quota for a user (e.g., 1GB)
docker exec mailserver setup quota set user@home.conorbriggs.com.au 1G

 

Set unlimited quota
docker exec mailserver setup quota set user@home.conorbriggs.com.au 0

 

Check Quotas

 

Check quota for specific user
docker exec mailserver setup quota get user@home.conorbriggs.com.au

 

List all quotas
docker exec mailserver setup quota list

 

Check quota usage
docker exec mailserver doveadm quota get -u user@home.conorbriggs.com.au

 

Delete Quotas

 

Remove quota (sets to default)
docker exec mailserver setup quota del user@home.conorbriggs.com.au

 

 

DKIM (Email Signing)

Generate DKIM Keys

 

Generate DKIM key for domain
docker exec mailserver setup config dkim

 

Generate for specific domain
docker exec mailserver setup config dkim domain home.conorbriggs.com.au

 

Generate with custom key size
docker exec mailserver setup config dkim keysize 2048

 

View DKIM Public Key

 

Show DKIM DNS record
docker exec mailserver setup config dkim help

 

View the public key directly
docker exec mailserver cat /tmp/docker-mailserver/opendkim/keys/home.conorbriggs.com.au/mail.txt

 

 

Fail2Ban (Security)

Fail2Ban Status

 

Check fail2ban status
docker exec mailserver setup fail2ban status

 

Check banned IPs
docker exec mailserver setup fail2ban

 

Unban an IP address
docker exec mailserver setup fail2ban unban <IP_ADDRESS>

 

Ban an IP address
docker exec mailserver setup fail2ban ban <IP_ADDRESS>

 

 

Debugging & Diagnostics

Service Status

 

Check all listening ports
docker exec mailserver ss -tlnp

 

Check specific service status
docker exec mailserver supervisorctl status

 

Check Postfix status
docker exec mailserver postfix status

 

Check Dovecot status
docker exec mailserver doveadm service status

 

Mail Queue

 

View mail queue
docker exec mailserver postqueue -p

 

Flush mail queue (retry sending)
docker exec mailserver postqueue -f

 

Delete all queued mail
docker exec mailserver postsuper -d ALL

 

Delete specific message from queue
docker exec mailserver postsuper -d <QUEUE_ID>

 

Logs

 

View mail logs
docker exec mailserver tail -f /var/log/mail/mail.log

 

View mail errors
docker exec mailserver tail -f /var/log/mail/mail.err

 

View specific log files
docker exec mailserver ls -la /var/log/mail/

 

Search logs for specific email
docker exec mailserver grep "user@domain.com" /var/log/mail/mail.log

 

Test Email Delivery

 

Test SMTP connection
docker exec mailserver nc -zv localhost 25

 

Send test email from command line
echo "Test email body" | docker exec -i mailserver sendmail test@home.conorbriggs.com.au

 

Test with swaks (if installed)
docker exec mailserver swaks --to user@home.conorbriggs.com.au --from test@home.conorbriggs.com.au

 

Connection Testing

 

Test IMAP connection
docker exec mailserver nc -zv localhost 143
docker exec mailserver nc -zv localhost 993

 

Test SMTP connection
docker exec mailserver nc -zv localhost 25
docker exec mailserver nc -zv localhost 587
docker exec mailserver nc -zv localhost 465

 

Check TLS/SSL certificates
docker exec mailserver openssl s_client -connect localhost:993 -showcerts
docker exec mailserver openssl s_client -connect localhost:465 -showcerts

 

 

Configuration Management

Reload Configuration

 

Reload postfix configuration
docker exec mailserver postfix reload

 

Reload dovecot configuration
docker exec mailserver doveadm reload

 

Restart all services
docker compose restart

 

View Configuration

 

View postfix configuration
docker exec mailserver postconf

 

View dovecot configuration
docker exec mailserver doveconf

 

View specific postfix setting
docker exec mailserver postconf | grep smtp_tls

 

Check all environment variables
docker exec mailserver env | grep -E '(SMTP|IMAP|SSL|TLS)'

 

Backup Configuration

 

Backup all mail data
tar -czf mailserver-backup-$(date +%Y%m%d).tar.gz ./mail-data ./mail-state ./config

 

Backup just configuration
tar -czf mailserver-config-$(date +%Y%m%d).tar.gz ./config

 

Backup specific user's mailbox
tar -czf user-backup-$(date +%Y%m%d).tar.gz ./mail-data/home.conorbriggs.com.au/user

 

 

Database/User Management

User Database

 

List all users in Dovecot
docker exec mailserver doveadm user '*'

 

Check if user exists
docker exec mailserver doveadm user user@home.conorbriggs.com.au

 

View user's mailbox location
docker exec mailserver doveadm mailbox status -u user@home.conorbriggs.com.au all '*'

 

Mailbox Management

 

List mailboxes for user
docker exec mailserver doveadm mailbox list -u user@home.conorbriggs.com.au

 

Create mailbox for user
docker exec mailserver doveadm mailbox create -u user@home.conorbriggs.com.au Folder.Name

 

Delete mailbox
docker exec mailserver doveadm mailbox delete -u user@home.conorbriggs.com.au Folder.Name

 

Rebuild mailbox index
docker exec mailserver doveadm force-resync -u user@home.conorbriggs.com.au INBOX

 

 

Performance & Monitoring

Check Resource Usage

 

Check container stats
docker stats mailserver

 

Check disk usage
docker exec mailserver df -h

 

Check memory usage
docker exec mailserver free -h

 

Check mail directory size
du -sh ./mail-data/*

 

Connection Monitoring

 

Show active connections
docker exec mailserver ss -tn | grep -E ':(25|587|465|143|993)'

 

Count connections by port
docker exec mailserver ss -tn | grep -E ':(25|587|465|143|993)' | wc -l

 

Show who's connected to IMAP
docker exec mailserver doveadm who

 

 

SSL/TLS Certificate Management

Check Certificates

 

Check SSL certificate expiry
docker exec mailserver openssl x509 -in /etc/letsencrypt/live/mx.home.conorbriggs.com.au/fullchain.pem -noout -dates

 

View certificate details
docker exec mailserver openssl x509 -in /etc/letsencrypt/live/mx.home.conorbriggs.com.au/fullchain.pem -noout -text

 

Test SSL/TLS for SMTP
openssl s_client -connect mx.home.conorbriggs.com.au:465 -showcerts

 

Test STARTTLS for SMTP
openssl s_client -connect mx.home.conorbriggs.com.au:587 -starttls smtp

 

 

Troubleshooting

Common Issues

 

Check if services are running
docker exec mailserver supervisorctl status

 

Restart specific service
docker exec mailserver supervisorctl restart postfix
docker exec mailserver supervisorctl restart dovecot

 

Check for permission issues
docker exec mailserver ls -la /var/mail/
docker exec mailserver ls -la /tmp/docker-mailserver/

 

Verify DNS records
dig mx home.conorbriggs.com.au
dig txt _dmarc.home.conorbriggs.com.au
dig txt mail._domainkey.home.conorbriggs.com.au

 

Test email authentication
docker exec mailserver opendkim-testkey -d home.conorbriggs.com.au -s mail

 

Reset and Clean Up

 

Remove all mail data (WARNING: deletes all emails)
docker compose down
rm -rf ./mail-data/*
rm -rf ./mail-state/*
docker compose up -d

 

Clear logs
docker exec mailserver truncate -s 0 /var/log/mail/mail.log

 

Rebuild entire container
docker compose down
docker compose pull
docker compose up -d --force-recreate

 

 

Quick Reference

Setup Script Help

 

Show all setup commands
docker exec mailserver setup help

 

Help for specific command
docker exec mailserver setup email help
docker exec mailserver setup alias help
docker exec mailserver setup config help

 

File Locations Inside Container

/tmp/docker-mailserver/          - Configuration files (mapped to ./config/)
/var/mail/                        - Mail data (mapped to ./mail-data/)
/var/mail-state/                  - State files (mapped to ./mail-state/)
/var/log/mail/                    - Mail logs (mapped to ./mail-logs/)
/etc/letsencrypt/                 - SSL certificates (read-only)
/etc/postfix/                     - Postfix configuration
/etc/dovecot/                     - Dovecot configuration

 

Important Configuration Files

./config/postfix-accounts.cf      - Email accounts
./config/postfix-virtual.cf       - Aliases
./config/dovecot-quotas.cf        - User quotas
./config/opendkim/                - DKIM keys

 

 

Advanced Operations

Database Operations

 

Export all accounts
docker exec mailserver cat /tmp/docker-mailserver/postfix-accounts.cf > accounts-backup.txt

 

Import accounts
cat accounts-backup.txt | docker exec -i mailserver tee /tmp/docker-mailserver/postfix-accounts.cf
docker compose restart

 

Verify account database
docker exec mailserver postmap -q user@home.conorbriggs.com.au /tmp/docker-mailserver/postfix-accounts.cf

 

Custom Scripts

 

Run custom maintenance script
docker exec mailserver /bin/bash -c "your-script-here"

 

Execute interactive shell
docker exec -it mailserver /bin/bash

 

 

Monitoring & Alerts

Set Up Monitoring

 

Watch logs in real-time
docker compose logs -f --tail=100

 

Monitor for failed logins
docker exec mailserver tail -f /var/log/mail/mail.log | grep "authentication failed"

 

Monitor mail queue size
watch -n 60 'docker exec mailserver postqueue -p | tail -1'

 

Check for errors
docker exec mailserver grep -i error /var/log/mail/mail.log | tail -20

 

 

Common Workflows

Adding a New User

 

1. Create account
docker exec -it mailserver setup email add newuser@home.conorbriggs.com.au

 

2. Set quota (optional)
docker exec mailserver setup quota set newuser@home.conorbriggs.com.au 2G

 

3. Verify account created
docker exec mailserver setup email list

 

4. Test login: Use an email client to connect via IMAP (993) or SMTP (587)

 

Migrating Mail

 

1. Backup old server
tar -czf old-mailserver-backup.tar.gz ./mail-data

 

2. Copy to new server
scp old-mailserver-backup.tar.gz newserver:/path/to/mailserver/

 

3. Extract on new server
tar -xzf old-mailserver-backup.tar.gz

 

4. Fix permissions
chown -R 5000:5000 ./mail-data

 

5. Restart mailserver
docker compose restart

 

Security Hardening

 

1. Enable Fail2Ban (in .env file)
ENABLE_FAIL2BAN=1

 

2. Check banned IPs regularly
docker exec mailserver fail2ban-client status postfix-sasl

 

3. Monitor authentication attempts
docker exec mailserver grep "authentication failed" /var/log/mail/mail.log

 

4. Review SSL/TLS settings
docker exec mailserver postconf | grep tls

 

 

Tips & Best Practices

  1. Always backup before major changes: tar -czf backup.tar.gz ./mail-data ./config
  2. Test email flow after changes: Send test emails in/out
  3. Monitor disk space: Check df -h regularly
  4. Keep certificates updated: Let's Encrypt certs expire every 90 days
  5. Review logs periodically: Look for authentication failures or delivery issues
  6. Set appropriate quotas: Prevent users from filling up disk
  7. Use strong passwords: Minimum 12 characters for email accounts
  8. Enable DKIM/SPF/DMARC: Improves deliverability
  9. Regular updates: docker compose pull && docker compose up -d
  10. Document your changes: Keep notes on custom configurations

Software & packages

Software & packages

Image manipulation

Compression

Install sudo apt-get install imagemagick

Compress command mogrify -quality 80% *.jpg

note: this will overwrite images

Software & packages

yt-dlp

Download video as mp3

yt-dlp -x --audio-format mp3 URL_HERE

Convert webm to mp4

 ffmpeg -i input.webm -c:v libx264 -preset slow -crf 22 -c:a aac -b:a 128k output.mp4

Download video as mp4

 yt-dlp -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best" SOURCE_URL

Convert video codec to x264

 ffmpeg -i input.mp4 -c:v libx264 -crf 23 -preset medium -c:a aac -b:a 192k output.mp4


Software & packages

Python packages and commands

Managing Virtual Envrionments

Create a new venv python3 -m venv /path/to/new/virtual/environment

Activate the new venv Linux / Mac source myenv/bin/activate

Activate an envrionment on windows myenv\Scripts\activate

Exit the venv deactivate

 

Installing and uninstalling packages

Installing a package pip install xyz

Uninstalling a package pip uninstall xyz

 

Useful packages

 

 

 

 

Software & packages

spotdl

Install & Setup

Best to load a venv first

Installing spotdl pip install spotdl

 Open the config file at .spotdl/config.json and update "output" to be "{album}/{title}.{output-ext}"

 

Updating

pip install --upgrade spotdl

 

 

 

 

 

Software & packages

ffmpeg

Installing

sudo apt install ffmpeg

Compress a video to h265

 ffmpeg -i example_h264.mp4 -c:v libx265 -crf 24 -preset fast -c:a aac -b:a 128k example_h265.mp4

CRF Info

For H.264 (x264): 

The typical CRF range is 18-28. 

CRF 18: Near lossless quality (very high quality, larger file size). 

CRF 23: Default value, good balance between quality and file size. 

CRF 28: Noticeable quality loss, much smaller file size.


Overwright a videos audio with a new audio track

 ffmpeg -i video_to_overwright.mkv -i audio_input.mp3 -c:v copy -map 0:v:0 -map 1:a:0 output.mp4

MP4 to MP3

 ffmpeg -i filename.mp4 filename.mp3

 

Change video format

ffmpeg -i input.m4v -c copy output.mp4

 

Software & packages

IPFS

Installation for Linux

1 - Download the package

 wget https://dist.ipfs.tech/kubo/v0.31.0/kubo_v0.31.0_linux-amd64.tar.gz

2 - Unzip the file

 tar -xvzf kubo_v0.39.0_linux-amd64.tar.gz

3 - Move into the kubo folder

 cd kubo

4 - Run the install script

 sudo bash install.sh

5 - Test the installation worked

 ipfs --version

Editing the config

Show Config:

 ipfs config show

Change connection count

ipfs config Swarm.ConnMgr.HighWater 1000 --json
ipfs config Swarm.ConnMgr.LowWater 500 --json

Enable filestore (Allow import with no copy)

ipfs config --json Experimental.FilestoreEnabled true
sudo systemctl restart ipfs
ipfs config Experimental.FilestoreEnabled

IPFS as a service

Open a new file at /etc/systemd/system/ipfs.service

[Unit]
Description=IPFS daemon
After=network.target

[Service]
ExecStart=/usr/local/bin/ipfs daemon
Restart=on-failure
User=conor
Group=conor
Environment=IPFS_PATH=/home/conor/.ipfs

[Install]
WantedBy=default.target

WantedBy=default.target means the service will start up at boot

get it to start at boot

 sudo systemctl enable ipfs

and start it

 sudo systemctl start ipfs

Usage

Check filestore is enabled and working

ipfs config Experimental.FilestoreEnabled

List files in filestore

ipfs filestore ls

Import files to filestore

ipfs add --nocopy --recursive --cid-version=1 /path/to/your/folder

Software & packages

Wireguard

Setup Script

#!/bin/bash

# Checks to see if script is being run as root
if [ "$EUID" -ne 0 ]; then
  echo "Please run as root"
  exit
fi

sudo apt install wireguard

# Deletes keys in case this script is being run again
rm -f /etc/wireguard/* 

# Create necessary directories, including parents
mkdir -p /etc/wireguard/client


# Create and store server private and public key
wg genkey | tee /etc/wireguard/server_priv.key | wg pubkey | tee /etc/wireguard/server_pub.key

# Get the server public and private key as well as the network interface
server_priv_key=$(cat /etc/wireguard/server_priv.key)
server_pub_key=$(cat /etc/wireguard/server_pub.key)
network_interface=$(ip -o -4 route show to default | awk '{print $5}')


# Client
# Create client public and private key and store it in vars for later
wg genkey | tee /etc/wireguard/client/client_priv.key | wg pubkey | tee /etc/wireguard/client/client_pub.key
client_priv_key=$(cat /etc/wireguard/client/client_priv.key)
client_pub_key=$(cat /etc/wireguard/client/client_pub.key)

# Create initial client config
echo "[Interface]
PrivateKey = $client_priv_key
Address = 10.0.0.2/24

[Peer]
PublicKey = $server_pub_key
Endpoint = servername_or_ip:51820
AllowedIPs = 0.0.0.0/0
" > /etc/wireguard/client/wg0.conf


# Create the config file for wireguard
echo "[Interface]
Address = 10.0.0.1/24
SaveConfig = true
ListenPort = 51820
PrivateKey = $server_priv_key
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o $network_interface -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o $network_interface -j MASQUERADE

[Peer]
PublicKey = $client_pub_key
AllowedIPs = 10.0.0.2/24
" > /etc/wireguard/wg0.conf

# Change permissions so only root can access the files
chmod 600 /etc/wireguard/server_priv.key
chmod 600 /etc/wireguard/wg0.conf

# Allow 51820 through ufw
ufw allow 51820/udp

# Start wireguard and set it to auto start on boot
wg-quick up wg0
systemctl enable wg-quick@wg0

Setup - Manual

1 - Update system and install wireguard

 apt update && apt upgrade && apt install wireguard

2 - Make the necessary folders 

 mkdir -p /etc/wireguard/client

3 - Create the server's public and private keys 

 wg genkey | tee /etc/wireguard/server_priv.key | wg pubkey | tee /etc/wireguard/server_pub.key

4 - Find the servers network interface

 ip -o -4 route show to default | awk '{print $5}

5 - Nano into 

 /etc/wireguard/wg0.conf

(Make sure to add network interface and server private key) 

 [Interface] Address = 10.0.0.1/24 
SaveConfig = true 
ListenPort = 51820 
PrivateKey = SERVER_PRIVATE_KEY 
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o NETWORK_INTERFACE_HERE -j MASQUERADE 
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o NETWORK_INTERFACE_HERE -j MASQUERADE

 6 - Update file permissions so only root can read the config file and private key 

 chmod 600 /etc/wireguard/server_priv.key chmod 600 /etc/wireguard/wg0.conf

 7 - Allow the vpn port through UFW

 ufw allow 51820/udp

 8 - Start wireguard and set it to auto start at boot 

 wg-quick up wg0 systemctl enable wg-quick@wg0

 9 - Create the client keys 

 wg genkey | tee /etc/wireguard/client/priv.key | wg pubkey | tee /etc/wireguard/client/pub.key

 10 - Create the client config file (copy to client device) 

[Interface] PrivateKey = CLIENT_PRIVATE_KEY 
Address = 10.0.0.2/24 [Peer] 
PublicKey = SERVER_PUBLIC_KEY 
Endpoint = SERVER_IP_OR_DOMAIN:51820 
AllowedIPs = 0.0.0.0/0

11 - Add peer info to the server config

[Interface] Address = 10.0.0.1/24 
SaveConfig = true 
ListenPort = 51820 
PrivateKey = SERVER_PRIVATE_KEY 
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o NETWORK_INTERFACE_HERE -j MASQUERADE 
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o NETWORK_INTERFACE_HERE -j MASQUERADE

# =================
[Peer]
PublicKey = PUBLIC_KEY
AllowedIPs = 10.0.0.2/24

Setup with SeaBee's setup script

wget https://raw.githubusercontent.com/seabee33/wireguard_helper/refs/heads/main/wg_helper.py && chmod +x wg_helper.py && sudo python3 wg_helper.py

 

Auto start at boot

1 - ensure the client config is at /etc/wireguard/wg0.conf
2 - enable it to start at boot with sudo systemctl enable wg-quick@wg0

 

 

Software & packages

Node & NVM

1 - install node version manager (NVM)

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash

2 - install node

nvm install node

General steps

1 - install components

npm create vite@latest my-charting-site --template react
cd my-charting-site
npm install

npm install -D tailwindcss@3 postcss autoprefixer
npx tailwindcss init -p

2 - test

npm run dev

 

build

npm run build

 

Software & packages

Syncthing

Software & packages

Screen

Screen lets you make virtual terminals with

screen

 

 

New session with name

screen -S session_name

 

Software & packages

Tailscale

Client setup

1. Install Tailscale client

curl -fsSL https://tailscale.com/install.sh | sh

2. Create a pre-auth key on your Headscale server

docker exec headscale headscale preauthkeys create --user 1 --reusable --expiration 24h

Copy the key that's generated.

3. Connect to your Headscale server

sudo tailscale up --login-server=https://headscale.conorbriggs.com.au --authkey=YOUR_PREAUTH_KEY --hostname=rpi3

Replace YOUR_PREAUTH_KEY with the key from step 2.

4. Verify connection

# On Raspberry Pi:
sudo tailscale status

# On your Headscale server:
docker exec headscale headscale nodes list

You should see your Raspberry Pi listed with the hostname "raspberry-pi" and an IP like 100.64.0.2.

SMB

Editing SMB shares on linux

1 - Open the config file at  /etc/samba/smb.conf

2 -  Edit the SMB file 

[ShareName]
    path = /path/to/share
    browseable = yes
    writable = yes
    valid users = user1, user2

3 - Save & restart smbd

 sudo systemctl restart smbd

SMB Share options

browsable = yes/no Whether the share should be listed when clients request a list of shares. 

Guest ok = yes/no  guests can access without a password, set `read only` to `yes` to just be guest read only `guest ok = yes`

Editing SMB Users

Creating an SMB user sudo smbpasswd -a new_username
Listing all SMB users pdbedit -L
Changing an SMB user's password sudo smbpasswd username


Mounting & Unmounting

Get disk information

List information about all available block devices, including disks and their partitions. 

lsblk -o NAME,MODEL,SIZE,ROTA,TYPE,MOUNTPOINTS

List information about disks and the partitions with UUID

lsblk -f

 List disks with filesystems sudo parted -l

Permanent mounting

Edit the file at /etc/fstab
UUID=xxxxxxxxx-xxxxx-xxxxxxx /mnt/new_drive ext4 defaults X Y

X = Backup order - 0 is no backup

Y = Check filesystem at boot - 0 is no check


after editing fstab:

sudo mount -a

 

 

 

 

Temporary mounting

sudo mount -t ext4 /dev/sdXY /mount/path

X = Disk letter

Y = Disk partition

Symbolic Links

 ln -s /path/to/target /path/to_where_new_shortcut_is_made

-s = softlink like a windows shortcut

Users & Permissions

Users and groups

List all groups and users in groups

getent group

Add a user to a group 

sudo adduser username groupname

which is the same as 

sudo usermod -aG groupname username

Create a new user 

sudo useradd -options_here username

Options available:

Permissions

To see if a particular user can run a command 

sudo -u USER COMMAND

 

To change a folders permission 

sudo chmod -R u+rwx,g+rwx,o-rwx /path/to_folder

or change ownership

sudo chown -R NEW_USER:NEW_GROUP FOLDER_NAME

-R = recursive

 Note: user or group can be left blank

Passwords

To change the curretly logged in user's password passwd 

To change a different users password sudo passwd USERNAME

General Management & commands

Get the size of a folder

ncdu /path/to/view/

Strip all metadata from files

exiftool -all= -overwrite_original *

SSH

Generate a keypair

1 - Generate the key 

ssh-keygen -t ed25519 -a 100 -f $HOME/.ssh/SSH_KEY_NAME

2 - Copy the key to the server

ssh-copy-id -i ~/.ssh/id_rsa.pub USERNAME@SERVER_IP

Fail2Ban

Setup

  1. Update and install apt update && apt upgrade -y && apt install -y fail2ban
  2. Start fail2ban and set it to auto start on boot sudo systemctl start fail2ban && sudo systemctl enable fail2ban 
  3. Make a local copy of the config file (why? because jail.conf get wiped after each update) sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
  4. Open jail.local and setup stuff

 

Actions

Unbanning IP fail2ban-client set YOUR_JAIL_NAME_HERE unbanip IP_ADDRESS

 Banning IP fail2ban-client JAIL_NAME banip IP_ADDRESS

 Changing max retry attempts

  1. Open file located at etc/fail2ban/jail.local
  2. Add maxretry = x under [sshd]
  3. Restart fail to ban sudo systemctl restart fail2ban

wget

General Usage

wget -Flags URL

Full site mirror

wget -mkprE -np -nc -l inf -e robots=off --restrict-file-names=windows -D example.com --no-check-certificate http://example.com

Flags

Bash

Edit shortcuts

Open .bashrc file and add

 alias 1="command"

Auto run commands on terminal open

Open .bashrc file and add command to bottom file

Fix: no bash after ssh login

chsh -s /bin/bash

 


UFW

General Info

By default, UFW is set to deny all incoming connections and allow all outgoing connections. This means anyone trying to reach your server would not be able to connect, while any application within the server would be able to reach the outside world. 

So make sure to allow ssh before enabling UFW!!! UFW requires you to add / deny ports in a certain order, if you want to allow a certain ip access a port but deny access by everyone else, the ip must be allowed first then deny all after


Setup

 sudo apt update && sudo apt install -y ufw

Basic Management

Turning UFW on: 

ufw enable

Turning UFW off: 

ufw disable

Check if UFW is enabled or disabled

sudo ufw status

Allowing access from a particular IP to ANY port

sudo ufw allow from IP_ADDRESS

Allowing access from a partiular IP to A SPECIFIED port:

 sudo ufw allow from IP_ADDRESS to any port PORT_NUMBER

Allowing any access to a specified port:

 sudo ufw allow PORT_NUMBER

Denying access to a specific port 

 sudo ufw deny PORT_NUMBER

List rule numbers

sudo ufw status numbered

 


Swap

Clear swap file

1 - Disable Swap

sudo swapoff -a

2 - Re-enable swap

sudo swapon -a


View Swap usage

sudo smem -rs swap

Swap config

Update swappiness config

sudo bash -c "echo 'vm.swappiness = 1' >> /etc/sysctl.conf"

Update value immediately

sysctl vm.swappiness=1

About swap

Swappiness in Linux is a kernel parameter that controls how aggressively the system moves inactive pages from RAM to the swap space on disk. The swappiness value is represented as a percentage from 0 to 100, and it affects the balance between using physical RAM and swap space. Lowering swappiness can help improve performance in certain scenarios, particularly when you have a decent amount of physical memory available.

How Swappiness Works
Low swappiness values (0–30): The kernel avoids swapping as much as possible and will only swap when RAM is close to being fully utilized. This setting is ideal when performance is prioritized and there’s enough RAM.

High swappiness values (60–100): The kernel uses swap space more aggressively, even when RAM is available. This setting can be helpful on systems with limited RAM or on systems running many applications simultaneously.

The default value for swappiness on most Linux systems is 60.

Disk Stuff

List disks:

 fdisk -l

OR

 df -h

Scanning

1 - Check if SMART scans are enabled

 smartctl -s on /dev/sdX

2 - Run a test

 smartctl -t DURATION /dev/sdX

Duration is either "long" or "short"

3 - Check results

 smartctl -a /dev/sdX

S.M.A.R.T scanning

Scan a disk

sudo smartctl -t long/short /dev/sdX

Check scan results

sudo smartctl -a /dev/sdX

Check scan progress

smartctl -c /dev/sdX

Auto updates

1 - Install and setup unattended-upgrades package

sudo apt update && sudo apt install unattended-upgrades

 

2 - Enable the auto updates

sudo dpkg-reconfigure --priority=low unattended-upgrades

 

3 - Edit the config to allow security updates

Open

sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

 

Uncomment this line

        "${distro_id}:${distro_codename}-updates";

 

SSH Tunnel

ssh -L 127.0.0.1:3389:192.168.4.222:3389 -N conor@cbcore

-N = Don't execute command on remote server
-L = Specifies local port forwarding