Skip to content
Articles and documentation
Virtualization and infrastructure
Automating host configuration using shell scripts

Automating host configuration using shell scripts

If you run multiple servers, virtual machines and LXC containers, and you often create new ones like me, you may have faced the issue of having to repetitively do the chore of typing in the same commands and changing the same configuration files over and over again.

The first thought then is probably to look for a dedicated tool that can help you, like Ansible which is created for just that - configuring hosts by applying a pre-defined playbook. There is something much simpler though, something that is very close to what you’ve already been doing when configuring the hosts manually - Shell Scripts.

Don’t get me wrong - Ansible is a great tool for managing configuration, automation, deployment and more. Go for it if you want to learn it, or if you just want to use it, it’s good and very scalable. But if your needs are not too big, and you want to quickly throw something together because you’re lazy with a few repetitive commands you type on each host, you can do what I did:

How I configure my hosts

The majority of my servers are LXC containers - specifically Debian 13. I’ve also got a few VMs running Debian or Debian-based systems like Proxmox. All those systems when installed are pretty bare and unconfigured, therefore I always do the following for my machines:

  • Update and Upgrade system packages
  • Install system utilities and tools: zsh, fastfetch, vim, tcpdump, nmap, traceroute, ufw, curl, btop, tmux, sudo, bind9-utils
  • Edit /etc/hosts to configure FQDN for the host bound to the link-local address (Important especially for Proxmox, as it doesn’t accept FQDN being loopback)
  • For LXCs: make Proxmox respect my hosts file overwrite
  • Create a /etc/motd that displays on login
  • Make the default interactive shell be zsh instead of bash
  • Install zsh-grml-config for better interactive experience
  • Add aliases and run fastfetch on shell launch from the local zsh config at /etc/zsh/zshrc.local
  • Backport a short fastfetch template from newer versions, as Debian 13 doesn’t yet have the fastfetch config I use

Quite a bit of typing for just one host, isn’t it. After doing it all manually on two or three machines, and not wanting to bother to do full-on automation with Ansible Playbooks (which honestly, I might do one day for the cleanliness of the setup), I realized all commands I type, might as well be just a shell script - as that’s what a script is, a list of commands.

The quick script I made

To avoid typing everything in manually every time I create a new LXC or a VM, I’ve gathered all the commands and threw together this very rough around the edges script. To explain what it does, I’ve just now annotated it with comments.

The script
#!/usr/bin/env sh

echo "==> Installing software with apt..."
apt update && apt upgrade  # Update the repos and upgrade packages
apt install zsh fastfetch vim tcpdump nmap traceroute ufw curl btop tmux sudo bind9-utils  # Install software I always need

echo "==> Creating .pve-ignore files..."
touch /etc/.pve-ignore.hosts  # Make Proxmox ignore the hosts file, so to not overwrite it on each LXC boot

echo "==> Copy the LLA and press enter"
ip -br a  # Displaying all IPs on the system
read garbage  # Freezing the execution, so the LLA can be manually copied

echo "==> Opening vim /etc/hosts ..."
vim /etc/hosts  # Editing /etc/hosts to paste the LLA and edit the Fully Qualified Domain Name

echo "==> Creating a new motd..."
rm /etc/motd  # Removing the default Debian motd

# Echoing a new motd into /etc/motd, including the FQDN from earlier now gotten via $()
echo "" >> /etc/motd
echo "╔════════════════════════════════════════╗" >> /etc/motd
echo "║                                  - = x ║" >> /etc/motd
echo "║ ══> $(hostname -f || hostname) ║" >> /etc/motd
echo "║                                        ║" >> /etc/motd
echo "║  ─> If you are not authorized          ║" >> /etc/motd
echo "║  ─> disconnect immediately             ║" >> /etc/motd
echo "║                                        ║" >> /etc/motd
echo "╚════════════════════════════════════════╝" >> /etc/motd
echo "" >> /etc/motd

echo "==> Opening vim /etc/motd ..."
vim /etc/motd  # Opening motd for manual edits and checking

echo "==> Replacing zsh config..."
rm /etc/zsh/zshrc  # Removing default zshrc
curl -L -o /etc/zsh/zshrc https://grml.org/console/zshrc  # Curling grml-zsh-config into zshrc

echo "==> Creating zshrc.local ..."
echo "alias ls='ls -av --group-directories-first --color=auto'" > /etc/zsh/zshrc.local  # Aliasing `ls` to how I like it to be
echo "alias ll='ls -lahv --group-directories-first --color=auto'" >> /etc/zsh/zshrc.local  # Adding alias for `ll` -> ls but with -l
echo "fastfetch -c examples/27.jsonc" >> /etc/zsh/zshrc.local  # Running fastfetch on user login

if ! ls -l /usr/share/fastfetch/presets/examples/27.jsonc  # Checking if the examples/27.jsonc template for fastfetch exists, if not, it creates it
then
	echo "==> Creating default fastfetch preset ..."
	printf '{\n    "$schema": "https://github.com/fastfetch-cli/fastfetch/raw/dev/doc/json_schema.json",\n    "logo": {\n        "type": "small",\n        "padding": {\n            "top": 1\n        }\n    },\n    "display": {\n        "separator": "  "\n    },\n    "modules": [\n        "break",\n        "title",\n        {\n            "type": "os",\n            "key": "os    ",\n            "keyColor": "red"\n        },\n        {\n            "type": "kernel",\n            "key": "kernel",\n            "keyColor": "green"\n        },\n        {\n            "type": "host",\n            "format": "{vendor} {family}",\n            "key": "host  ",\n            "keyColor": "yellow"\n        },\n        {\n            "type": "packages",\n            "key": "pkgs  ",\n            "keyColor": "blue"\n        },\n        {\n            "type": "uptime",\n            "format": "{?days}{days}d {?}{hours}h {minutes}m",\n            "key": "uptime",\n            "keyColor": "magenta"\n        },\n        {\n            "type": "memory",\n            "key": "memory",\n            "keyColor": "cyan"\n        },\n        "break"\n    ]\n}\n
' >> /usr/share/fastfetch/presets/examples/27.jsonc
fi

echo "==> Changing default shell for root..."
chsh root -s /bin/zsh  # As above :)

echo "==> FINISHED"

It was put together pretty much as quickly as enrolling a single host by hand, but now, new hosts can be instantly configured. Making it, I barely had any Shell scripting experience, but quite a bit of Shell interactive experience. The script isn’t the prettiest, but as an executable procedure / list of commands to get the job done, it worked well.

Room for improvement

As I mentioned, the script is very rough around the edges. We can very quickly make it work just a bit better.

Shell options

First thing that I’ll improve, is to make the script fail the moment any of the commands fail. This is default behavior for most programming languages or scripting languages like Python, but in Shell, it’s not default - it will just keep executing commands one after another.

The behavior can be changed using the command below, added at the beginning of the script

set -e

Making some parts less lazy

The script was thrown together before I had any deeper Shell knowledge - a bunch of copy-pasted lines to just make it work, Vim edited lines using macros for repetition, and redundant commands everywhere. It’s ugly and not best practice. Let’s clear a bit of the repetition and make a bit more fancy just for fun.

Multiple echo statements

These, can be thrown into a cat command with redirected output, and reading until EOF like so:

Before
echo "==> Creating zshrc.local ..."
echo "alias ls='ls -av --group-directories-first --color=auto'" > /etc/zsh/zshrc.local
echo "alias ll='ls -lahv --group-directories-first --color=auto'" >> /etc/zsh/zshrc.local
echo "fastfetch -c examples/27.jsonc" >> /etc/zsh/zshrc.local
After
echo "==> Creating zshrc.local ..."
cat <<EOF > /etc/zsh/zshrc.local
alias ls='ls -av --group-directories-first --color=auto'
alias ll='ls -lahv --group-directories-first --color=auto'
fastfetch -c examples/27.jsonc
EOF

Now the code is cleaner. With the use of «EOF in cat, and terminating the lines with another EOF, everything in between will be thrown into cat, and then into the file specified.

Fix for the fastfetch config

Same <<EOF trick can be used for the fastfetch config, though I’ll single quote it and add a dash before it, so it can be indented with tabs in the code, and won’t execute parts of the json thinking it’s shell code.

I’ll unfold the \n spam in Vim, the same way I folded newlines inth \n before but in reverse, by highlighting it, and doing this in command mode:

:s/\\n/\r/g

This uses the sed command to replace occurences of \n (the slash had to be escaped with another dash not to make it a shortcode for a new line - thus \\n ), with \r - a shortcode for carriage return, so a new line. The g at the end lets sed do it for the whole line and not just for a single occurence.

There is also the elephant in the room of using ls to check if the file exists:

if ! ls -l /usr/share/fastfetch/presets/examples/27.jsonc
then

Now knowing Shell scripting, it hurts to look at it, it works but it’s bad practice. Let’s change the check to use the [] syntax, being a shorthand for the test command:

if [ ! -f /usr/share/fastfetch/presets/examples/27.jsonc ]
then

The -f option, checks if the file is there or not. The ! inverts the output, so the if statement block gets executed if the file is NOT there.

The MOTD

A wall of echo statements. I know. Also one that each time I had to pad with spaces myself manually, to make the ASCII box look nice.

Making these not only can be done in a much cleaner way than inexperienced me from the past did, it can also require no manual padding. Let’s use the «EOF cat method, and use a feature of printf that allows us to pad text with spaces.

After
echo "==> Creating a new motd..."

HOST_NAME=$(hostname -f || hostname)  # Made redundant later on
PADDED_HOSTNAME=$(printf "%-34s" "$HOST_NAME")

cat <<EOF > /etc/motd
╔════════════════════════════════════════╗
║                                  - = x ║
║ ══> $PADDED_HOSTNAME ║
║                                        ║
║  ─> If you are not authorized          ║
║  ─> disconnect immediately             ║
║                                        ║
╚════════════════════════════════════════╝
EOF

The %-34s makes the string stick to the left, and make it’s lenght 34 characters by appending spaces.

Redundant statements and QOL

I’ve also removed some redundant statements, like rm before curling the zsh config - Curl with -o actually overwrites the file by default, so no removal needed beforehand. The apt install statements also now have the -y flag, to not need extra input from the user.

Adding new things

I haven’t touched this script in a while, so there have been things that I started doing on each host that aren’t included here. Let’s add them!

Systemd-Networkd

Before I even run the script, I made sure my network is configured. Recently I made a switch for all my hosts from the built-in ifupdown, controlled by Proxmox from the outside of the LXC, to a solution that’s also built-in, but modern and scalable - systemd-networkd. So I would always:

  • Disable networking service (ifupdown)
  • Create a .network file for the interface
  • Enable systemd-networkd service

Let’s implement it!

What I want to achieve

This is the config I want on each machine, with some variables here of course.

[Match]
Name=eth0

[IPv6AcceptRA]
Token=::9999:9
Implementation

The new code
echo "==> Configuring the network..."

printf "Provide an interface name (default: eth0): "
read IP_INTERFACE
IP_INTERFACE=${IP_INTERFACE:-eth0}

printf "Provide the Token for IPv6 creation (default: eui64): "
read IP_TOKEN
IP_TOKEN=${IP_TOKEN:-eui64}

cat <<EOF > /etc/systemd/network/05-$IP_INTERFACE.network
[Match]
Name=$IP_INTERFACE

[IPv6AcceptRA]
Token=$IP_TOKEN
EOF

systemctl enable --now systemd-networkd
systemctl disable --now networking
networkctl reload
networkctl reconfigure ${IP_INTERFACE:-eth0}
This will ask the user for variables, and also default to eth0 and eui64 as values if the user just presses enter. Next it puts it in the .network file, and modifies relevant systemd services.

Configuring locale

For Debian 13 LXCs available in Proxmox, the locale settings are not configured by default. This causes problems running btop for example. Let’s run locale configuration from the script:

echo "==> Configuring locale..."
dpkg-reconfigure locales || echo "No changes made."

When no changes are made, dpkg returns 1. I’ll handle errors with || so they are ignored and a message is printed.

Installing FreeIPA Client

All of my hosts are enrolled with FreeIPA. Yes, even LXCs - it’s possible, see: &TBD

I always did it manually after the initial installation. Let’s now add this to the script and be done with it forever:

echo "==> Installing and running freeipa-client..."
apt install -y freeipa-client

ipa-client-install --mkhomedir -N

Adding fancy stuff

Automated /etc/hosts

The procedure
  • Take the interface name already given in the networkd step
  • Yank the Link-local address from the ip command
  • Ask the user for the domain to create an FQDN
  • Put it all in /etc/hosts
Implementation

This code will ask the user to choose a domain from the domains I manage in my homelab, and then match the choice using a case statement.
Then, a Link Local IP will be taken from ip, and split using awk. The CIDR mask will then get cut off by cut.
Finally, all will be written to /etc/hosts together with the default host definitions.

The new code
echo "==> Creating /etc/hosts..."

HOST_NAME=$(hostname)

cat <<EOF
Domains available:
  0. inf.fibermouse.xyz (default)
  1. net.fibermouse.xyz
  2. inf.ekspzoo.pl
  3. net.ekspzoo.pl
EOF
printf "Choose the domain or provide your own (default: 0): "
read domainchoice

case "$domainchoice" in
	0|"")
		HOST_FQDN=$HOST_NAME.inf.fibermouse.xyz
		;;
	1)
		HOST_FQDN=$HOST_NAME.net.fibermouse.xyz
		;;
	2)
		HOST_FQDN=$HOST_NAME.inf.ekspzoo.pl
		;;
	3)
		HOST_FQDN=$HOST_NAME.net.ekspzoo.pl
		;;
	*)
		HOST_FQDN=$domainchoice
		;;
esac

IP_LLA=$(ip -6 -brief address show $IP_INTERFACE scope link | awk '{print $3}' | cut -d/ -f1)

cat <<EOF > /etc/hosts
127.0.0.1	localhost
::1		localhost ip6-localhost ip6-loopback
ff02::1		ip6-allnodes
ff02::2		ip6-allrouters

$IP_LLA	$HOST_FQDN $HOST_NAME
EOF
I’ve also adjusted the MOTD creation to reflect the changes to $HOST_NAME and $HOST_FQDN

Getting rid of all echos

Throughout the script, I still extensively use echo to print log messages to the user, or printf and read to get input. Let’s wrap it, by implementing some functions around it, and then using vim’s :s to replace the echo statements.

This will reduce the use of repetitive ==> SOMETHING info messages, and also as a bonus - add color for visibility.

log_info() {
	printf "\033[1;96m==> \033[97m%s\033[0m\n" "$*"
}

log_normal() {
	printf "\033[0;36m -> \033[0m%s\n" "$*"
}

log_success() {
	printf "\033[0;32m==> \033[97m%s\033[0m\n" "$*"
}

log_prompt() {
	printf "\033[1;32m %%> \033[97m%s: \033[0m" "$*"
}

Now let’s do a few :s commands in vim, and get to the final, now good looking and well coded version of the script!

New and improved script

The new script
#!/usr/bin/env sh

set -e

# ---------------
# ==> Functions

log_info() {
	printf "\033[1;96m==> \033[97m%s\033[0m\n" "$*"
}

log_normal() {
	printf "\033[0;36m -> \033[0m%s\n" "$*"
}

log_success() {
	printf "\033[0;32m==> \033[97m%s\033[0m\n" "$*"
}

log_prompt() {
	printf "\033[1;32m %%> \033[97m%s: \033[0m" "$*"
}

# ---------------
# ==> Runtime

# ----
# -> Network configuration

log_info "Configuring the network"

log_prompt "Provide an interface name (default: eth0)"
read IP_INTERFACE
IP_INTERFACE=${IP_INTERFACE:-eth0}

log_prompt "Provide the Token for IPv6 creation (default: eui64)"
read IP_TOKEN
IP_TOKEN=${IP_TOKEN:-eui64}

cat <<EOF > /etc/systemd/network/05-$IP_INTERFACE.network
[Match]
Name=$IP_INTERFACE

[IPv6AcceptRA]
Token=$IP_TOKEN
EOF

systemctl enable --now systemd-networkd
systemctl disable --now networking
networkctl reload
networkctl reconfigure $IP_INTERFACE

# ----
# -> Configuring locales

log_info "Configuring locale"
dpkg-reconfigure locales || log_normal "No changes made"

# ----
# -> Installing software

log_info "Installing software with apt"
apt update -y
apt upgrade -y
apt install -y zsh fastfetch vim tcpdump nmap traceroute ufw curl btop tmux sudo bind9-utils

# ----
# -> Creating /etc/hosts

log_info "Creating /etc/hosts"

touch /etc/.pve-ignore.hosts

HOST_NAME=$(hostname)

log_normal "Domains available:"
cat <<EOF
  0. inf.fibermouse.xyz (default)
  1. net.fibermouse.xyz
  2. inf.ekspzoo.pl
  3. net.ekspzoo.pl
EOF
log_prompt "Choose the domain or provide your own (default: 0)"
read domainchoice

case "$domainchoice" in
	0|"")
		HOST_FQDN=$HOST_NAME.inf.fibermouse.xyz
		;;
	1)
		HOST_FQDN=$HOST_NAME.net.fibermouse.xyz
		;;
	2)
		HOST_FQDN=$HOST_NAME.inf.ekspzoo.pl
		;;
	3)
		HOST_FQDN=$HOST_NAME.net.ekspzoo.pl
		;;
	*)
		HOST_FQDN=$domainchoice
		;;
esac

IP_LLA=$(ip -6 -brief address show $IP_INTERFACE scope link | awk '{print $3}' | cut -d/ -f1)

cat <<EOF > /etc/hosts
127.0.0.1	localhost
::1		localhost ip6-localhost ip6-loopback
ff02::1		ip6-allnodes
ff02::2		ip6-allrouters

$IP_LLA	$HOST_FQDN $HOST_NAME
EOF

# ----
# -> Creating /etc/motd

log_info "Creating a new motd"

PADDED_FQDN=$(printf "%-34s" "$HOST_FQDN")

cat <<EOF > /etc/motd

╔════════════════════════════════════════╗
║                                  - = x ║
║ ══> $PADDED_FQDN ║
║                                        ║
║  ─> If you are not authorized          ║
║  ─> disconnect immediately             ║
║                                        ║
╚════════════════════════════════════════╝

EOF

# ----
# -> Setting up ZSH and Fastfetch

log_info "Replacing zsh config"
curl -L -o /etc/zsh/zshrc https://grml.org/console/zshrc

log_info "Creating zshrc.local"
cat <<EOF > /etc/zsh/zshrc.local
alias ls='ls -av --group-directories-first --color=auto'
alias ll='ls -lahv --group-directories-first --color=auto'
fastfetch -c examples/27.jsonc
EOF

if [ ! -f /usr/share/fastfetch/presets/examples/27.jsonc ]
then
	log_info "Creating default fastfetch preset"
	cat <<-'EOF' > /usr/share/fastfetch/presets/examples/27.jsonc
	{
	    "$schema": "https://github.com/fastfetch-cli/fastfetch/raw/dev/doc/json_schema.json",
	    "logo": {
	        "type": "small",
	        "padding": {
	            "top": 1
	        }
	    },
	    "display": {
	        "separator": "  "
	    },
	    "modules": [
	        "break",
	        "title",
	        {
	            "type": "os",
	            "key": "os    ",
	            "keyColor": "red"
	        },
	        {
	            "type": "kernel",
	            "key": "kernel",
	            "keyColor": "green"
	        },
	        {
	            "type": "host",
	            "format": "{vendor} {family}",
	            "key": "host  ",
	            "keyColor": "yellow"
	        },
	        {
	            "type": "packages",
	            "key": "pkgs  ",
	            "keyColor": "blue"
	        },
	        {
	            "type": "uptime",
	            "format": "{?days}{days}d {?}{hours}h {minutes}m",
	            "key": "uptime",
	            "keyColor": "magenta"
	        },
	        {
	            "type": "memory",
	            "key": "memory",
	            "keyColor": "cyan"
	        },
	        "break"
	    ]
	}
	EOF
fi

log_info "Changing default shell for root"
chsh root -s /bin/zsh

# ----
# -> Installing freeipa-client

log_info "Installing and running freeipa-client"
apt install -y freeipa-client

ipa-client-install --mkhomedir -N

# ----
# -> SUCCESS

log_success "FINISHED"

Configuring a host in practice

Now I gotta test it in practice! I’ve created an LXC Debian 13 container in my DMZ VLAN named kgm-git-run01 (will later become a Forgejo runner: &TBD)

Let’s attach to the container, and copy-paste the script. You can watch the process here:

All works! The host enrollment is now as easy as a copy paste, for me with now implemented extra functionality making it even easier than before.

Last updated on