Building Edge Native Artifacts

10 min
Justin Barksdale
06 Dec 2023

Build Edge Native Artifacts

Palette Edge Native is modeled on an immutable operating system. It is not another Linux distribution, but rather a method to consume the distribution of your choice in an immutable way. Palette Edge Native leverages the open-source project, Kairos to create these artifacts.

This how-to-guide will walk you through the basics of creating the edge native artifacts with Ubuntu to support K3s:

  • Installer Image - Used to install or “Flash” a device or virtual machine.
  • Provider Images - Used to provision new Kubernetes clusters and used to provide upgrades.

Edge Components

We will customize the installer image to support auto registration giving you a low-touch feeling when we deploy edge nodes in later Edge tutorials .

Prerequisites

To create these artifacts we will use a build machine with the following prerequisites. We will leverage a tool called Earthly with some provided variables in order to get the desired installer ISO and the provider images.

  • Internet Access
  • Palette Account
  • Command line tool to access the physical or virtual hardware via ssh
  • Kubectl
  • Physical or Virtual Hardware that can be “Flashed” with a new OS. (x86 Based)
  • DHCP is required for the physical or virtual edge device.
  • An additional available IP address on the same network as your DHCP scope.

Software

Minumum Hardware

  • x86 Based Platform
  • 4CPU
  • 8GB Memory
  • 50GB HD

The ability to run privileged containers is required.

This how-to guide was written with the following versions and hardware. These versions may differ slightly from what you are using.

Palette UI Build Host Hardware Build Host OS Docker Version GIT Version
v3.4.25 4-vCPU - 8GB Memory - 100GB Hard Disk Ubuntu 22.04.2 LTS 23.0.1 2.34.1

The GitHub Repo

  1. Clone the repo at CanvOS 
git clone https://github.com/spectrocloud/CanvOS.git

Sample Output

Cloning into 'CanvOS'...
remote: Enumerating objects: 133, done.
remote: Counting objects: 100% (133/133), done.
remote: Compressing objects: 100% (88/88), done.
Receiving objects: 100% (133/133), 40.16 KiB | 5.02 MiB/s, done.
Resolving deltas: 100% (60/60), done.
remote: Total 133 (delta 60), reused 101 (delta 32), pack-reused 0
  1. Change into the CanvOS directory that was created.
cd CanvOS
  1. Fetch Tags
git fetch --tags --all
  1. View Available tags
git tag

SAMPLE OUTPUT

v3.3.3
v3.4.0
v3.4.1
v3.4.11
v3.4.3
v3.4.7
v3.4.9
v4.0.2
v4.0.3
v4.0.4
v4.0.5
v4.0.6
v4.1.2
v4.1.4
v4.2.3
  1. Checkout the latest tag in the above example this is v4.2.3
git checkout <latest tag version>

Sample Output

Note: switching to 'v4.2.3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 942f385 pe version update, image macro update (#123)

Update Earthly Build Variables

Earthly is a super simple CI/CD framework that gives you repeatable builds that you write once and run anywhere; has a simple, instantly recognizable syntax; and works with every language, framework, and build tool.
We are leveraging Earthly to build the Installer image to flash our Physical or Virtual Machine, and our provider images to load, run, and upgrade our Kubernetes distribution and version.

The .arg file is used to pass variables to Earthly for our builds.

  1. Copy the .arg.template file to .arg
cp .arg.template .arg
  1. Edit the .arg file. Only modify the CUSTOM_TAG value. Make it your initials in lowercase. Everything else can stay the same.

The CUSTOM_TAG variable should adhere to standard image naming conventions which require it to be all lowercase. We use this variable to name the images as unique for the purpose of this documentation.

vi .arg
Parameter Description Type Default Value
CUSTOM_TAG Environment name for provider image tagging. The default value is demo. String demo
IMAGE_REGISTRY Image registry name that will store the image artifacts. The default value points to the ttl.sh image registry, an anonymous and ephemeral Docker image registry where images live for a maximum of 24 hours by default. If you wish to make the images exist longer than 24 hours, you can use any other image registry to suit your needs. String ttl.sh
OS_DISTRIBUTION OS distribution of your choice. For example, it can be ubuntu, opensuse-leap, rhel or sles String ubuntu
IMAGE_REPO Image repository name in your chosen registry. String $OS_DISTRIBUTION
OS_VERSION OS version. For Ubuntu, the possible values are 20.04, and 22.04. Whereas for openSUSE Leap, the possible value is 15.4. For sles, possible values are 5.4. This example uses 22 for Ubuntu. String 22.04
K8S_DISTRIBUTION Kubernetes distribution name. It can be one of these: k3s, rke2, kubeadm, or kubeadm-fips. String k3s
ISO_NAME Name of the Edge installer ISO image. In this example, the name is palette-edge-installer. String palette-edge-installer
ARCH Type of platform to use for the build. Used for Cross Platform Build (arm64 to amd64 as example). string amd64
BASE_IMAGE Base image to be used for building installer and provider images. String
FIPS_ENABLED to generate FIPS compliant binaries. true or false string false
HTTP_PROXY URL of the HTTP Proxy server to be used if needed (Optional) string
HTTPS_PROXY URL of the HTTPS Proxy server to be used if needed (Optional) string
NO_PROXY URLS that should be excluded from proxying (Optional) string
PROXY_CERT_PATH Absolute path of the SSL Proxy certificate in PEM format if needed (Optional) string
UPDATE_KERNEL Determines whether to upgrade the Kernel version to the latest from the upstream OS provider boolean false
BASE_IMAGE Base Image Override when using custom Kairos Base Images. (Optional) string

Depending on your editor the way you save may be different.
This lab was written using VIM.

  • To enable editing in VIM press i.
  • Use the arrow keys to scroll to the value of the CUSTOM_TAG variable.
  • Delete the value demo and replace it with your initials in lowercase.

Sample .arg File

CUSTOM_TAG=demo
IMAGE_REGISTRY=ttl.sh
OS_DISTRIBUTION=ubuntu
IMAGE_REPO=$OS_DISTRIBUTION
OS_VERSION=22.04
K8S_DISTRIBUTION=k3s
ISO_NAME=demo-edge-installer
ARCH=amd64
HTTPS_PROXY=
HTTP_PROXY=
PROXY_CERT_PATH=
UPDATE_KERNEL=false
# BASE_IMAGE=
  1. Save the file.
    To save with VIM, press esc then type :wq! and press enter

Depending on your editor the way you save may be different.

Create the Registration Token in Palette.

For enhanced security reasons a Registration Token needs to be embedded into our image. In the next steps, we will create the token for use later in this How-To-Guide.

  1. Open a browser window and navigate to the Palette Console
  • Login to your organization
    If you have not signed up you can sign up for a free trial Here
  1. Navigate Tenant Settings on the left hand menu (Bottom Left).
  • Select Registration Tokens
  • Click Add New Registration Token
  • Set Token Name as Demo
  • Set the Default Project as Default
  • Set the Expiration Date for 7 Days

Registration Token

  • Click Confirm

Registration Token

  1. Create the token variable on your build host using the value from the previous step.
token="<Paste Your Token Value Here>"

Create a user-data file

  1. Copy and paste the contents below. to create the user-data file with the token we created.
cat <<EOF > user-data
#cloud-config
stylus:
  site:
    edgeHostToken: $token
install:
  poweroff: true
stages:
  initramfs:
    - name: "Core system setup"
      users:
        kairos:
          groups:
            - admin
          passwd: kairos
EOF

For security purposes, we do not provide a username or password for the images that are being created. To create a username and password for lab/demo purposes we will create a user-data file with the attributes that are needed. Additionally, to aid in the auto-registration of the nodes that we will create and use in later tutorials, we are going to add a edgeHostToken to the user-data file. This will enable the nodes to automatically register with your Palette Tenant without user intervention.

This creates a user-data file that will be used by our agent to inject the values when the images are created. This sets the password for the user kairos to kairos and instructs the installer to turn the machine off once the installation is complete. It also tells the agent to use the auto-registration functionality and authenticate with the token we provided. For more parameters available in the user-data configuration checkout the Stylus Parameters page. We leverage YIP to apply stages and configuration attributes.

  1. Confirm the user-data file was created and has the appropriate values.
cat user-data

Sample Output

#cloud-config
stylus:
  site:
    edgeHostToken: aUAxxxxxxxxx0ChYCrO
install:
  poweroff: true
stages:
  initramfs:
    - name: "Core system setup"
      users:
        kairos:
          groups:
            - admin
          passwd: kairos

The user-data file is written in yaml and requires specific formatting 3. Validate the user-data format is valid formatting.

cloud-init schema --config-file user-data

SAMPLE OUTPUT

cloud-init schema --config-file user-data
Valid cloud-config: user-data

Create Edge Artifacts

The settings shown above point the image registry to ttl.sh. This is an ephemeral registry where the images live for a maximum of 24 hours (this is the default). If you wish to use your registry to make the artifacts exist longer than 24 hours you can modify the value of the IMAGE_REPO to match your needs.

This is outside the scope of this guide.

  1. Build the artifacts.
./earthly.sh +build-all-images

This will create your provider docker images and an iso that can/will be used in the Edge Native Tutorials.

SAMPLE OUTPUT

Copy the content from the output on your screen. The below output is a sample.

##########################################################################################################

PASTE THE CONTENTS BELOW INTO YOUR CLUSTER PROFILE IN PALETTE REPLACING ALL THE CONTENTS IN THE PROFILE

##########################################################################################################

SAMPLE OUTPUT

##########################################################################################################

PASTE THE CONTENT BELOW INTO YOUR CLUSTER PROFILE IN PALETTE REPLACING ALL THE CONTENTS IN THE PROFILE

##########################################################################################################


pack:
  content:
    images:
      - image: "{{.spectro.pack.edge-native-byoi.options.system.uri}}"
  # Below config is default value, please uncomment if you want to modify default values
  #drain:
    #cordon: true
    #timeout: 60 # The length of time to wait before giving up, zero means infinite
    #gracePeriod: 60 # Period of time in seconds given to each pod to terminate gracefully. If negative, the default value specified in the pod will be used
    #ignoreDaemonSets: true
    #deleteLocalData: true # Continue even if there are pods using emptyDir (local data that will be deleted when the node is drained)
    #force: true # Continue even if there are pods that do not declare a controller
    #disableEviction: false # Force drain to use delete, even if eviction is supported. This will bypass checking PodDisruptionBudgets, use with caution
    #skipWaitForDeleteTimeout: 60 # If pod DeletionTimestamp older than N seconds, skip waiting for the pod. Seconds must be greater than 0 to skip.
options:
  system.uri: "{{ .spectro.pack.edge-native-byoi.options.system.registry }}/{{ .spectro.pack.edge-native-byoi.options.system.repo }}:{{ .spectro.pack.edge-native-byoi.options.system.k8sDistribution }}-{{ .spectro.system.kubernetes.version }}-{{ .spectro.pack.edge-native-byoi.options.system.peVersion }}-{{ .spectro.pack.edge-native-byoi.options.system.customTag }}"


  system.registry: ttl.sh
  system.repo: ubuntu
  system.k8sDistribution: k3s
  system.osName: ubuntu
  system.peVersion: v4.2.3
  system.customTag: demo
  system.osVersion: 22
  1. Copy and save this output and formatting to a notepad, it will be used later for building the cluster profile.

Validation

  1. Validate that the Docker Images are created.
docker images

Sample Output

docker images
REPOSITORY          TAG                                   IMAGE ID       CREATED          SIZE
ttl.sh/ubuntu       k3s-1.26.8-v4.2.3-demo                2437e667204a   5 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.26.8-v4.2.3-demo_linux_amd64    2437e667204a   5 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.26.4-v4.2.3-demo                40e49b2487ee   5 minutes ago    4.31GB
ttl.sh/ubuntu       k3s-1.26.4-v4.2.3-demo_linux_amd64    40e49b2487ee   5 minutes ago    4.31GB
ttl.sh/ubuntu       k3s-1.25.2-v4.2.3-demo                4559ec6471b0   5 minutes ago    4.28GB
ttl.sh/ubuntu       k3s-1.25.2-v4.2.3-demo_linux_amd64    4559ec6471b0   5 minutes ago    4.28GB
ttl.sh/ubuntu       k3s-1.27.2-v4.2.3-demo                3087dad84bce   5 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.27.2-v4.2.3-demo_linux_amd64    3087dad84bce   5 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.27.5-v4.2.3-demo                67862466c987   6 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.27.5-v4.2.3-demo_linux_amd64    67862466c987   6 minutes ago    4.3GB
ttl.sh/ubuntu       k3s-1.25.13-v4.2.3-demo               fcc51bbb4e5b   9 minutes ago    4.32GB
ttl.sh/ubuntu       k3s-1.25.13-v4.2.3-demo_linux_amd64   fcc51bbb4e5b   9 minutes ago    4.32GB
ttl.sh/ubuntu       k3s-1.24.6-v4.2.3-demo                becbaab05cf1   10 minutes ago   4.28GB
ttl.sh/ubuntu       k3s-1.24.6-v4.2.3-demo_linux_amd64    becbaab05cf1   10 minutes ago   4.28GB
earthly/buildkitd   v0.7.4                                83e74e1d7f7b   7 months ago     246MB
  1. Validate the iso image and checksum are created.
ls -al build/

Sample Output

ls -al build/
total 1365200
drwxr-xr-x  2 root root       4096 Dec  6 13:12 .
drwxrwxr-x 11 jb   jb         4096 Dec  6 12:58 ..
-rw-r--r--  1 root root 1397948416 Apr 16  2020 pwp-edge-installer.iso
-rw-r--r--  1 root root         89 Apr 16  2020 pwp-edge-installer.iso.sha256

Next Steps

Edge Native Deployment