# Portainer and GitOps

**URL:** <https://forum.hhf.technology/t/portainer-and-gitops/353>\
**Category:** Self-hosted Applications\
**Tags:** portainer, gitops\
**Created:** [November 4, 2024, 4:47pm UTC](https://forum.hhf.technology/t/portainer-and-gitops/353 "2024-11-04T16:47:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hhf.technoloy](https://forum.hhf.technology/user_avatar/forum.hhf.technology/hhf.technoloy/32/68_2.png) [@hhf.technoloy](https://forum.hhf.technology/u/hhf.technoloy)\
**Post date:** [November 4, 2024, 4:47pm UTC](https://forum.hhf.technology/t/portainer-and-gitops/353/1 "2024-11-04T16:47:05Z")

</div>

## The Scenario

Our goal is to make use of GitOps (a variation of DevOps, using GitHub) to have a infrastructure-as-code set up, allowing us to easily manage and deploy changes with minimal touch and 1 source of truth.

Whilst Portainer is a GUI based application; it can read code from GitHub which allows it to automatically push changes based on our code.

The lifecycle of the container (starting, stopping and deleting) is still managed via the Portainer GUI.

### Prerequisites

- A server
- A GitHub account

### Recommended

- Multiple Servers
- Ubuntu Server for the OS

 ![image](https://forum-cdn.hhf.technology/original/1X/a2ed55b29c2ca31d0a16d3a59ce5b6947afeb148.png)

## Set up your Repo

### Create a Private Repo

_If you have a pre-existing repo you wish to use, skip the creation part._

[Follow GitHubs documentation on creating a repo](https://docs.github.com/en/get-started/quickstart/create-a-repo)

### Ensure Repo is Private

[Ensure your repo is set to private, or switch it to private](https://docs.publishing.service.gov.uk/manual/make-github-repo-private.html)

### Somethings to think about

#### Private vs Public repo

I would recommend creating a Private repo so you can store private information, such as API keys in it.

If you’re going to obfuscate your files with variables or secrets (which you will need to store elsewhere), you could make the repo Public. But in my opinion, its better to be safe than sorry.

_Please note that the version history is also public, so if you accidentally save a password or API key its saved forever._

#### What to name your repo

Your repo should be named something you remember and unique to you. It doesn’t really matter though. I would suggest something short so your repo url is short

## Create a Private Access Token for Portainer

As your repo is private, you will need to create a PAC for Portainer to use and access the repo

1. Click on your profile in the top right, then select Settings

2. On the left, click on Developer Settings

3. Click on Personal Access Tokens, then ‘Tokens (classic)’

4. Click on ‘Generate new token’, then select ‘Classic’

5. Input the below information\  

6. Scroll down and click on ‘Generate Token’

7. Save your PAC somewhere safe and smart

## Install Renovate Bot

The Renovate Bot watches for dependancies and automatically creates merge requests to update the contents of your Repo. This allows you to update your containers outside of Portainer as well as review changes made etc.

### Get your Repo ready

1. Create a folder ‘.github’ in the root of your repo

2. Create a folder ‘docker-compose’ in the root of your repo

3. In the ‘.github’ folder, create a file ‘renovate.json5’ with the below contents

```json
{
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": [
        "config:base",
        ":disableRateLimiting"
    ],
    "docker-compose": {
        "fileMatch": ["docker-compose/.+\\.ya?ml$"]
    }
}

```

_This code block tells the bot to watch any ‘.yml’ or ‘.yaml’ file in the ‘docker-compose’ folder_

### Install the Bot

1. You can set renovate to run on all repo’s you own, or only your repo created in this doco _This is up to you. If you actually use GitHub for development, it may be best to select only this repo_

### Confirm the bot is installed

1. Browse to your GitHub repo

2. Click on ‘Issues’, you should see the Dependency Dashboard  

## Configuration suggestion

I would suggest configuring a main Portainer with Edge agents and putting no or minimal containers on the main Portainer instance.

This config requires minimum;

- One Portainer instance (VM or Bare Metal)

- One Portainer Edge Agent (VM or Bare Metal)

The advantages of a set up like this are;

- Portainer Edge can browse Docker Volumes. The main Portainer installation cannot browse volumes, thus you cannot download, edit and upload config files from the browser

- Reduced security risk if the public facing instance is breaked

- Segregation

# Install Portainer

## Configuration suggestion

I would suggest configuring a main Portainer with Edge agents and putting no or minimal containers on the main Portainer instance.

This config requires minimum;

- One Portainer instance (VM or Bare Metal)
- One Portainer Edge Agent (VM or Bare Metal)

The advantages of a set up like this are;

- Portainer Edge can browse Docker Volumes. The main Portainer installation cannot browse volumes, thus you cannot download, edit and upload config files from the browser
- Reduced security risk if the public facing instance is breaked
- Segregation
  - A VM for test
  - a VM for prod external services (eg Minecraft server, website)
  - a VM for prod internal only services (eg DNS, PXE)

## Install Portainer

1. SSH into your Docker host and run the below command to login as root. 

2. Copy paste the below commands to install Docker (assumes Ubuntu)

3. Run the below command to install Portainer

4. Run the below command and confirm you see that the Portainer container is running

## Configure Portainer

Now that Portainer is installed, we can behind configuring it

### Account Creation

1. Browse to [https://YOURSERVERIP:9443](https://YOURSERVERIP:9443)
2. Create your credentials, I would suggest following this guide

### (Optional) Install Portainer Edge Agents

1. On the left hand pane, click on Environments
2. On the far right, click on ‘Add environment’
3. Select ‘Docker Standalone’ and click on ‘Start Wizard’
  1. Click on ‘Edge Agent Standard’

  2. Give the instance a name, such as the host name for the server / machine

  3. The Portainer API server URL should be something like:

  4. Click on Create

  5. Scroll down and locate the ‘Docker Standalone’ install script and copy this to notepad

  6. SSH into your server that will have the Edge Agent installed and log in as the Root account with the below command

  7. Copy paste the below commands to install Docker (assumes Ubuntu)

  8. Install the Edge Agent by copy pasting the text copied and altered in step 5

  9. Wait for the container to download and launch

  10. Click on ‘Home’ and confirm all your agent’s show a green Heartbeat

4. Repeat step 3 for any other Edge Agent’s you’re configuring

### Create Tags & Groups

1. On the left hand panel, click on Environments
2. Click on Tags
3. Create any relevant tags, such as ‘Test’ or ‘Production’
4. On the left, click on Groups
5. Click on Add Group
  1. Provide a name, eg ‘Test’ or ‘Production’
  2. Provide a description, eg ‘Test servers’
  3. Click on tags and select the relevant tag
  4. Click on ‘Create group’

6. Repeat step 5 for any other groups

#### Apply Tags

1. On the left hand menu, click on Home
2. Click on the pen icon next to your first Portainer instance
3. under Metadata
  1. Under Group, select the relevant group (eg ‘test’)
  2. Under Tags, select the relevant tag/s (eg ‘test’, ‘linux’)

4. Click on Update environment
5. Repeat for each Portainer instance

### Enable Edge Compute

1. On the left hand panel, click on Settings
2. Click on ‘Edge Compute’
3. tick ‘Enable Edge Compute features’

### Set up the WatchTower Edge stack

1. On the left, click on ‘Edge Stacks’ and click on ‘Create’
  1. Name your stage ‘WatchTower’
  2. Click on Edge Groups and select all your groups
  3. In the web editor, provide the below compose file and hit deploy

```yaml
version: '3'
services:
  app:
    image: ghcr.io/containrrr/watchtower:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TZ=$TZ
      - WATCHTOWER_ROLLING_RESTART=true
      - WATCHTOWER_CLEANUP=true
      #- WATCHTOWER_DEBUG=true
      - WATCHTOWER_INCLUDE_STOPPED=true
      - WATCHTOWER_POLL_INTERVAL=86400
      - WATCHTOWER_RUN_ONCE=true
      - WATCHTOWER_LABEL_ENABLE=true
    #command: --label-enable
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

```

1. Take note of the state of your stacks  
 ![image](https://forum-cdn.hhf.technology/original/1X/fa3eaca3772643809bafa2c99f89e9f7093315b3.png)

2. Click on ‘WatchTower’ and you will be shown the same screen as step 1. You can make any adjustments to the stack here, such as removing groups or editing the compose file

3. Click on the Environments tab. This will show any server the stack has been deployed too and their state. The stack will be deloyed to any devices in the groups chosen in step 1.2

This stack will enable auto-updates for anything with the `com.centurylinklabs.watchtower.enable=true` label

_Please note: This is NOT a recommended solution for updating containers. This guide will only assist you with updating Portainer, as it is potentially a public facing resource and needs to be patched. The WatchTower auto update may (and probably will) break Portainer at some stage. Keep backups._

# Create your first GipOps Compose file

## Create your first Compose file

In this example, we’re going to massage the [LinuxServer.io](http://LinuxServer.io)’s Ombi image to match our best practices, per below

### Create your Compose file

1. Open your GitHub repo and enter the docker-compose folder

2. Click on ‘Add file’ \> ‘Create new file’

3. Name the file ‘ombi.yml’, and copy the compose file [from here](https://docs.linuxserver.io/images/docker-ombi)  
It should look something like this:  

4. Refer to the best practices and tips documentation below and edit the file

5. Hit ‘Commit changes…’

6. hit ‘Commit changes’

Oh, you want **me** to give you the answer to step 4?

As of writing, here is the ombi.yml compose file

```yaml
---
version: "2.1"
services:
  ombi:
    image: lscr.io/linuxserver/ombi:latest
    container_name: ombi
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - BASE_URL=/ombi #optional
    volumes:
      - /path/to/appdata/config:/config
    ports:
      - 3579:3579
    restart: unless-stopped

```

Here is my tweaked file

```yaml
---
version: "2.1"
services:
  ombi:
    image: ghcr.io/linuxserver/ombi:4.39.1
    #container_name: ombi
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=$TZ
      - BASE_URL=/$BASE_URL #optional
    volumes:
      - config:/config
    ports:
      - $PORT_HTTP:3579
    restart: unless-stopped
volumes:
  config:

```

### Best Practices and tips

#### Variables

To keep your compose files as system agnostic as possible, its best to use variables where you can. Some suggestions are

- Ports (eg $HTTP\_PORT, $DB\_PORT, $API\_PORT)
- Mounted volumes (eg $CONFIG, $DATA)
- Timezones (eg $TZ)

These variables can then be set in Portainer when creating the stack

#### Don’t name containers unless there is a requirement for it

Some compose files may name a database ‘db’, which will then block you from using that name anywhere else. Its best to let Portainer manage names of containers.

#### Container Image versioning

Do not use the ‘latest’ tag, as it does not allow Renovate to update the compose file

When versioning your compose files, locate the ‘latest’ tag on the DockerHub, GitHub etc and use the relevant version number.

 ![image](https://forum-cdn.hhf.technology/original/1X/73463f340fa21dc4d08bed71bca5245a0fa88806.png)

This is harder to do on DockerHub. Per the screenshot below, we’ve located the ‘tatest’ tag and used the Digest to locate the correct version.

 ![image](https://forum-cdn.hhf.technology/original/1X/d18062e60ae72aec5f61a27130691d4ce4a58f01.png)

#### Container Image locations

As Docker has now started rate limiting non-paying users, a lot of container creators are putting their containers on GitHub or other services

Its quite easy to figure out if a container exists on GitHub;

1. If the ‘Packages’ option exists, the container is on GitHub\

2. The container URL is underlined\  

# Deploy your first GitOps stack

## What is a Stack?

Stacks is how Portainer handles Docker Compose files, which are infrastructure as code documents for spinning up multiple containers, volumes and networks in 1 go.

Most docker container developers now provide example compose files with their projects. Its also possible to Google and find examples online

* * *

For example, you may have a service that has a separate container for

- _a database_
- _a web app_
- _back end compute_

_The database and back end compute may be on Network 1, while the web app is on Network 2._

_All 3 containers have access to the same ‘config’ volume, but the database and webapp have their own unique volumes_

_The database and backend need the same credentials to the 2 systems can talk, but the web app has separate credentials for the gui. These are environmental variables_

* * *

Instead of manually setting up and inputting environmental variables, networks and volumes for each container this can all be written into the compose and and span up in 1 click. This also makes the set up system agnostic.

## Deploy your Stack

In this practice, we’re going to deploy the Ombi compose file we created in create-your-first-gipops-compose-file.md

1. Log into Portainer

2. On the left hand menu, select ‘Stacks’

3. Click on ‘Add stack’

If you have any issues, refer to the Notifications tab at the top of the page;

 ![image](https://forum-cdn.hhf.technology/original/1X/8509970a1a1709942bbc439e756cfd8b02a954b2.png)

# Managing your production compose files

## Updating Containers

Updates are managed via the renovate bot and automatically pulled by Portainer

### Updating the Compose file

When a new update is released, the renovate bot will create a pull request in your repo;

 ![image](https://forum-cdn.hhf.technology/original/1X/37cd049d446a8cbc8070f5fd2c57a093895e1f5b.png)

We can see that this pull request is updating the ‘package’, ‘mariadb’. It is a minor update, going from version 10.5 to 10.11

Clicking on the ‘Files changed’ tab will list any files that will be updated;

 ![image](https://forum-cdn.hhf.technology/original/1X/6ee16d320eccfa0ee4be0e98b312be3161c51fe7.png)

If we are happy with this update, we can click on the ‘Merge pull request’ button at the bottom of the homepage

### Updating Portainer

Portainer will check in with GitHub and download the updated compose file and update the container image.

You can tell Portainer to manually pull by;

1. Browse to the stack
2. Click on ‘Pull and redeploy’

## Changing Compose Files

Compose files can be changed in your GitHub repo. These will sync automatically to Portainer

---

<div class="post-metadata">

**Author:** ![gyrene2083](https://forum.hhf.technology/user_avatar/forum.hhf.technology/gyrene2083/32/67_2.png) [@gyrene2083](https://forum.hhf.technology/u/gyrene2083)\
**Post date:** [November 5, 2024, 2:39pm UTC](https://forum.hhf.technology/t/portainer-and-gitops/353/2 "2024-11-05T14:39:20Z")

</div>

Thank you for this. I actually just finished testing a similar project. I actually setup gitea in portainer with renovate. Just finished setting up the bot last night.