*ฅ^•ﻌ•^ฅ* ✨✨  HWisnu's blog  ✨✨ о ฅ^•ﻌ•^ฅ

Coolify part 3 : deploying via Docker

Introduction

Sometimes the app that we are building have a specific dependencies which cannot be added just by including it then run bun install / uv sync / pip install. That is what I've had to deal with which become the inspiration of this post.

However this post will not only discuss about how we built our own Docker image, but also how we can apply integration with Github Actions so all of this process becomes an automatic CI/CD.

Pre-build stage

The dependency I mentioned was TA-Lib, which is a technical analysis library written in C. This C program is a requirement before we can start using the Python wrapper under the same name. Before building the Docker Image, first we got to have our Dockerfile, mine is as follow:

FROM mcr.microsoft.com/devcontainers/python:1-3.10-bookworm

RUN mkdir /code
WORKDIR /code

COPY install_talib.sh /code/install_talib.sh
RUN chmod +x /code/install_talib.sh && /code/install_talib.sh

COPY requirements.txt /code/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

# Create a non-root user and set permissions
RUN useradd -m -u 1001 user
USER user
ENV HOME=/home/user PATH=/home/user/.local/bin:$PATH

WORKDIR $HOME/app
COPY --chown=user . $HOME/app

EXPOSE 3000

CMD ["python", "app.py"]

The install_talib.sh

wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz &&
    tar -xzf ta-lib-0.4.0-src.tar.gz &&
    cd ta-lib/ &&
    ./configure --prefix=/usr &&
    make &&
    sudo make install

As for requirements.txt you can fill in the libraries you need for your project, mine:

quart==0.18.4
Werkzeug==2.2.2
pandas==1.4.1
numpy==1.22.2
ta-lib==0.4.24

Note if you check the versions above it's an older version of the libraries. This is due to the project is an old project I made a couple of years ago, which I recently moved to a self hosted infrastructure instead of using 3rd party host.

Build and Test the Docker Image

When dealing with our own images we need to make sure the app is working as intended. Run the command:

Make sure the program built and ran well before proceeding to the next stage. Next you need to have a Docker repo, you can use Docker Hub or Github Container Registry (ghcr) or Azure Container Registry (acr). I'm not interested on providing the step by step on creating a Docker repo coz it's a 101 topic, there are lots of external references you can find.

Note the next stages assumes we are working with a private Docker image, so please put that in mind.

Tag and Push to Docker Hub / Container Registry

Next is tagging and then push the image, with these simple commands:

  1. docker tag myapp:latest myrepo/myapp:latest
  2. docker push myrepo/myapp:latest

Once you've finished the above steps, check your Docker repo to see if you've successfully pushed the image. Next stages we are going to start dealing with Coolify and how to make the process an automated CI/CD.

Coolify

Here comes the fun part where you "Vercelify" aka automate everything via Coolify. Please bear with me coz the chain of processes is a bit long and could be seen as complicated. Well everything's complicated in the first time, not so much after! Make sure you have read the first and second part of my Coolify series before continuing!

Note don't use port mappings if you want to do rolling release when everytime you're deploying / rebuilding your app. Rolling release enables the app to keep working while we are rebuilding it, so the app still accessible to users. Non-rolling release will stop working when we are rebuilding the app..if it takes 5 minutes to rebuild, then your app will not be accessible for 5 minutes. If you do rebuilding several times a day, now you get the idea how annoying this is for your users.

Chain of continuous deployment

Next step we are going to make Github and Coolify to work together for continuous deployment. You need to get credentials from several sources: Docker and Coolify.

Docker credentials

This is depending on the container registry you're using, in my case I'm using Docker Hub.

Coolify credentials

Yellow = URL, Red = Resource ID.

Add these credentials to Github secrets

Go to your Github repo's Settings -> Secrets and variables -> Actions

gh-secrets-actions

Expand Github Actions workflow

Next we need to use these credentials into the expanded github actions. Sample yml file, pay attention to where we are using the credentials in each stage of the process:

name: Run with Docker Image -- process data, rebuild & push image

on:
  schedule:
    - cron: '0 14 * * 1-5'
  workflow_dispatch:

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    permissions:
      contents: write

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Log in to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_REPO_USERNAME }}
          password: ${{ secrets.DOCKER_REPO_PASSWORD }}

      - name: Pull pre-built Docker image
        run: docker pull myrepo/myapp:latest

      - name: Run Docker container and execute script
        run: |
          docker run --rm \
            -v $(pwd):/home/user/app \
            myrepo/myapp:latest \
            python process_data.py

      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: myrepo/myapp:latest

      - name: Commit Files (if needed)
        run: |
          git config --local user.email "action@github.com"
          git config --local user.name "GitHub Action"
          git add -A
          git commit -m "Automated : process data, rebuild and push Docker image" || echo "No changes to commit"

      - name: Push Changes (if needed)
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          branch: master
  
  coolify-deployment:
    runs-on: ubuntu-latest
    needs: build-and-push
    steps:
      - name: Trigger Coolify deployment via API Token
        uses: fjogeleit/http-request-action@v1
        with:
          url: ${{secrets.COOLIFY_URL}}/api/v1/deploy?uuid=${{secrets.COOLIFY_RESOURCE_ID}}&force=false
          method: GET
          bearerToken: ${{secrets.COOLIFY_API_TOKEN}}

Conclusion

So there you go! You have your own "better Vercel" where everything is automated everytime you push an update, or when your Github Actions updates something in the Docker image. You don't need to touch anything, it just works.

Why "better Vercel" you ask? Coz Vercel does not support deploying via Docker image which is sometimes a necessary part of deploying / building our apps.

This post took me 1 week to write coz I need to recollect every stages of the process. As I've said previously: this CI/CD DevOps stuff is somewhat harder compared to writing programs even in the low level languages such as C-Zig-Rust, due to all the various moving parts involved.

What's next

I'm afraid my low level posts need to be postponed for a while (pushed back several weeks?) due to me having to deal with mounting technical debt (the dependencies are getting old / obsolete). I need to refresh and update to newer versions, and this requires a lot of work.

But I might pop in some short posts here and there, we'll see!