A/B testing is an essential part at the core of a business. Many companies don't release a newer version of their product to customers before they are 100% sure that the newer version with all of its features works properly. A working feature, from a business standpoint is a feature that converts well, whereas from a technical perspective, it's a feature that does not contain any bugs or errors.
Eventually both points of views lead to the same goal, which is improvement of the overall performance and having happy customers.
In this tutorial I want to explain how simple an A/B test could be using <a href="https://en.wikipedia.org/wiki/Nginx">NGINX</a>.
I'll start from the basics and add further explanations down the road.
Let's suppose we have an application that has a simple index.html page:
<pre><code class="hljs language-html">&#x3C;!DOCTYPE html>
&#x3C;html lang="en">
 &#x3C;head>
 &#x3C;meta charset="UTF-8" />
 &#x3C;meta http-equiv="X-UA-Compatible" content="IE=edge" />
 &#x3C;meta name="viewport" content="width=device-width, initial-scale=1.0" />
 &#x3C;title>My Application&#x3C;/title>
 &#x3C;/head>
 &#x3C;body>
 &#x3C;style>
 .green {
 color: white;
 background: green;
 text-transform: uppercase;
 }
 &#x3C;/style>
 &#x3C;button class="green">Add to Cart&#x3C;/button>
 &#x3C;/body>
&#x3C;/html>
</code></pre>
I'll use basic HTML5 structure, so it's easier for everyone to follow.
If we open this page in the browser we'll see a button with a text ADD TO CART in a green color.
<img src="/img/posts/ab_testing_button_green.png" alt="Green Button" title="Green Button">
Now for some reason this button doesn't convert well enough and the UI/UX team decided they want to try a different color. Of course the developers could just change the color or maybe the UI/UX team can pass this task to the content team and they will log into some Content Management System (CMS) and change the color, but doing that affects all traffic.
What if the new color that they want to change for the button is not effective enough ? What if it even drops the conversion rate ? How can we test that ?
So the UI/UX team asked the developers, "Hey, we would like to have this button in a blue color, but please try it first on 5% of the traffic!".
Alright, let's implement an A/B test for this button, we'll send only 5% of the traffic and monitor how well the new color converts, after we're sure this button works better, we'll expose it to the rest of the customers.
Ok, so how can we achieve that ? given that we have the following NGINX default.conf configuration file:
<pre><code class="hljs language-nginx">server {
 listen 80;
 server_name localhost;

 location / {
 root /usr/share/nginx/html;
 index index.html;
 }
}
</code></pre>
(I kept the above configuration very basic, NGINX creates a server that listens on port 80 bounded to localhost, when the user request / path, i.e <a href="http://localhost:80/">http://localhost:80/</a>, we serve the index.html from /usr/share/nginx/html path, i.e /usr/share/nginx/html/index.html).
But of course we need first a running NGINX web-server.
For the purpose of simplicity I'll use docker to run an NGINX web-server that uses the above configuration file (feel free to install NGINX however you like in order to follow along):
<pre><code class="hljs language-sh">docker run --rm -it -v ${PWD}/html:/usr/share/nginx/html -v ${PWD}/conf.d:/etc/nginx/conf.d -p 80:80 nginx:alpine
</code></pre>
My directory structure is as follows:
<pre><code class="hljs language-sh">├── conf.d
│ └── default.conf
└── html
 └── index.html
</code></pre>
Now that we have everything setup, let's go back to the button. How do we allow only 5% of the traffic to see the blue button like the UI/UX team originally requested ?
Well, we need to tweak the NGINX configuration a bit, since we're dealing here with simple HTML page, we'll use <a href="https://en.wikipedia.org/wiki/Server_Side_Includes">Server Side Includes (SSI)</a> approach, where we replace only a certain part of the HTML for a specific group (i.e group A or B):
<pre><code class="hljs language-nginx">split_clients "${remote_addr}V1" $appversion {
 95% version_a;
 * version_b;
}

server {
 ...
 location / {
 ssi on;
 set $group $appversion;
 ...
 }
}
</code></pre>
Notice that I modified the NGINX configuration file a bit, I've turned on SSI in the location block / and I set a variable called group to whatever comes out from split_clients module.
split_client module decides to what group a particular client request belongs to based on the hashed value, in our case we use the IP address of the client since it's unique per client we can rely on it, but in general for the hash you could use any type of unique string.
Notice I also concatenate V1 to the IP address string, this is often referred to as <a href="https://en.wikipedia.org/wiki/Salt_(cryptography)">salt</a> for the hash, it's necessary in case we want to run a completely new experiment, when I change the V1 to V2 it will reshuffle the groups completely and ensure that we get a fresh new experiment.
split_clients ensures 95% of the visitors will get version_a and 5% (which is the rest) will get version_b and this value will be assigned to appversion variable.
That's it for the NGINX configuration, but how do we use it on the frontend ?
Let's tweak the frontend a bit:
<pre><code class="hljs language-html">&#x3C;!DOCTYPE html>
&#x3C;html lang="en">
 ...
 &#x3C;body>
 &#x3C;style>
 .green {
 color: white;
 background: green;
 text-transform: uppercase;
 }
 .blue {
 color: white;
 background: blue;
 text-transform: uppercase;
 }
 &#x3C;/style>
 Group:
 &#x3C;!--# echo var="group" -->
 &#x3C;br />
 IP Address:
 &#x3C;!--# echo var="remote_addr" -->
 &#x3C;br />
 &#x3C;!--# if expr="$group=/version_a/" -->
 &#x3C;button class="green">Add to Cart&#x3C;/button>
 &#x3C;!--# else -->
 &#x3C;button class="blue">Add to Cart&#x3C;/button>
 &#x3C;!--# endif -->
 &#x3C;/body>
&#x3C;/html>
</code></pre>
All I did is adding a new CSS class as a new style for the new button, as the UI/UX team requested it to be blue and then with an if check we're checking for a match of the group if it's version_a we'll show the green button, otherwise we'll show the blue one. I also added some debugging information, like group and IP address so we can understand what's happening.
Let's rerun the nginx web-server (<code>Ctrl+C</code> to exit out of the previous nginx container):
<pre><code class="hljs language-sh">docker run --rm -it -v ${PWD}/html:/usr/share/nginx/html -v ${PWD}/conf.d:/etc/nginx/conf.d -p 80:80 nginx:alpine
</code></pre>
Ok, after restarting you should be able now to see either the blue or the green button based on the group nginx assigned your ip to and that's the gist of it.
If you're curious what happens under the hood, how does NGINX make all of the "magic" happen, the <a href="https://github.com/nginx/nginx/blob/master/src/core/ngx_murmurhash.c">implementation</a> NGINX is using is an algorithm called <a href="https://en.wikipedia.org/wiki/MurmurHash">murmur_hash2</a>.
SSI is a good use-case when you just want to replace small chunks / segments of the UI on the same application.
However, the split_client module isn't just for that particular use-case, it's possible to also use it in the context of entirely different upstream applications.
Let's take the previous example request from the UI/UX team and now instead of replacing just the button, they want to run another experiment, a new application that was developed by another team. They want to check which application converts better.
They shared with us that url of the new application, <a href="http://localhost:3000">http://localhost:3000</a>, now of course this is all made up, let's tweak the nginx configuration to make that happen:
<pre><code class="hljs language-nginx">split_clients "${remote_addr}V1" $appversion {
 50% application_a;
 * application_b;
}

upstream application_a {
 server 127.0.0.1:8080;
}

upstream application_b {
 server 127.0.0.1:3000;
}

server {
 listen 80;
 server_name localhost;

 location / {
 proxy_pass http://$appversion;
 }
}

server {
 listen 8080;
 server_name application_a;

 location / {
 root /usr/share/nginx/html;
 index index.html;
 }
}

server {
 listen 3000;
 server_name application_b;

 location / {
 root /usr/share/nginx/html;
 index index2.html;
 }
}
</code></pre>
I'll explain what I've changed here. Instead of 1x server block we now have 3x server blocks, the very top one is the one that passes the client requests using proxy_pass module to the downstream applications (i.e application_a and application_b).
For demonstration purposes I've just created another index2.html file, which will resemble the other new application that listens on port 3000.
At the top you can see we have two upstreams, one for each application.
split_client was slightly changed, for readability purposes instead of returning version_a or version_b I called it application_a or application_b.
Since UI/UX would like to know which application performs better, we need a fair 1 to 1 comparison, I'll split the traffic by 50% 50%.
That's it for the nginx configurations.
Now what about the frontend ? what have I changed there, index.html now looks pretty simple, without custom if checks:
<pre><code class="hljs language-html">&#x3C;!DOCTYPE html>
&#x3C;html lang="en">
 &#x3C;head>
 &#x3C;meta charset="UTF-8" />
 &#x3C;meta http-equiv="X-UA-Compatible" content="IE=edge" />
 &#x3C;meta name="viewport" content="width=device-width, initial-scale=1.0" />
 &#x3C;title>Application A&#x3C;/title>
 &#x3C;/head>
 &#x3C;body>
 This is Application A
 &#x3C;/body>
&#x3C;/html>
</code></pre>
For demonstrating purposes I duplicated the above file and called it index2.html that has the title "Application B" with the body "This is Application B".
Let's rerun the nginx web-server (<code>Ctrl+C</code> to exit out of the nginx container):
<pre><code class="hljs language-sh">docker run --rm -it -v ${PWD}/html:/usr/share/nginx/html -v ${PWD}/conf.d:/etc/nginx/conf.d -p 80:80 nginx:alpine
</code></pre>
Now based on your client IP address there is 50% chance that you get either Application A or B.
The split_clients module is great, but are there any alternative ways of achieving similar results ?
Of course there are, consider the above mentioned example. Let's say we have two different applications that are represented by different index.html files.
We can also use server weight, on a single upstream to tell NGINX what weight of the visitors (imagine buckets of clients) an upstream server should get out of the total weight available in the upstream block.
Consider the following nginx configuration, in particularly 3rd part from the left in the upstream a_b_testing:
<pre><code class="hljs language-nginx">upstream a_b_testing {
 server 127.0.0.1:8080 weight=50;
 server 127.0.0.1:3000 weight=50;
}

server {
 listen 80;
 server_name localhost;

 location / {
 proxy_pass http://a_b_testing;
 }
}

server {
 listen 8080;
 server_name application_a;

 location / {
 root /usr/share/nginx/html;
 index index.html;
 }
}

server {
 listen 3000;
 server_name application_b;

 location / {
 root /usr/share/nginx/html;
 index index2.html;
 }
}
</code></pre>
This might looks a bit simpler, it's another way of running an A/B test in NGINX, notice the weight, I gave 50 weight to the first upstream server, and 50 weight to the second upstream server, what nginx would do is calculate the total weight on that block, and sense 50 + 50 is equal 100, these values will be considered in percentage.
Give it a try, notice that when you run this, and refresh the browser, something unexpected is happening, you will be bounced from Application A to Application B on each browser refresh.
Why is it happening ?
Well we just told NGINX the percentage we want each application to be prioritized, but what we didn't tell NGINX is based on what criteria we want to split it, by default NGINX is using <a href="https://en.wikipedia.org/wiki/Round-robin_DNS">round-robin</a> technique that sends the clients to the list of servers that is supplied inside of the upstream block, one after the other. Of course this does not fit our intended use-case, so we can change that default behavior, let's use also the IP address of the client, since it's unique for each visitor. We can use the directive ip_hash inside of the upstream block in order to achieve that, and if you run the test again, you will notice that once you hit one of the applications, you stay on each browser refresh on that same application.
Just modify the configurations a bit as follows:
<pre><code class="hljs language-nginx">upstream a_b_testing {
 ip_hash;
 server 127.0.0.1:8080 weight=50;
 server 127.0.0.1:3000 weight=50;
}
...
</code></pre>
<h2>Conclusion</h2>
What did we learn ?
A/B testing is great for maintaining a great customer's experience.
It's very straightforward to run an A/B test with NGINX using split_client module.
With the split_client module we can either split the traffic between two entirely different applications or within the same application by replacing HTML parts of the application using SSI.
Depending how simple your use-case is, you could run an A/B test using NGINX weight flag, if you set an upstream server to 20 and the other one to 80, it will be considered in percentage, that means that one app will receive 20% of the traffic and the other one will receive 80%. Caution - it's important to use ip_hash directive, to let NGINX know that the weight of the upstream servers will be considered by the hash string made from the client's IP address, to ensure <a href="https://wiki.genexus.com/commwiki/servlet/wiki?45282,Server+Affinity+%28Sticky+Sessions%29">sticky sessions</a> for each unique visitor, this will ensure these clients will receive the same server / app the next time they visit.
We learned the fundamentals of how an A/B test in NGINX works, if you understand this, it will be fairly simple to understand how this works on Kubernetes - I'll create a tutorial for this too. Kubernetes only abstract some of the core concepts here and make it a bit simpler to deploy.
Additionally, you can see how split_client module isn't just good for A/B testing, rather it is also good for <a href="https://en.wikipedia.org/wiki/Feature_toggle">Canary deployments</a>, where you want to shift a smaller portion of the traffic just to verify the new version of the application truly works before you shift the entire traffic, to minimize the damage.
(For simplicity and readability purposes I didn't include flags like proxy timeouts etc, note that those are necessary for the production environment and you don't want to rely on nginx default configurations, so make sure you research about the proper value before applying those configurations to production).

<code>ArgoCD</code> is a declarative, <code>GitOps</code> continuous delivery tool for Kubernetes. It follows the <code>GitOps</code> principle of using Git as a 'single source of truth' for declarative infrastructure and applications. One of the powerful features of <code>ArgoCD</code> is the ability to create preview environments.
<h2>What is a Preview Environment?</h2>
A preview environment is a replica of your production environment where you can preview your changes before they go live. This includes not just application code changes, but also infrastructure, configuration, and more.
<h2>Why Use ArgoCD for Preview Environments?</h2>
<code>ArgoCD's</code> declarative model is perfectly suited for managing preview environments. By defining your environment as code in a Git repository, <code>ArgoCD</code> ensures that your Kubernetes cluster aligns with this desired state. This not only simplifies the process of creating and dismantling environments as required but also optimizes resource utilization. By automatically tearing down unused environments, <code>ArgoCD</code> helps to prevent wasting resources, leading to significant cost savings and easier maintenance.
<h2>How to Create a Preview Environment with ArgoCD</h2>
To create a preview environment with <code>ArgoCD</code>, you would typically follow these steps:
Define your environment as code in a Git repository. This could include Kubernetes manifests, Helm charts, Kustomize applications, and more.
Create an <code>ArgoCD</code> Application that points to this Git repository and the desired path within it:
<pre><code class="hljs language-yaml">apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
 name: preview-environments
spec:
 generators:
 - pullRequest:
 github:
 # The GitHub organization or user.
 owner: myorg
 # The Github repository
 repo: myrepository
 # For GitHub Enterprise (optional)
 api: https://git.example.com/
 # Labels is used to filter the PRs that you want to target. (optional)
 labels:
 - preview
 requeueAfterSeconds: 1800
 template:
 metadata:
 name: 'myapp-{{.branch}}-{{.number}}'
 spec:
 project: 'my-project'
 source:
 repoURL: 'https://github.com/myorg/myrepo.git'
 targetRevision: '{{.head_sha}}'
 path: kubernetes/
 helm:
 parameters:
 - name: 'image.tag'
 value: 'pr-{{.head_sha}}'
 destination:
 server: https://kubernetes.default.svc
 namespace: default
</code></pre>
The template in this context is essentially a standard <code>ArgoCD Application</code>. Assuming you're already familiar with creating an <code>ArgoCD Application</code>, let's provide a quick recap. The template requires you to specify the application's name, the repository, and the path to the Helm chart. Lastly, you need to define the target cluster and namespace for the deployment.
The <code>generators</code> field is where you specify how new <code>Applications</code> should be created. In this case, it's using a pullRequest generator with GitHub as the source. This generator will create a new Application for each open Pull Request in the specified GitHub repository (myorg/myrepository).
Use ArgoCD's CLI or API to create a new instance of this Application for your preview environment. You can parameterize your Application to customize each environment as needed (for example, to use a different database for each environment).
ArgoCD will then sync this Application with your Kubernetes cluster, creating the necessary resources.
Once you've finished with the preview environment, there's no need for manual cleanup. The <code>requeueAfterSeconds</code> field is configured to 1800 seconds (equivalent to 30 minutes). This means that the generator will automatically scan for new Pull Requests every half an hour and remove any preview applications that are no longer linked to an existing Pull Request.
<h2>Pull Requests / Merge to Deploy</h2>
As with any technology, there's always a possibility of misinterpreting its purpose and using it incorrectly.
In traditional development workflows, developers often deploy changes by clicking a button. While this may seem like the correct approach initially, it's not the most effective method when using <code>ArgoCD</code>.
<code>ArgoCD</code> operates on the principle of declarative configuration. This means you specify the desired state of your application, rather than the steps to achieve it. Consequently, the concept of a 'deploy button' doesn't align with <code>ArgoCD's</code> philosophy.
While it's technically feasible to keep an application <code>out of sync</code> and only initiate the <code>sync</code> when you're ready to deploy, this approach is strongly discouraged. It's generally considered poor practice for several reasons. For instance, consider a scenario where a colleague returns from vacation and, unaware of the current state, triggers the sync from the UI after a week-long delay. This could inadvertently result in an idle feature being deployed to production, which was not the colleague's intention.
Instead, a more fitting approach with <code>ArgoCD</code> is to utilize Pull Requests. These Pull Requests can be reviewed and approved before merging. In this context, the act of merging a Pull Request effectively becomes the 'deploy action'. Once a Pull Request is merged, the changes are deployed, aligning with the declarative nature of <code>ArgoCD</code>.
Within a Pull Request, it's also possible to identify the changes that caused the application to break. These changes can then be easily reverted, adhering to <code>GitOps</code> principles.
<h2>Conclusion</h2>
<code>ArgoCD's</code> preview environments provide a powerful tool for testing changes in a safe, isolated environment before they go live. By defining your environments as code and using <code>ArgoCD</code> to manage them, you can ensure consistency and repeatability across your environments, reducing the risk of issues in production.

In this tutorial I want to cover how powerful raw HTML and Javascript could be and how they could improve the speed of your website.
I'll be covering an important fundamental concept in the web industry, which is a Static Site Generator(SSG).
<h2>What is a Static Site Generator(SSG) ?</h2>
A static site generator is a tool that generates a full static HTML website based on raw data and a set of templates. Essentially, a static site generator automates the task of coding individual HTML pages and gets those pages ready to serve to users ahead of time.
Browsers understand Javascript and HTML natively, by doing so the website is going to gain a boost of performance.
<h2>How to use SSG</h2>
For this tutorial I'm going to use NextJS version 12.0.9 with typescript, if you don't want to read all of this you can also checkout the <a href="https://github.com/edenreich/examples/nextjs-ssg">repository</a> I've created on github.
Let's create new NextJS with typescript app:
<pre><code class="hljs language-sh">mkdir nextjs-ssg &#x26;&#x26; cd nextjs-ssg
yarn create next-app --typescript .
</code></pre>
Open it with your favorite code editor, I'm going to use vscode:
<pre><code class="hljs language-sh">code .
</code></pre>
The folder structure should look like this:
<pre><code class="hljs language-sh">.
├── README.md
├── next-env.d.ts
├── next.config.js
├── package.json
├── pages
│ ├── _app.tsx
│ ├── api
│ └── index.tsx
├── public
│ ├── favicon.ico
│ └── vercel.svg
├── styles
│ ├── Home.module.css
│ └── globals.css
├── tsconfig.json
└── yarn.lock
</code></pre>
First thing I like to do is ensure that I can package this application using docker, so we can later on deploy it to any other cloud vendors.
Let's create a simple Dockerfile:
<pre><code class="hljs language-dockerfile">FROM node:17.1.0-alpine3.12 AS development
WORKDIR /app
ENV HOST=0.0.0.0
ENV PORT=3000
ENV NODE_ENV=development
EXPOSE 3000
CMD [ "yarn", "dev" ]
</code></pre>
This dockerfile is based on node:17.1.0-alpine3.12 and I've created only a development target and common C libraries for alpine, alpine images are normally very small and a lot of things needs to be explicitly installed.
Alright, let's build and run it:
<pre><code class="hljs language-sh">docker build -t blog --target development .
docker run --rm -it --name blog -p 3000:3000 -v ${PWD}:/app -w /app blog
</code></pre>
Open <a href="http://localhost:3000">http://localhost:3000</a> and you should see the default index page of the NextJS app.
Now that we have a layer for development we also need a layer for production, in production we won't use volumes, instead we'll generate a static site with NextJS and copy it over to an nginx alpine container, NGINX is the web server that will serve all of the generated HTML and Javascript files.
Let's add dependencies, builder and production layers to the dockerfile:
<pre><code class="hljs language-dockerfile">...
FROM node:17.1.0-alpine3.12 AS dependencies
ENV NODE_ENV=production
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile

FROM node:17.1.0-alpine3.12 AS builder
ENV NODE_ENV=development
WORKDIR /app
COPY . .
RUN yarn install --frozen-lockfile &#x26;&#x26; NODE_ENV=production yarn build

FROM node:17.1.0-alpine3.12 AS production
WORKDIR /app
ENV HOST=0.0.0.0
ENV PORT=3000
ENV NODE_ENV=production
COPY --chown=node --from=builder /app/next.config.js ./
COPY --chown=node --from=builder /app/public ./public
COPY --chown=node --from=builder /app/.next ./.next
COPY --chown=node --from=builder /app/yarn.lock /app/package.json ./
COPY --chown=node --from=dependencies /app/node_modules ./node_modules
USER node
EXPOSE 3000
CMD [ "yarn", "start" ]
</code></pre>
I'll shortly explain what the above means:
<ul>
<li>Docker dependencies layer - builds the node_modules for production</li>
<li>Docker builder layer - builds the NextJS framework optimized version for production</li>
<li>Docker production layer - this is the final layer, here is where I copy all necessary artifacts and create the final container image (~ 354MB)</li>
</ul>
To build the application for production we can use:
<pre><code class="hljs language-sh">docker build -t blog --target production .
</code></pre>
For development we can just mount a volume to our local environment:
<pre><code class="hljs language-sh">docker build -t blog-development --target development .
docker run --rm -it --name blog -p 3000:3000 -v ${PWD}:/app blog-development /bin/sh
yarn
yarn dev
</code></pre>
Now that we have a way to ship the application from development to production let's continue with the application layer.
First thing I like to do on brand new NextJS application, is to remove whatever I'm not using, so the first thing would be to remove the API endpoints:
<pre><code class="hljs language-sh">rm -rf ./pages/api
</code></pre>
Let's create a new dynamic page, I'll call it [slug].tsx, slug will be captured as props to our page component:
<pre><code class="hljs language-tsx">import type { NextPage } from 'next';
import Head from 'next/head';

const Blog: NextPage = () => {
 return (
 &#x3C;div>
 &#x3C;Head>
 &#x3C;title>My Blog&#x3C;/title>
 &#x3C;meta name="description" content="My Blog" />
 &#x3C;link rel="icon" href="/favicon.ico" />
 &#x3C;/Head>
 &#x3C;main>&#x3C;/main>
 &#x3C;footer>&#x3C;/footer>
 &#x3C;/div>
 );
};

export default Blog;
</code></pre>
Let's adjust this opened file to generate the static content ahead of time, capture a dynamic slug at runtime and fetch the right resource:
<pre><code class="hljs language-tsx">import type {
 NextPage,
 GetStaticProps,
 GetStaticPaths,
 GetStaticPropsContext,
 GetStaticPropsResult,
 GetStaticPathsResult,
} from 'next';
import Head from 'next/head';
import { PropsWithChildren } from 'react';

type Request = {
 slug: string | string[];
};

type Page = {
 slug: string;
 title: string;
 content: string;
};

const mockedPages: Page[] = [
 {
 slug: 'about',
 title: 'About',
 content: 'About page',
 },
 {
 slug: 'contact',
 title: 'Contact',
 content: 'Contact page',
 },
];

export const getStaticPaths: GetStaticPaths = async (): Promise&#x3C;
 GetStaticPathsResult&#x3C;Request>
> => {
 // Get all existing paths from an API for generating static pages ahead of time
 // Probably use Contentful API to get this data
 // But for now let's mock that data
 const allPossiblePaths = mockedPages.map((page) => ({
 params: { slug: page.slug },
 }));
 return {
 paths: allPossiblePaths,
 fallback: false,
 };
};

export const getStaticProps: GetStaticProps = async (
 context: GetStaticPropsContext
): Promise&#x3C;GetStaticPropsResult&#x3C;Page>> => {
 // Lookup for a page by slug, this is where you'd use Contentful API
 const page: Page = mockedPages.filter(
 (page) => page.slug === context.params?.slug
 )[0];

 return {
 props: {
 ...page,
 },
 };
};

const Blog: NextPage&#x3C;Page> = ({ title, content }: PropsWithChildren&#x3C;Page>) => {
 return (
 &#x3C;div>
 &#x3C;Head>
 &#x3C;title>{title}&#x3C;/title>
 &#x3C;meta name="description" content={title} />
 &#x3C;link rel="icon" href="/favicon.ico" />
 &#x3C;/Head>
 &#x3C;main>{content}&#x3C;/main>
 &#x3C;footer>&#x3C;/footer>
 &#x3C;/div>
 );
};

export default Blog;
</code></pre>
Normally you would get the pages from some sort of external datastore, but for the purpose of demonstrating I just mocked it.
I don't want to focus here on the external API, otherwise this tutorial is going to be very long, but if you would like to know how I'll just recommend you to take a look at <a href="https://www.contentful.com/">Contentful</a>, which is a great Headless CMS.
Notice that there are two important functions, getStaticPaths getStaticProps.
<ul>
<li>getStaticPaths - Needs to know about the paths of the to be generated static files ahead of time (at build-time, i.e when you run <code>next build</code>)</li>
<li>getStaticProps - Fetches the props ahead of time (i.e when running <code>next build</code>), in above case it fetches a single record from the statically generated files for each slug</li>
</ul>
The default component which is exported will then receive the properties that are delivered by getStaticProps.
Let's verify that this actually works, let's run:
<pre><code class="hljs language-sh">yarn build
</code></pre>
Now you can inspect .next/server/pages folder, there are a bunch of HTML files, a part of them are the about.html, contact.html, along aside there is json format for those pages, if you open the contact.json file for example you should see:
<pre><code class="hljs language-json">{
 "pageProps": {
 "slug": "contact",
 "title": "Contact",
 "content": "Contact page"
 },
 "__N_SSG": true
}
</code></pre>
Those are the properties that NextJS internally would inject to your component when a client requests the /contact path.
You can also verify it in the browser, let's run:
<pre><code class="hljs language-sh">yarn dev
</code></pre>
And open <a href="http://localhost:3000/contact">http://localhost:3000/contact</a>, you should see the contact component outputted to the page.
Now you are probably asking yourself, but if it's just an HTML page, why do we actually need NextJS for production ? Can't we just get those HTML files at build time and pack them all into a smaller version of NGINX alpine container image for example ?
In fact, you can, and I already tried it, the container would end up being very small ~ 36MB instead of ~ 354MB, which means downloading speed and running this container is way faster and you could also deploy it to a bucket, you don't necessarily need NGINX to run this application, for that you can have a look on <a href="https://nextjs.org/docs/advanced-features/static-html-export">next export</a> command.
But of course there are downsides for this approach, because NextJS also provides further features like Incremental Static Regeneration(<a href="https://nextjs.org/docs/basic-features/data-fetching/incremental-static-regeneration">ISR</a>).
<a href="https://nextjs.org/docs/basic-features/data-fetching/incremental-static-regeneration">ISR</a> allows you to fetch the content on the fly after the clients requested the page, it will also check if the page in fact changed before making this new API call to the external datastore, this could be very useful if the content team want to change the dynamic content, you don't want to redeploy all those pages all over again, just because of a content change.
Also if you think about it, what if there are 2 million static pages, would you run a build for all of them when a single page has been changed ? absolutely no.
To add <a href="https://nextjs.org/docs/basic-features/data-fetching/incremental-static-regeneration">ISR</a> all there is to do is to add another property to the returned object from getStaticProps function, the key of that property called revalidate and the value it accepts is in seconds - how long since the last time a user visited the page the getStaticProps should be called. If the content is changing very frequently this value should be low and if not that frequently then this value should be high, depending on the use-case, for testing purposes let's add it with a high value and log "called" on the function:
<pre><code class="hljs language-tsx">...
export const getStaticProps: GetStaticProps = async (
 context: GetStaticPropsContext
): Promise&#x3C;GetStaticPropsResult&#x3C;Page>> => {
 // Lookup for a page by slug, this is where you'd use Contentful API
 const page: Page = mockedPages.filter(
 (page) => page.slug === context.params?.slug
 )[0];
 console.log("called");
 return {
 props: {
 ...page,
 },
 revalidate: 30,
 };
};
...
</code></pre>
Let's build and start this code:
<pre><code class="hljs language-sh">yarn build
yarn start
</code></pre>
To verify you can visit <a href="http://localhost:3000/contact">http://localhost:3000/contact</a> and inspect the server logs, refreshing the page, doesn't call the function. You can even refresh multiple times, you will not see the "called" log, but if you wait for 30 seconds, and refresh again, you'll notice that function indeed gets called once in order to fetch the new content.
I normally set this value to 30 seconds and add an <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag">ETag</a> verification that will be sent to the server to verify if the content indeed has been changed, to avoid passing unnecessary data over the http wire, if the content hasn't been changed the http server response should be 304 Not Modified.
Many Headless CMS also provide a webhook to listen for change events of content and you want to avoid clients receiving an outdated one, therefore you can listen for that event and send a curl request to the page that was modified, this will ensure that after 30 seconds the next client that will visit this page, getStaticProps function will get called and the client will get the up to date content.
If the use case of creating new dynamic pages from the external datastore without redeploying the application is important for you, it's also possible to enable in NextJS the fallback flag that is returned from getStaticPaths function, this will allow the calling of your render function instead of a 404 page even if the page is not statically generated ahead of time.
Then you can check if a page has been passed as an argument to the component if nothing is passed, you could show a loading component, while NextJS will generate a new static page and recall getStaticProps and fetch the new data, everything happens on the fly so the next time you call this page, that page would already exists and will be loaded way faster.
<h2>Conclusion</h2>
NextJS provides great features for <a href="https://nextjs.org/docs/basic-features/pages#static-generation-with-data">SSG</a>.
NextJS provides those features out of the box and makes it very easy to use.
Not all pages need to be dynamic, but in most cases a page could be statically generated ahead of time.
By using <a href="https://nextjs.org/docs/basic-features/data-fetching/incremental-static-regeneration">ISR</a>, it's even possible to apply this strategy to pages that require frequent data changes incrementally without generating the whole pages every time a change to the content is made.
By using external webhooks of Headless CMS like <a href="https://www.contentful.com/">Contentful</a> for example, it's possible to listen for a change event and revalidate a page content by issuing a curl request.
By using <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag">ETag</a> we can further optimize the loads on the external API's for fetching the updated content.
It's also possible to generate static pages at runtime, so we don't necessarily need to redeploy all static pages build process every time a new page is created on the backend.
Static pages are blazing fast because there is no computation needed, those pages are generated and fetched ahead of time.
It's easy to package NextJS as a container and deliver it to different environments.
If you would like to review all the changes on github: <a href="https://github.com/edenreich/examples/tree/master/nextjs-ssg">nextjs-ssg</a>

In this article I want to explain / summarize how light weight and yet powerful tools like <a href="https://man7.org/linux/man-pages/man1/strace.1.html">strace</a> and <a href="https://man7.org/linux/man-pages/man2/ptrace.2.html">ptrace</a> are.
The first time I started using those tools were when I was trying to figure out what an application is actually doing, it may also happened to you or perhaps few of the following scenarios:
<ul>
<li>You received a code that someone else wrote(probably a few years back) and now you have to maintain it and sense you didn't write it, you can't really tell what the application is actually doing by just looking at the code(because the code is written in a very complex way and the person that written this code is no longer available for questions)</li>
<li>You accessed a machine that hasn't been touched for years, and you have to troubleshot a running process to understand why this application is so slow, that process was deployed onto this machine by someone, somehow, and you don't have access to that code</li>
<li>You are using a framework, that does a bunch of things under the hood, but you can't really figure out why is it so slow</li>
</ul>
If you came across one of the following above mentioned scenarios, then strace and ptrace are probably your best bet.
Before I dive deeper on how to use these tools I think first it's important to understand what each tool basically does.
Both tools are prefixed with a letter, this letter has a meaning.
<ul>
<li>strace - the first letter stands for System-Calls, meaning this tool is for tracing what calls a particular process is sending to the linux kernel.</li>
<li>ptrace - the first letter stands for Process-Calls, meaning this tool is for tracing what calls a particular process is sending to a linux kernel. But in addition it allows a tracer(=the process which running ptrace) to mutate the traceree(=the process which is being examined).</li>
</ul>
The great thing about using strace and ptrace, is that they are language agnostic(meaning you can strace and ptrace a program written in almost any language javascript, php, rust, go etc, as long as it's a unix program and it's running, we can simply attach to it and trace it to get the low level kernel operations). Although there are some high level diagnostic tools for each specific language, that makes it a bit more structural and easier to understand for that particular language, I still think it's a good practice to get a rough overview about what the program is doing first, before diving into implementation details, and I think strace does a great job at providing that information.
First we need an example program that we can verify the results, so let's say we have a running application written in NodeJS, we'll start to reverse engineer the system calls which are issued by this program, so let's create it:
<pre><code class="hljs language-sh">mkdir demo
cd demo
</code></pre>
Let's create a simple hello world index.js file:
<pre><code class="hljs language-javascript">setInterval(() => {
 console.log('Hello World!');
}, 1000);
</code></pre>
Above code would console log out "Hello World!" every 1000 millisecond which is every second.
Ok on the first terminal let's start this simple program, and let it run for a while:
<pre><code class="hljs language-sh">node index.js
</code></pre>
Let's start to dive deeper into strace, I won't go through all options here, because there are many of them but I'll try to explain what works best for me when troubleshooting a bug.
There is a lot of information that is not needed in order to debug an issue so I'll use few filters and explain them afterward.
On another terminal session, let's execute and explore the following command:
<pre><code class="hljs language-sh">PID=`pgrep node | tail -n1`
strace -f -T -p $PID
</code></pre>
<code>pgrep node | tail -n1</code> resolves to the process id of the script we ran earlier, same as if you were running <code>ps -ef</code> and lookup for that process id of the command:
<pre><code class="hljs language-sh">/usr/bin/node index.js
</code></pre>
Note the -f flag that was added, this will ensure the tracing is done for every forked child processes(if there are any) by this main process, without this flag we're not able to see what a child processes are doing.
Note the -T flag that was added, this will provide us the time spent on a system call. You could also change it to -t which will show you the current time when this operation got executed or -r that will provide us the relative elapsed time for each system operation(with -r you normally need to look on the previous system calls to figure out slow operations, therefore I much prefer to use -T).
Now the output should be similar to the following:
<pre><code class="hljs language-sh">[pid 5882] epoll_wait(13, [], 1024, 999) = 0 &#x3C;1.000285>
[pid 5882] write(17, "Hello World!\n", 13) = 13 &#x3C;0.000453>
[pid 5882] epoll_wait(13, [], 1024, 0) = 0 &#x3C;0.000215>
</code></pre>
Your output should be similar as the above mentioned, the process id number on the left side and the time spent on the right side may be different.
It's also easy to identify on first sight the Hello World! string followed by a line-break, in addition it's possible to see that the system call <a href="https://man7.org/linux/man-pages/man2/write.2.html">write</a> (if you're familiar with C the signature the function signature is: <code>ssize_t write(int fd, const void *buf, size_t count);</code>) is issued to the linux kernel. write accepts 3 arguments, the first argument is the <a href="https://en.wikipedia.org/wiki/File_descriptor">file descriptor</a> which is reserved by the linux kernel for this particular process, the second one is our string and lastly the third argument is the amount of bytes this function writes, i.e 13 chars = 13 bytes and finally it returns how many bytes have been written(i.e = 13 section).
Ok, so we can identify the write operation, but what is this epoll_wait all about ?
Well, <a href="https://man7.org/linux/man-pages/man2/epoll_wait.2.html">epoll_wait</a> (in it's full function signature in C: <code>int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);</code>) is the system call that was produced by <code>setInterval(function, ms);</code> which we defined on index.js, It's waiting for an event on that epoll file descriptor or until this is timing out in milliseconds(i.e in the above mentioned output the third argument 999 milliseconds). If there was an error it returns -1 or 0 if no file descriptor became ready during the requested timeout in milliseconds.
Alright so it's possible to understand what's an high level written program like NodeJS is doing on the operation system to the kernel without even knowing about the content of index.js !?
All we needed is strace and the pid of that application. Of course, in a real case scenario a program contains more details and it maybe more verbose than just an "Hello World" that is outputted to the console. I just wanted to share with you the process of debugging.
If I don't know something about this particular low level function I always read the man pages (i.e <code>man strace</code> or <code>man syscalls</code> will open the manual with all the necessary information you need to know about this particular function).
Ok, that's covers strace usage "in a nutshell", ptrace let you do the same things, the only different is that ptrace a dependency of strace and many other debuggers out there, actually if not most of the debuggers exiting out there (i.e gdb, dbx etc.), there is also a tool called ltrace which is another layer of debugging, it let you intercept calls made by the application to C libraries, but I won't go over these in this post, because you can get almost all information necessary to troubleshoot a bug by using just strace.
<h2>Bonus Tip</h2>
Another useful thing is at first to get a brief overview of all system calls and their counts sent to the kernel by using flag -c
<pre><code class="hljs language-sh">strace -c -f -p $PID
</code></pre>
Run <code>Ctrl+C</code> to interrupt this command after a few seconds and you should see a similar output:
<pre><code class="hljs language-sh">% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
 63.12 0.000936 52 18 epoll_wait
 36.88 0.000547 60 9 write
------ ----------- ----------- --------- --------- ----------------
100.00 0.001483 27 total
</code></pre>
Alright, all of these calls are actually non-failing commands, but what if we want to only be focused on the errors that this program may produce, it could be useful isn't it ?
By adding -Z flag we can find all failing system calls let's try it:
<pre><code class="hljs language-sh">strace -Z -f -p $PID
</code></pre>
Of course, there is nothing outputted because there is no error in the script. Let's modify the index.js and produce an error:
<pre><code class="hljs language-javascript">const fs = require('fs');

setInterval(() => {
 console.log('Hello World!');
 try {
 fs.readFileSync('non-existing-file.txt');
 } catch {}
}, 1000);
</code></pre>
In the above example I imported file system module and tried to read a file that does not exists on the disk. By putting a try catch block around this operation I simply ignore the exception and I do not provide any information when it fails.
Well this code is obviously badly written, without error feedback or anything that let you know when something went wrong, but on legacy systems such things may still exists, and it's perfect for this example. It will produce a system call error, let's verify it using the -Z flag I mentioned earlier:
<pre><code class="hljs language-sh">strace -Z -f -p $PID
</code></pre>
Now we can see the suppressed error by the program, but it is still visible for us, you should see a similar output:
<pre><code class="hljs language-sh">[pid 11620] openat(AT_FDCWD, "non-existing-file.txt", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
</code></pre>
As you can see <a href="https://linux.die.net/man/2/openat">openat</a> kernel function was called and it was failing to read a file, because that file does not exist.
A program may do a lot of things, a program can call an external API by issuing an HTTP request over the network, a program may write to the disk or read out from it (i.e I/O), a program may allocate a bunch of things into the memory and perhaps use CPU intensively. All of those are traceable with this simple util.
There are many other filters you may apply to an strace, for this you can use the -e flag followed with a value like trace=network, here is a list of filter we can use:
<ul>
<li>-e trace=ipc – Track communication between processes (IPC)</li>
<li>-e trace=memory – Track memory syscalls (like brk, mmap, mmap2 etc)</li>
<li>-e trace=network – Track network syscalls (like sock, accept, accept4 etc)</li>
<li>-e trace=process – Track process calls (like fork, exec etc)</li>
<li>-e trace=signal – Track process signal handling (like HUP, exit)</li>
<li>-e trace=file – Track file related syscalls</li>
</ul>
This will allow us to filter out the output to find relevant information about that particular application. For example if we know the issue relay on slow interaction of that application with external API's then we might look for network calls and measure the time it took for each network call to execute by adding flag -T:
<pre><code class="hljs language-sh">strace -T -e trace=network -f -p $PID
</code></pre>
<h2>Conclusions</h2>
We learned how strace can help for inspecting what's going on a running process. A process that you may or may not have a source code for it but you still need somehow to figure out what is failing and how to fix it. Those tools are very easy to install and does not require any complicated setup, they are there sense 1992! and they are language agnostic (with one exception: if it's for debugging high level programs). Depending on the language that the process is running the system calls may be different, but in most cases they are the same regardless of what language is being used, I've chosen NodeJS because I thought it would be easier to understand for the most of us, but feel free to try it with other dynamically typed languages, you'll see that you get similar results.
If you liked this post and you still have some questions, feel free to throw your questions at me and I'll make sure to cover them on another post.

<h2>What is DAPR ?</h2>
DAPR stands for Distributed Application Runtime, it's a <a href="https://www.cncf.io/">Cloud Native Computing Foundation(CNCF)</a> supported project that is well integrated into the Kubernetes space and aims to solve the sprawl of different tools for service to service communication by applying a unified solution, that is very declarative and flexible.
<h2>Why DAPR ?</h2>
Some companies often end up with a sprawl of tools, some of them would use Kafka, some would use RabbitMQ, some would use Cloud Services like AWS Kinesis, SQS or Google PubSub etc.
The end goal of all of the above mentioned tools is usually to serve one purpose, being the broker for the messages which will be exchanged from service A to N number of services when using the publish/subscribe model(also known as pub/sub), and yet DAPR also can get more verbose than just a pub/sub model, there are tons of available configurations.
In this article I'm going to focus on the pub/sub model.
<h2>Installation</h2>
To install DAPR in your cluster on its own namespace, run(beforehand make sure you have a ready cluster, feel free to follow the <a href="https://github.com/edenreich/examples/tree/master/distributed-application-runtime">example</a>):
<pre><code class="hljs language-sh">helm repo add dapr https://dapr.github.io/helm-charts
helm repo update
helm install \
 dapr dapr/dapr \
 --namespace dapr-system \
 --create-namespace \
 --wait
kubectl --namespace dapr-system get all
</code></pre>
Then you need to configure a message broker, it could be anything you would like, just to name a few: Kafka, RabbitMQ, redis, Amazon SQS, Amazon Kinesis, Amazon SQS, Google Cloud PubSub. I'll choose to use redis:
<pre><code class="hljs language-sh">helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install \
 redis bitnami/redis \
 --namespace development \
 --create-namespace \
 --wait
</code></pre>
After DAPR is deployed then we can proceed and create components with the provided <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">Custom Resource Definitions</a>.
<h2>DAPR Use-cases ?</h2>
I'm going to focus in this article on the pub/sub model and I'll give some real world use-cases.
Let's consider the following architecture:
<img src="/img/posts/dapr_architecture.png" alt="Typical Ecommerce Architecture" title="Typical Ecommerce Architecture">
Code for this example is provided in the following link: <a href="https://github.com/edenreich/examples/tree/master/distributed-application-runtime">https://github.com/edenreich/examples/tree/master/distributed-application-runtime</a>
As you can see in the diagram above there are 4 different micro-services: order, billing, notification and shipping.
Now that we have DAPR installed in our cluster we can create the components to tell DAPR where is our message broker and how to connect to it:
<pre><code class="hljs language-yaml">---
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
 name: pubsub
spec:
 type: pubsub.redis
 version: v1
 metadata:
 - name: redisHost
 value: redis-master.development.svc.cluster.local:6379
 - name: redisPassword
 secretKeyRef:
 name: redis
 key: redis-password
auth:
 secretStore: kubernetes
</code></pre>
Now that DAPR can connect to our message broker, we can deploy the manifests that would tell DAPR what are the topics that the services are going to subscribe to, we're going to create 2 different subscriptions:
<pre><code class="hljs language-yaml">---
apiVersion: dapr.io/v1alpha1
kind: Subscription
metadata:
 name: order-subscription
spec:
 topic: orders
 route: /orders
 pubsubname: pubsub
scopes:
 - notification
 - billing
 - shipping
---
apiVersion: dapr.io/v1alpha1
kind: Subscription
metadata:
 name: billing-subscription
spec:
 topic: bills
 route: /bills
 pubsubname: pubsub
scopes:
 - order
</code></pre>
The billing-subscription / bills topic has only one subscriber which is order service and order events are going to be published to all different subscribers specified in the scope(i.e notification service, billing service and shipping service), they will receive the message on the expected endpoint /orders.
When a service declare the endpoint /orders the message would be received as a post request into the web-hook with the event payload, the consumer can then process the event. If returned 200 or 204 status code the message would be acknowledged, otherwise if returned non-ok status code (i.e above 400) the message would be redelivered, DAPR ensures delivery, this is nothing new to all other message brokers and that's how they normally work.
Last thing that is left to do, is to tell the different deployments that we need a Kubernetes <a href="https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns/#example-1-sidecar-containers">sidecar container</a> in our pods. Each sidecar container would be identified with a unique id. Also we need to tell it on what port the application is listening for connections. Those configurations could be deployed using Kubernetes annotations on the deployment manifest of each service. for billing service for example you would find the following annotations:
<pre><code class="hljs language-yaml">---
annotations:
 dapr.io/enabled: 'true'
 dapr.io/app-id: 'billing'
 dapr.io/app-port: '8080'
</code></pre>
I'm also using a configmap to declare some environment variables to tell the application which topics will be used for publishing the messages, the application then knows to split it by comma and publish the events to all of the topics specified on that comma separated list.
That's all we need, now let's deploy the applications, of course you don't have to write all of this by yourself, you can use the example and follow the <a href="https://github.com/edenreich/examples/tree/master/distributed-application-runtime">README.md</a> guide, on this blog post I want to just focus on explaining what's happening.
Assuming you followed the example and you deployed the application, every time you create an order by sending a post request to the order service(for simplicity the order service, is just a dummy service that also generates a dummy order when you call it so you don't have to pass it any payload), the other services would be notified and process that order, you could use the make load command which will send a post request to the order service and simulate orders incoming.
Notice that all other services don't expose anything to the public, they are private, they just receive the event message and process the requests asynchronously.
If the system happens to go down, bringing back the services online would continue to process from where it stopped.
You could also automatically scale up the system by application <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">HPA</a> to the subscribing deployments, <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">HPA</a> is built-into Kubernetes core(I'm not going to go over how to scale up a deployment automatically but you can follow the Kubernetes docs, it's pretty straight forward).
If you decide one day to switch from Redis broker to another message broker like Kafka for example you can just replace the deployed DAPR component with Kafka for example for that specific environment without changing anything on the application code, all the application is aware of is that there is a sidecar service listening on the default port of DAPR which is 3500 and that's where the application is sending the payload, that would then be published to the corresponding subscribers by the proxy.
Notice that I'm also using CloudEvents specification, to make this code also compatible with Cloud functions, if you want for example you can publish those events to an AWS Lambda or Google cloud functions, and they will receive that event with the correct spec for further processing.
But this is not it yet, DAPR true power is not ended yet, now let's say we want to also add tracing to know how long it takes for each service to process a message so we can find out if there are any bottlenecks.
To achieve that all we need to do is deploy <a href="https://zipkin.io/">Zipkin</a>:
<pre><code class="hljs language-sh">kubectl create namespace tracing
kubectl --namespace tracing create deployment zipkin --image=openzipkin/zipkin
kubectl --namespace tracing expose deployment zipkin --type ClusterIP --port 9411
kubectl --namespace tracing rollout status deployment zipkin
</code></pre>
And configure DAPR to tell it where the Zipkin service is running, we'll use the internal Zipkin service we've just deployed, but you can always change that to another host if you need <a href="https://en.wikipedia.org/wiki/High_availability">HA</a> etc:
<pre><code class="hljs language-yaml">---
apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
 name: tracing
spec:
 tracing:
 samplingRate: '1'
 zipkin:
 endpointAddress: 'http://zipkin.tracing.svc.cluster.local:9411/api/v2/spans'
</code></pre>
Lastly, we just need to add annotation to each deployment that we want to use this configuration:
<pre><code class="hljs language-yaml">...
 annotations:
 dapr.io/config: "tracing"
 ...
...
</code></pre>
Apply the new configurations to the deployment and the deployment is going to send tracing information to dapr without us needing to touch any application code. Behind the scenes the DAPR proxy sidecar is going to collect kernel system-calls and send them to zipkin, with the application id so we know by using the UI of zipkin what service has a bottleneck.
You can use zipkin Dashboard to inspect the different issues the application might have by port-forwarding the service to your local machine:
<pre><code class="hljs language-sh">kubectl --namespace tracing port-forward svc/zipkin 9411:9411
</code></pre>
And open it in your browser: <a href="http://localhost:9411">http://localhost:9411</a>
<h2>Conclusion</h2>
DAPR is incredibly easy to configure and brings a lot of benefits in the long run, the application doesn't need to know where we publish the message, DAPR takes care of the underlying technology that is used for sending the messages to a given broker.
The broker that is used could be easily swapped, without any complex configurations or migrations.
DAPR is a <a href="https://www.cncf.io/">CNCF</a> supported project, so you can gain some confidence about how mature this tool is, it's actually used by microsoft and some other companies in production.
By using DAPR you're ensuring that the infrastructure remains vendor neutral - today you can use Kafka Cloud, tomorrow you can switch to AWS kinesis streams, after tomorrow you can switch to your own solution running it using a cluster of Redis in memory-store. It avoids the <a href="https://en.wikipedia.org/wiki/Vendor_lock-in">vendor lock-in</a> issue.
DAPR allows software developers to focus on the application side and abstract away the complex infrastructure so it's easier for the operation / infrastructure team to fulfill business requirements without interrupting the software developers workflow.
I think that DAPR finally brings a unified solution in the Cloud Native space for the different message brokers available, which IMO is a great step to abstract complexity and make it easier to scale and maintain complex service oriented architectures.
I kept the example project very simple without any SDK or client library so it's easier to understand what happens, if you want you can also use the <a href="https://docs.dapr.io/developing-applications/sdks/">SDK</a> provided by DAPR for any type of programming language.
Word of caution - when you wish to go with DAPR to production don't use the example code provided in this article, and if you decide to go with it anyways to production then please make sure you take the necessary steps to make it fit(i.e the message broker probably deserve its own managed solution somewhere else outside of the cluster, separated from the rest of the services).
Please consider the example code as an example only.
Hope you enjoyed reading this article.

In this article, we'll break down the concept of quantization and show how it makes AI models, like Large Language Models (LLMs), smaller, faster, and more efficient, all through a simple spice rack analogy.
Imagine you have a magical kitchen with thousands of spices. With these spices, you can cook anything! But here’s the thing: carrying all those spices around every time you cook is exhausting, and you don’t even use most of them in one recipe. What if we could keep just the spices we really need for each dish? That’s where the magic of quantization comes in!
Quantization in the world of big computer brains — also called Large Language Models (LLMs) — is a little like organizing your kitchen spices. Just as grouping your most-used spices makes cooking easier, reducing the complexity of numbers in LLMs helps them use less memory and work faster. Instead of keeping every single spice on your counter, you store just the ones you need for most dishes within easy reach. The model gets smaller, faster, and easier to use. Let’s break it down step by step.
<h2>Full Precision</h2>
When LLMs are built, they think with super precise numbers called “floating points.” These numbers have lots of decimal places, like 3.14159265359, which help the model be super accurate and detailed. It’s like having thousands of spices, where every spice has a slightly different flavor.
These precise numbers allow LLMs to handle complex calculations but require a lot of memory and computing power, making them less practical for everyday use.
But do we always need all those super tiny details? Not really! Sometimes, we can still cook amazing meals with fewer spices.
<h2>The Spice Rack: Why Quantization Helps</h2>
Imagine you’re moving to a new kitchen. You don’t want to bring your whole rack of spices because it’s too big and heavy. So, you pick just 16 essential spices that you use the most.
Quantization involves reducing the precision of the numbers used by the model, such as switching from 32-bit floating-point numbers to 8-bit integers. This works because many tasks don't need super precise calculations to give good results, allowing the model to remain effective while using fewer resources.. This makes the model:
<ul>
<li>
Smaller: It needs less memory (like your lighter spice rack).
</li>
<li>
Faster: It works quicker because it’s not trying to manage all those extra details.
</li>
<li>
More portable: It can run on smaller devices, like your phone or a laptop, instead of needing a supercomputer.
</li>
</ul>
<h2>But Wait, Are There Trade-Offs?</h2>
Yes! Just like taking fewer spices means you might not capture every subtle flavor in a recipe, quantization has its trade-offs too. Here’s what happens:
<ol>
<li>
Less Detail: With fewer spices, you might miss some tiny nuances in your dish. Similarly, the model might lose a little bit of accuracy or nuance in its answers.
</li>
<li>
Occasional Mistakes: Sometimes, the model might get confused because it’s using simpler numbers to think. Imagine trying to bake a layered cake but realizing you only have one kind of sugar when the recipe calls for three types.
</li>
<li>
Balancing Act: Engineers have to find the right balance between making the model small and keeping it smart. If they simplify too much, the model might not work as well.
</li>
</ol>
<h2>Why Do We Do It?</h2>
Even with some trade-offs, quantization is super helpful. Here’s why:
<ul>
<li>
Faster and Cheaper: Quantized models can run on smaller, cheaper computers, making them more accessible for everyone.
</li>
<li>
Energy Efficient: Smaller models use less power, which is better for the environment.
</li>
<li>
Scalability: Quantized models can be used in more places, like phones, tablets, and even tiny robots!
</li>
</ul>
<h2>Conclusion</h2>
Quantization is like organizing your spice rack to keep just the right spices handy without cluttering your kitchen. This also improves accessibility, as it allows more people and devices to use powerful AI models efficiently. For LLMs, it means simplifying how they think so they can work faster and fit into smaller spaces while still being super helpful. Sure, they might lose a bit of detail, but most of the time, they’re still amazing at what they do.

In the dynamic world of software development, where methodologies and tools are continually evolving to improve efficiency, maintainability, and scalability, the OpenAPI Specification (OAS) stands out as a critical standard for designing and documenting RESTful APIs. Traditionally, developers have implemented APIs using code-first approaches, often embedding Swagger annotations directly into their codebases. However, there is a growing trend towards adopting a design-first methodology, where APIs are meticulously designed using plain OpenAPI Specification files prior to any coding. This article delves into the rationale behind this shift, emphasizing the myriad benefits of using plain OAS to generate both code and documentation over the traditional method of embedding Swagger annotations in the code.
Let's look on an example, let's say I've built a router in Go and I want to generate out of my code the docs, this code might looks like this:
<pre><code class="hljs language-golang">// ShowAccount godoc
// @Summary Show an account
// @Description get string by ID
// @Tags accounts
// @Accept json
// @Produce json
// @Param id path int true "Account ID"
// @Success 200 {object} model.Account
// @Failure 400 {object} httputil.HTTPError
// @Failure 404 {object} httputil.HTTPError
// @Failure 500 {object} httputil.HTTPError
// @Router /accounts/{id} [get]
func (c *Controller) ShowAccount(ctx *gin.Context) {
 id := ctx.Param("id")
 aid, err := strconv.Atoi(id)
 if err != nil {
 httputil.NewError(ctx, http.StatusBadRequest, err)
 return
 }
 account, err := model.AccountOne(aid)
 if err != nil {
 httputil.NewError(ctx, http.StatusNotFound, err)
 return
 }
 ctx.JSON(http.StatusOK, account)
}

</code></pre>
While it looks like a very flexible approach, it is absolutely not! Let me explain why:
<ul>
<li>
It assumes everyone on your team is a Go developer. This can be exclusionary and limit the participation of team members who might excel in other areas but are not proficient in Go. It effectively creates a barrier to entry for contributing to the API's design and documentation.
</li>
<li>
Often, libraries that enable you to write annotations support older versions of the OpenAPI Specification, which means your code may not benefit from the latest features introduced in OpenAPI until the library maintainers update it to include these enhancements.
</li>
<li>
It reduces drastically the possibility of parallel work (i.e frontend team / backend team). When documentation and endpoint details are embedded directly within the code, frontend developers must wait for backend updates before they can proceed with their integration tasks. This serialized workflow hinders rapid development and testing.
</li>
<li>
It increases the risk of documentation drift. Every change in the code needs a corresponding update in the annotations. If these updates are not meticulously managed, the documentation can easily become outdated, misleading, or incorrect, which defeats the purpose of having API documentation in the first place.
</li>
<li>
It complicates the codebase. Embedding API specifications within code annotations clutters the codebase, making it harder to read and maintain. This additional complexity can slow down new developers trying to understand the system and increase the likelihood of errors.
</li>
<li>
It limits the use of design-first tools and practices. By embedding the API design within the code, you lose out on many of the benefits of a design-first approach, such as the ability to use mock servers and automatic validation tools that can leverage a standalone API specification document.
</li>
</ul>
Taking these factors into account, it's clear that while annotating code directly for Swagger documentation might seem convenient at first, it becomes very quickly with significant drawbacks that can hinder a team's effectiveness and the overall quality of the API.
Instead, what you could write is a standard <a href="https://swagger.io/specification/">OpenAPI Spec</a>, that describes this endpoint:
<pre><code class="hljs language-yaml">openapi: 3.0.0
info:
 title: Example API
 version: 1.0.0
paths:
 /accounts/{id}:
 get:
 summary: Show an account
 description: get string by ID
 tags:
 - accounts
 parameters:
 - in: path
 name: id
 schema:
 type: integer
 required: true
 description: Account ID
 responses:
 '200':
 description: Successful response
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/Account'
 '400':
 description: Bad request
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/HTTPError'
 '404':
 description: Not found
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/HTTPError'
 '500':
 description: Internal server error
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/HTTPError'
components:
 schemas:
 Account:
 type: object
 properties:
 # Define the properties of Account model according to your application's requirements
 HTTPError:
 type: object
 properties:
 message:
 type: string
 code:
 type: integer
</code></pre>
Using this YAML-based approach, the document serves as the universal source of truth, accessible to any engineer within your company. This clarity ensures that team members can readily understand and interact with the API, regardless of their specific role or expertise. Additionally, this standardized document allows for the generation of codebases in various programming languages, offering flexibility rather than constraining developers to a single technology stack.
Consider this YAML document as a universal interface guides not only the API’s functionality but also facilitates the automated creation of documentation, codebases, and SDKs. The simplicity and comprehensibility of the YAML format make it an indispensable tool that supports a wide range of development activities, enhancing both productivity and collaboration across diverse teams.
<h2>Enhanced API Design Consistency and Clarity</h2>
One of the most compelling arguments for using plain OAS files is the consistency and clarity it brings to API design. With a design-first approach, the entire API contract is defined upfront in an OAS document. This centralized API specification ensures all stakeholders, including developers, product managers, and external partners, have a clear and consistent understanding of the API’s capabilities and constraints before any development begins.
Conversely, when Swagger annotations are used in the code, the API documentation and design can become fragmented and inconsistent. Annotations spread across multiple code files can lead to discrepancies and make it difficult for new team members to grasp the overall API architecture.
<h2>Streamlined Collaboration Across Teams</h2>
A design-first approach using OAS facilitates better collaboration across different teams. Non-developers, such as product managers and design teams, can easily contribute to the API’s design discussions and review processes without needing to understand the codebase. This inclusivity in the design process can enhance the API's usability and ensure it more effectively meets user needs and business objectives.
In the code-first approach, non-technical stakeholders may find it challenging to participate actively in the API development process due to the technical barrier that code presents. This can lead to miscommunications and a final product that may not fully align with business or user requirements.
<h2>Improved API Lifecycle Management</h2>
Managing the lifecycle of an API from design to deprecation is more straightforward with a design-first approach. Changes to the API are made directly in the OAS file, which can then be used to automatically generate code stubs and documentation. This method ensures that documentation is always up-to-date and aligned with the actual API implementation.
Using Swagger annotations, changes in the API require manual updates to both the code and the associated documentation. This process is error-prone and can lead to outdated or incorrect documentation if not meticulously maintained.
<h2>Faster Iteration and Prototyping</h2>
Developing APIs from an OAS document allows for rapid prototyping. Tools that parse OAS can generate mock servers and API stubs in various programming languages, enabling frontend teams to start working with the API even before backend implementation begins. This parallel development process can significantly reduce the time to market for new features.
Conversely, the code-first approach with Swagger requires the backend API to be at least partially implemented before any real interaction can occur, slowing down the initial phases of development.
<h2>Reduction in Code Clutter and Complexity</h2>
Maintaining a codebase with extensive Swagger annotations can lead to clutter and increased complexity. As APIs evolve, the volume of annotations can become cumbersome, making the code harder to read and maintain.
By keeping the API specification in a separate OAS file, the codebase remains clean and focused solely on functionality, reducing complexity and enhancing maintainability.
<h2>Extensibility</h2>
Furthermore, should you wish to enhance the generator in the future to produce outputs beyond just documentation and codebases, you can achieve this by incorporating custom attributes. This is done in a declarative YAML format using the <code>x-</code> prefix, allowing for flexible and scalable extensions tailored to your evolving needs. You don't need to depend on the features of the annotation library.
<h2>Conclusion</h2>
While the code-first approach using Swagger annotations has been popular, the shift towards a design-first methodology using plain OAS files offers substantial benefits. These include improved design clarity, enhanced collaboration, easier API lifecycle management, faster iteration cycles, and cleaner codebases. As the industry continues to recognize these advantages, the move towards designing APIs with OAS before coding is likely to become a standard practice, fostering more efficient and effective API development processes.

Infrastructure as Data (IaD) isn't a new concept, it just gained a bit more popularity and attention since Kubernetes and Serverless started to appear. I would assume that you're already familiar with the concept of <a href="https://en.wikipedia.org/wiki/Infrastructure_as_code">Infrastructure as Code (IaC)</a>, so before I continue any further about IaD, I want first to explain the sequence of how those concepts started to evolve.
IaC first gained it's major attention when <a href="https://en.wikipedia.org/wiki/HashiCorp">HashiCorp</a> released <a href="https://en.wikipedia.org/wiki/Terraform_(software)">Terraform</a> in 2014 (just one year before Kubernetes officially started its journey in the open-source community).
People were starting to declare their infrastructure writing code with HashiCorp Configuration Language (HCL). You would just write a small amount of .tf files and your infrastructure on any Cloud Provider out there would be provisioned, that was a major change in the way we think about managing infrastructure. The nice thing about Terraform is that it was also keeps the state, and compares it always with your current infrastructure when you want to add another VM instance.
People saw its potential and quickly started to write modules for their infrastructure based on HCL. Also imagine you had to maintain two different Cloud Providers, Terraform quickly became the most popular tool for managing a Multi-Cloud environment. It was and maybe still is a great tool for managing your infrastructure, but this article isn't just about IaC or Terraform (there are already many articles over the internet covering this topic), in this article I want to focus on another concept that recently got its attention called IaD.
So what's the difference between IaD and IaC approach ?
Well on a high-level the difference is simple - IaD uses popular data formats like JSON or YAML and IaC on the other hand uses code to declare the infrastructure, this code may also contain other bits of logic (i.e loops, ternary operators etc).
So why I would personally prefer to use IaD approach over IaC, in a nutshell:
<ul>
<li>YAML configuration files do not contain any custom logic like ternary operator, loops etc. Results are easier to predict</li>
<li>YAML configuration files are more portable</li>
<li>Using YAML format for declarations in my perspective is easy to write and to read</li>
<li>When using Kubernetes I can leverage the reconciling loop and ensure my infrastructure remains in-sync when manual edits are made to resources on the Cloud Provider via the UI console</li>
<li>IaC, specifically Terraform (which is considered IMO the best tool for IaC) is not fully GitOps conform by design</li>
</ul>
Now some of you may argue, you can achieve the same using Terraform, and although it might be true, because there are some workarounds that allow Terraform to have drift detection, but it's not there by design, those are just workarounds.
Let's say you follow all the best practices of terraforming your infrastructure, you store the state file on a backend Bucket Cloud Storage (so the changes remain consistent). But then comes the scenario, that one of your co-workers decides to log into the Cloud Provider's UI Console and starts to edit some of the resources you managed with Terraform.
In these scenarios, you can run <code>terraform refresh</code>, to refresh the current state file with the real-world state, but what if you forget to run this ? or what if you actually don't want to fetch the latest state and you'd rather revert those changes that were made manually by your colleague, yes you can run <code>terraform refresh</code>, afterwards <code>terraform plan</code> and <code>terraform apply -auto-approve</code>, but IMO it becomes very tedious. Terraform doesn't track those scenarios, and you can have drifted state for a longer period of time until you actually finds out that something is broken, unless you run your Terraform commands on a scheduled basis, there is no way for you to know when something has been manually changed on the infrastructure and that your infrastructure is basically out of sync with what Terraform has in its state file.
Some companies would actually revoke all write / edit permissions of users who log into the UI console, then you at least can avoid people from editing things they should not edit, since those resources are already managed by Terraform, but IMO that's not a viable solution.
Considering the above mentioned workarounds and not having this reconciling feature by design makes Terraform fragile and this could be frustrating.
Meet <a href="https://crossplane.io/">Crossplane</a>, it's a Kubernetes Add-On that provides a set of <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">CRD's (Custom Resource Definitions)</a> for almost every Cloud Provider.
Installation is fairly simple, prerequisites are that you only need to have a Kubernetes cluster for hosting those YAML files.
If you want for example to create simple Google Kubernetes Engine (GKE) cluster on Google Cloud Platform (GCP) the YAML definition would look like:
<pre><code class="hljs language-yaml">---
# API Reference: https://doc.crds.dev/github.com/crossplane/provider-gcp/container.gcp.crossplane.io/GKECluster/v1beta1@v0.16.0
apiVersion: container.gcp.crossplane.io/v1beta1
kind: GKECluster
metadata:
 name: my-gke-cluster
spec:
 forProvider:
 initialClusterVersion: '1.19'
 network: 'projects/&#x3C;project_id>/global/networks/opsnet'
 subnetwork: 'projects/&#x3C;project_id>/regions/us-central1/subnetworks/opsnet'
 ipAllocationPolicy:
 useIpAliases: true
 defaultMaxPodsConstraint:
 maxPodsPerNode: 110
 addonsConfig:
 cloudRunConfig:
 disabled: true
 dnsCacheConfig:
 enabled: false
 gcePersistentDiskCsiDriverConfig:
 enabled: true
 horizontalPodAutoscaling:
 disabled: true
 httpLoadBalancing:
 disabled: false
 istioConfig:
 disabled: true
 auth: 'AUTH_NONE'
 kalmConfig:
 enabled: false
 kubernetesDashboard:
 disabled: true
 networkPolicyConfig:
 disabled: false
 location: us-central1-a
 binaryAuthorization:
 enabled: false
 legacyAbac:
 enabled: false
 loggingService: 'none'
 masterAuth:
 clientCertificateConfig:
 issueClientCertificate: false
 monitoringService: 'none'
---
# API Reference: https://doc.crds.dev/github.com/crossplane/provider-gcp/container.gcp.crossplane.io/NodePool/v1alpha1@v0.16.0
apiVersion: container.gcp.crossplane.io/v1alpha1
kind: NodePool
metadata:
 name: my-np
spec:
 forProvider:
 autoscaling:
 autoprovisioned: false
 enabled: true
 maxNodeCount: 2
 minNodeCount: 1
 clusterRef:
 name: my-gke-cluster
 config:
 diskSizeGb: 100
 imageType: cos_containerd
 labels:
 test-label: crossplane-created
 machineType: n1-standard-4
 oauthScopes:
 - 'https://www.googleapis.com/auth/devstorage.read_only'
 - 'https://www.googleapis.com/auth/logging.write'
 - 'https://www.googleapis.com/auth/monitoring'
 - 'https://www.googleapis.com/auth/servicecontrol'
 - 'https://www.googleapis.com/auth/service.management.readonly'
 - 'https://www.googleapis.com/auth/trace.append'
 initialNodeCount: 2
 locations:
 - us-central1-a
 management:
 autoRepair: true
 autoUpgrade: true
</code></pre>
Notice how clean that is, it's almost identical to the configurations when describing the cluster using GCP client, <code>gcloud container clusters describe my-gke-cluster --zone us-central1-a</code>. Both the Cloud Provider (which is Google) and Crossplane are talking the same language.
That's the beauty about it IMO, there is no need to write any custom code, written in different languages (i.e HCL), instead just define a YAML file and deploy it, that's it (IDE's also provide yaml-language-servers with schema for auto-completion to know what available options are there, and if not it's always possible to refer to the <a href="https://crossplane.io/docs/v1.5/api-docs/overview.html">documentation</a>).
Although it brings us closer to <a href="https://www.gitops.tech/">GitOps workflow</a>, it's still not perfect by itself, because in reality you don't want to run <code>kubectl apply -f &#x3C;Above mentioned yaml files></code> as it defeats the purpose of maintaining everything using Git Versioning. Additionally you can leverage another tool called <a href="https://argo-cd.readthedocs.io/en/stable/">ArgoCD</a>, that will be configured to watch the git repository for changes, once you push those YAML files, ArgoCD will detect the changes and deploy it to the configured Crossplane cluster. If you remove this above mentioned file from the git repository, ArgoCD will detect that and delete my-gke-cluster mentioned above.
If one decides to log into GCP and modify the resources manually, i.e add another worker node to the cluster, this worker node would be deleted because the only source of truth is the YAML file definition that is stored in a git remote repository and is being tracked by Kubernetes API reconciling loop.
The following diagram visualize the way this workflow works:
<img src="/img/posts/infrastructure_as_data.png" alt="Infrastructure as data" title="Infrastructure as data">
You could potentially also separate ArgoCD and Crossplane with two different clusters if you happen to need them to be <a href="https://en.wikipedia.org/wiki/High_availability">Highly Available (HA)</a> for your configuration and avoiding <a href="https://en.wikipedia.org/wiki/Single_point_of_failure">single point of failure (SPOF)</a>, although since both domain logics handle configurations, and for simplicity purposes I've decided to have them in one single cluster, I think it's perfectly fine, depending on your specific use-case.
<h2>Conclusion</h2>
Although I really like this KNative Crossplane tool, this article isn't just about that, I mentioned this tool because I think it's a great example for an IaD and it's resembles how the perfect GitOps workflow should be.
Although I used Terraform for many years and I really liked it for most use-cases, I want to mention that Crossplane isn't just yet a full replacement of Terraform (considering the amount of modules developed for Terraform etc), but it's getting there and it needs support from the open source community. If something is missing on their documentation or API you can easily support their project on <a href="https://github.com/crossplane/crossplane">github</a> by sending them a pull request.
Personally I like the idea of having one to one configurations comparison, where with IaC I have to compare Code with YAML configurations provided by the different cloud providers, not ideal.
Using Terraform to manage your infrastructure requires less setup than Crossplane (i.e for Terraform you don't need to have it running inside of a Kubernetes cluster), but honestly creating a Kubernetes cluster isn't such a difficult task to do with today's tools, so if you look on the overall advantages and security policies you get out of the box, I think it's definitely worth it.

As we all know there are few tools out there to use the Kubernetes for local development, just to name a few - minikube, draft etc, however those tools are mostly consuming a lot of time to create, due to the fact they are creating nodes in a hardware based virtualization rather than process based virtualizations. The main advantage of using a process based virtualizations is the speed of the creation for those instances.
<img src="/img/posts/kind.png" alt="Kubernetes in Docker" title="KinD">
In this short tutorial we're going to create 3 node cluster with a Metal LB deployed so we can test our services locally. duration of creating a cluster on the cloud is normally 5 to 10 min, whereas for us creating it locally should take at most 3 min, this is especially useful when you quickly want to test stuff.
For simplicity purposes this tutorial is going to assume you're running these commands on a linux OS.
<h2><a href="#prerequisites">Prerequisites</a></h2>
To start we need to have docker installed:
<pre><code class="hljs language-shell">curl -sSL https://get.docker.com | sh
</code></pre>
We'll also need to download KinD binary and put it in our system path:
<pre><code class="hljs language-bash">curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.9.0/kind-linux-amd64
chmod +x ./kind
mv ./kind /usr/local/bin/kind
</code></pre>
Now that we have both required components installed, we need to create simple directory for our project or change directory to our existing project:
<pre><code class="hljs language-sh">mkdir kind-tutorial
cd kind-tutorial
</code></pre>
<h2><a href="#create-the-cluster">Create the cluster</a></h2>
In our project we'll create the manifest kubernetes-local-cluster.yaml that tells KinD how many nodes and of what type we need:
<pre><code class="hljs language-sh">kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
- role: worker
</code></pre>
Everything is now ready for creation, so let's create the cluster by running:
<pre><code class="hljs language-sh">kind create cluster --name local-cluster --config kubernetes-local-cluster.yaml
</code></pre>
After approx. 3 min the cluster should be ready and you should be able to list the nodes (for this you'll of course need the Kubernetes client kubectl, so ensure you've it downloaded and available in your path):
<pre><code class="hljs language-sh">kubectl cluster-info
kubectl get nodes
</code></pre>
By running the following command you should see a similar output as the following:
<pre><code class="hljs language-sh">Kubernetes master is running at https://127.0.0.1:42487
KubeDNS is running at https://127.0.0.1:42487/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
</code></pre>
<pre><code class="hljs language-sh">local-cluster-control-plane Ready master 106s v1.19.1
local-cluster-worker Ready &#x3C;none> 72s v1.19.1
local-cluster-worker2 Ready &#x3C;none> 72s v1.19.1
local-cluster-worker3 Ready &#x3C;none> 72s v1.19.1
</code></pre>
Now that we've successfully created the kubernetes cluster locally, we need a way to be able to expose services. As you may know on a cloud environment of Kubernetes if you want to have a service exposed you'll set type: LoadBalancer on the service manifest and the kubernetes API will pickup the request using a k8s operator to provision a load-balancer by the cloud provider and associate it with your cluster. However on a local environment we don't have such API available and the k8s operator detect that you're using kubernetes on a bare metal and would simply leave the external ip address on pending status. We need a way to expose services to our local machine, to solve this issue we'll deploy MetalLB in our cluster, on the high level MetalLB providing us a way to load balance between the nodes (using speakers and ) targeting on-premises / bare metal servers, I won't get too deep into details because there is much more to it how this load balancer work and those are just implementation details, if you're curious you can always refer to the <a href="https://metallb.universe.tf/">documentation of Metal LB</a>.
<h2><a href="#deploy-metallb">Deploy MetalLB</a></h2>
To deploy MetalLB we can follow the docs on <a href="https://metallb.universe.tf/installation/">MetalLB website</a>:
<pre><code class="hljs language-sh">kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
</code></pre>
Output should be as follow:
<pre><code class="hljs language-sh">namespace/metallb-system created
</code></pre>
<pre><code class="hljs language-sh">podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
</code></pre>
<pre><code class="hljs language-sh">secret/memberlist created
</code></pre>
To verify that our MetalLB deployment went successfully, we can issue the following command and ensure you get a similar output:
<pre><code class="hljs language-sh">kubectl -n metallb-system get all
</code></pre>
Output should be like the following:
<pre><code class="hljs language-sh">NAME READY STATUS RESTARTS AGE
pod/controller-65db86ddc6-qp5s5 1/1 Running 0 10m
pod/speaker-66dhl 1/1 Running 0 10m
pod/speaker-c5pdx 1/1 Running 0 10m
pod/speaker-cxzp7 1/1 Running 0 10m
pod/speaker-q2ddw 1/1 Running 0 10m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 10m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 10m

NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-65db86ddc6 1 1 1 10m
</code></pre>
Now that we've MetalLB successfully deployed in our local cluster and all Metallb speakers are on a running state, we need to configure MetalLB and tell it about our IP range that we want to reserve for the purpose of exposing deployments with Load Balancers, so we'll create a metallb-config.yaml with the following content:
<pre><code class="hljs language-sh">apiVersion: v1
kind: ConfigMap
metadata:
 namespace: metallb-system
 name: config
data:
 config: |
 address-pools:
 - name: default
 protocol: layer2
 addresses:
 - 172.19.xxx.xxx-172.19.xxx.xxx
</code></pre>
To figure out what range you should specify in the metallb configurations you can run the following command to first get the running nodes ip addresses:
<pre><code class="hljs language-sh">docker inspect network kind | grep IPv4Address
</code></pre>
Output should be similar to mine if you're running docker on linux:
<pre><code class="hljs language-sh">"IPv4Address": "172.19.0.2/16",
"IPv4Address": "172.19.0.5/16",
"IPv4Address": "172.19.0.4/16",
"IPv4Address": "172.19.0.3/16",
</code></pre>
This tells us that we can use the ip range of 172.19.255.1 - 172.19.255.250, so we'll modify the metallb configurations created earlier to use that range, save it and deploy it to the cluster:
<pre><code class="hljs language-sh">kubectl create -f metallb-config.yaml
</code></pre>
If you followed this tutorial and reach this point by now, you should be able to create a deployment and expose it with an LB layer 2 in your local cluster, let's verify that it works:
<pre><code class="hljs language-sh">kubectl create deploy nginx --image nginx
</code></pre>
Now let's expose the deployment and watch for changes:
<pre><code class="hljs language-sh">kubectl expose deploy nginx --port 80 --type LoadBalancer
watch kubectl get svc
</code></pre>
Output should be like the following:
<pre><code class="hljs language-sh">NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 &#x3C;none> 443/TCP 77m
nginx LoadBalancer 10.96.214.126 172.19.255.1 80:31048/TCP 5s
</code></pre>
Alright, let's curl it:
<pre><code class="hljs language-sh">curl http://172.19.255.1
</code></pre>
And you should see now the default welcome page of your nginx deployment :) Now of course you could create multiple deployments locally that are exposed by multiple services within that specified ip range.
That's it for this tutorial, to recap the skills we've gained after reading this:
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" checked disabled> The ability to create a local cluster very quickly, without extra hardware needed</li>
<li class="task-list-item"><input type="checkbox" checked disabled> Test http services with MetalLB on a local cluster</li>
</ul>
Hope you'll find this article very useful for your local development.

The integration of <a href="https://en.wikipedia.org/wiki/Large_language_model" target="_blank" rel="noopener noreferrer">Large Language Models (LLMs)</a> into applications has become an essential part of innovative solutions. Running these models in a Kubernetes environment can bring scalability, fault tolerance, and efficiency to your AI workflows. In this blog post, we will explore how you can deploy and manage LLMs on Kubernetes.
I'm not going to fully touch the parts of scaling, but I will guide you through with this setup to have the basics, so once you do need to scale it, it will be possible.
The full manifests repo is here: <a href="https://github.com/edenreich/ollama-kubernetes" target="_blank" rel="noopener noreferrer">ollama-kubernetes</a>.
<h2>Why Run LLMs on Kubernetes?</h2>
Kubernetes provides a robust orchestration layer for managing containerized applications. For LLMs, Kubernetes offers:
<ul>
<li>Scalability: Scale your models dynamically based on demand.</li>
<li>High Availability: Ensure continuous operation even if some nodes fail.</li>
<li>Resource Efficiency: Optimize GPU and CPU usage through resource scheduling and quotas.</li>
<li>Flexibility: Deploy multiple models or services side by side.</li>
</ul>
<h2>Getting Started with Ollama-Kubernetes</h2>
The <code>ollama-kubernetes</code> repository provides a streamlined way to deploy LLMs. Let’s walk through the steps to get started.
<h3>Prerequisites</h3>
Before diving in, ensure you have the following:
<ol>
<li><a href="https://flox.dev/docs/install-flox/" target="_blank" rel="noopener noreferrer">flox</a>: Used for the virtual environment, we'll deploy everything locally.</li>
<li><a href="https://docs.docker.com/desktop/setup/install/mac-install/" target="_blank" rel="noopener noreferrer">docker</a>: For running the local cluster.</li>
</ol>
<h3>Clone the Repository</h3>
Start by cloning the <code>ollama-kubernetes</code> repository:
<pre><code class="hljs language-bash">git clone https://github.com/edenreich/ollama-kubernetes.git
cd ollama-kubernetes
</code></pre>
<h3>Review the Manifests</h3>
I kept everything simple, by using kubernetes vanilla manifests, this also helps to understand how it structured, I didn't want to reach helm because it comes with a lot of templating and could be hard to understand.
At the beginning I reached a repository that claims to have a Kubernetes Operator with CRDs which seemed to be official from Ollama, apparently it's not, and I figured it's partially implemented so I've decided to skip it.
<ul>
<li><a href="https://github.com/edenreich/ollama-kubernetes/blob/main/ollama/phi3/01-namespace.yaml" target="_blank" rel="noopener noreferrer">01-namespace</a>: The namespace I choose to deploy all the LLMs called <code>ollama</code>.</li>
<li><a href="https://github.com/edenreich/ollama-kubernetes/blob/main/ollama/phi3/02-config.yaml" target="_blank" rel="noopener noreferrer">02-config</a>: Contains the environment variables for each Ollama server.</li>
<li><a href="https://github.com/edenreich/ollama-kubernetes/blob/main/ollama/phi3/03-deployment.yaml" target="_blank" rel="noopener noreferrer">03-deployment</a>: The deployment file contains the necessary deployment specs - the model I choose to use is side-car, the side-car container will pull the model on startup and serve it. So each LLM can be completely decoupled by pod when deployed. I left a comment for the nvidia GPUs which you can enable if you bring the workload to a production cluster that has GPUs</li>
<li><a href="https://github.com/edenreich/ollama-kubernetes/blob/main/ollama/phi3/04-service.yaml" target="_blank" rel="noopener noreferrer">04-service</a>: I'm defining the service port, from 8080 to the container port 11434.</li>
</ul>
<h3>Create the Cluster</h3>
When running it just locally to experiment with it. Just run:
<pre><code class="hljs language-sh">flox activate
task cluster-create
</code></pre>
You should have a running 3 node cluster.
<h3>Deploy to Kubernetes</h3>
To deploy the LLM as a service (let's start with the small model, to make sure it fits on the local hardware):
<pre><code class="hljs language-bash">kubectl apply -f ollama/phi3/
</code></pre>
It may take some time to download the LLM, so be patient, the container will be up and running once the LLM has been downloaded.
<h3>Verify the Deployment</h3>
Check the status of your pods:
<pre><code class="hljs language-sh">kubectl get pods
</code></pre>
You should see pods running the LLM service. To access the API:
<ol>
<li>Forward the service port locally:</li>
</ol>
<pre><code class="hljs language-sh">kubectl -n ollama port-forward svc/ollama 8080:8080
</code></pre>
<ol start="2">
<li>Test the API:</li>
</ol>
<pre><code class="hljs language-sh">curl http://localhost:8080/api/tags
</code></pre>
This will retrieve the LLM that was pulled earlier via the API.
<h2>GPU Support</h2>
If your cluster supports GPUs, ensure the <code>NVIDIA</code> device plugin is installed. It could be installed via helm chart on all cloud providers.
Don't forget to comment back in the <code>nvidia.com/gpu</code> resource quotas, so the pod will be automatically scheduled on the nodes that have sufficient nvidia GPUs.
<h2>OpenWeb-UI</h2>
You can interact with the model using port-forward and a bunch of curl requests from the terminal to generate responses or...
You can interact with it like I like to do, by simply using <code>OpenWeb-UI</code> which is fully open source - it's awesome.
The interaction with the API directly, only make sense if you're planning to build agents. If you'd like me to cover it, let me know.
Back to the subject,
To deploy it:
<pre><code class="hljs language-sh">kubectl apply -f openweb-ui/
</code></pre>
Verify the UI is ready:
<pre><code class="hljs language-sh">kubectl -n openweb-ui rollout status deploy openweb-ui
</code></pre>
It might take some time to go up.
To view the UI, run:
<pre><code class="hljs language-sh">kubectl -n openweb-ui port-forward svc/openweb-ui-service 8080:8080
</code></pre>
Open <a href="http://localhost:8080">http://localhost:8080</a> and create a sample account.
<h2>OpenWeb-UI Configurations</h2>
Note that there is an environment variable <code>OLLAMA_BASE_URLS</code> - a <code>;</code> separated list of the deployed LLMs services. So if you deploy another LLM ensure to add an entry so it will be discoverable by the UI.
<h2>Conclusion</h2>
Running LLMs on Kubernetes is super simple, once you get the initial setup correctly.
There are some solutions out there for Kubernetes, but they are not tailored for LLMs, so I've decided to drop them. Also for clarity you can think about it more of a guide like "Kubernetes the hard way" to understand the fundamentals without many abstractions that reduces clarity.
With this building-blocks you can take that same solution and implement an helm chart easily or perhaps use the Kubernetes operator once it's a more mature project.
Using <a href="https://github.com/open-webui/open-webui" target="_blank" rel="noopener noreferrer">OpenWeb-UI</a> can help you get a seamless experience as if you are using ChatGPT. Personally I like the features of OpenWeb-UI better, due to it's flexibility and the fact that it's fully open source.
Now that you have the fundamentals how to deploy the LLMs and interact with them with a UI, you can easily add Home-assistant, which is also fully open-source into the mix and communicate with those local LLMs using your voice, which will allow you to communicate with your home smart devices.
Running an LLM fully locally can be cost effective, not as expensive as you might think.
All the manifests and a quick guide are in <a href="https://github.com/edenreich/ollama-kubernetes">here</a>.
If you'd like me to cover deploying and managing Home-assistant, how to deploy this on Google Kubernetes Engine, let me know.
Let me know what you think or if you have any questions, by dropping a comment down below.

Kubernetes, the leading container orchestration platform, offers several mechanisms to maintain the health and efficiency of applications running in a cluster. Among these mechanisms are liveness, readiness, and startup probes. This article assumes you are already familiar with liveness and readiness probes. While liveness and readiness probes are widely discussed, startup probes are equally crucial, particularly for applications with significant initialization times. This article focuses on startup probes, explaining their purpose, configuration, common pitfalls, and common practices.
<h2>What are Startup Probes?</h2>
<h3>Purpose</h3>
Startup probes are designed to manage application startup issues by ensuring that an application is given enough time to start before liveness and readiness probes are activated. This is particularly useful for applications that require an extended initialization period.
<h3>Key Functions</h3>
<ul>
<li>
Prevent Premature Failures: Ensuring that liveness and readiness probes do not interfere with the application during its startup phase.
</li>
<li>
Handle Long Initialization: Providing sufficient time for applications with complex or lengthy startup processes to become fully operational.
</li>
</ul>
<h2>When to Use Startup Probes</h2>
<h3>Slow-Starting Applications</h3>
Applications that require significant time to initialize, such as those loading large datasets, establishing numerous connections, or performing intensive computations during startup, benefit from startup probes. Without a startup probe, liveness and readiness probes might mistakenly identify these applications as unhealthy and restart them prematurely.
<h3>Heavy Initialization Processes</h3>
Applications that perform extensive setup operations, such as warming up caches, initializing large configuration files, or connecting to numerous external services, can use startup probes to complete these tasks without disruption.
<h3>Critical Dependencies</h3>
Applications dependent on the availability of critical services or databases can use startup probes to ensure all dependencies are available and fully initialized before starting the main application processes.
<h3>Complex Configuration and Bootstrapping</h3>
For applications with complex configuration and bootstrapping sequences, startup probes allow sufficient time to complete these sequences without being prematurely killed by liveness probes.
<h2>Configuration of Startup Probes</h2>
Startup probes are configured similarly to liveness and readiness probes but typically have longer timeouts and fewer retries to accommodate the longer startup time. Here’s an example configuration:
<pre><code class="hljs language-yaml">startupProbe:
 httpGet:
 path: /healthz
 port: 8080
 initialDelaySeconds: 10
 periodSeconds: 10
 failureThreshold: 30
</code></pre>
In this configuration, Kubernetes will wait for 10 seconds before sending the first request to /healthz on port 8080. It checks every 10 seconds and allows up to 30 failures before marking the startup as failed.
<h2>Common Pitfalls</h2>
<h3>Missing Startup Probes</h3>
One common pitfall is neglecting to define startup probes for applications with long initialization times. Without a startup probe, liveness probes might incorrectly assume that the application is stuck or unresponsive, leading to repeated restarts. This can cause a cascade of failures, where the application never has enough time to fully start and stabilize.
<h3>Insufficient Initial Delays</h3>
Setting too short an initial delay for the startup probe can result in premature checks, which may fail and trigger unnecessary restarts. It’s crucial to accurately estimate and configure the initial delay based on the application's startup characteristics.
<h3>Misconfigured Failure Thresholds</h3>
Using a low failure threshold can also cause premature restarts. Ensure that the failure threshold is high enough to meet the application's worst-case startup time.
<h3>Overlapping Probes</h3>
Improper coordination between startup, liveness, and readiness probes can lead to conflicts. Ensure that the startup probe has sufficient time to complete before liveness and readiness probes take over.
<h2>Best Practices for Using Startup Probes</h2>
<ol>
<li>
Accurate Initial Delays:
Set <code>initialDelaySeconds</code> to a value that allows the application to complete its initial setup tasks before the probe checks its status.
</li>
<li>
Appropriate Check Intervals:
Configure <code>periodSeconds</code> to balance between frequent checks and allowing enough time for the application to progress in its startup.
</li>
<li>
High Failure Thresholds:
Use a high <code>failureThreshold</code> to give applications enough time to start without being prematurely marked as failed.
</li>
<li>
Continues Checks with Liveness and Readiness Probes:
Once the application is started, liveness and readiness probes take over to ensure ongoing health and readiness to handle traffic.
</li>
<li>
Monitor and Adjust:
Continuously monitor the application's startup behavior and adjust the startup probe configuration as necessary to align with any changes in the startup process.
</li>
<li>
Log and Debug:
Implement detailed logging within the application’s startup process to facilitate debugging when startup probes fail.
</li>
</ol>
<h2>Example Scenario</h2>
Consider a Java application that loads a large dataset into memory upon startup. This initialization takes approximately 3 minutes. Without a startup probe, liveness and readiness probes might kill the container, assuming it's unresponsive. By configuring a startup probe with a sufficient initial delay and failure threshold, the application gets the necessary time to load the dataset fully before handling traffic.
<pre><code class="hljs language-yaml">startupProbe:
 exec:
 command:
 - cat
 - /tmp/healthy
 initialDelaySeconds: 180
 periodSeconds: 10
 failureThreshold: 18
</code></pre>
In this configuration, the startup probe checks for the presence of a /tmp/healthy file, which the application creates after completing its initialization.
You might wonder if you could achieve the same result by setting a long initialDelaySeconds on the liveness and readiness probes. However, this approach is not ideal because it leads to duplicated values and does not provide the same flexibility and clarity as using a dedicated startup probe. Startup probes are specifically designed for this purpose, ensuring that the initialization phase is handled correctly without interfering with the ongoing health checks managed by liveness and readiness probes.
<h2>Conclusion</h2>
Startup probes are an essential in Kubernetes for managing the lifecycle of applications with long or complex startup sequences. They ensure that such applications have the necessary time to initialize fully before liveness and readiness probes begin their checks. By accurately configuring and utilizing startup probes, you can prevent premature restarts, ensuring that applications are healthy and ready to serve traffic once fully started. This enhances the resilience and reliability of your Kubernetes deployments.
However, not every application requires a startup probe. They are typically needed for slow-starting applications, so use them wisely to avoid unnecessary complexity in your configurations.

Engineering Blog

Blog Posts

Running LLMs on Kubernetes with Ollama

Quantization Explained: Making AI Efficient with a Simple Spice Rack Analogy

The Forgotten Kubernetes Probes: Understanding Startup Probes and Their Importance

Design-First approach with OpenAPI

ArgoCD Preview Environments

Loosely coupled Micro-services with Distributed Application Runtime(DAPR)

NextJS Static Site Generator(SSG)

A/B Testing with NGINX

Infrastructure as Data

Debugging running processes with strace and ptrace

Build your own Kubernetes Cluster with K3S

Kubernetes for local development with Kubernetes in Docker(KinD)