Skip to content

NLB-6740 Updated documentation #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 25 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The following is a set of guidelines for contributing to the nginx_loadbalancer_kubernetes. We really appreciate that you are considering contributing!

#### Table Of Contents
## Table Of Contents

[Getting Started](#getting-started)

Expand All @@ -24,24 +24,37 @@ We are not currently accepting contributions to this project. Please create an i

### Report a Bug

To report a bug, open an issue on GitHub with the label `bug` using the available bug report issue template. Please ensure the bug has not already been reported. **If the bug is a potential security vulnerability, please report using our security policy.**
To report a bug, open an issue on GitHub with the label `bug` using the available [bug report issue form](/.github/ISSUE_TEMPLATE/bug_report.yml). Please ensure the bug has not already been reported. **If the bug is a potential security vulnerability, please report it using our [security policy](/SECURITY.md).**

### Suggest a Feature or Enhancement

To suggest a feature or enhancement, please create an issue on GitHub with the label `feature` or `enhancement` using the available feature request issue template. Please ensure the feature or enhancement has not already been suggested.
To suggest a feature or enhancement, please create an issue on GitHub with the label `enhancement` using the available [feature request issue form](/.github/ISSUE_TEMPLATE/feature_request.yml). Please ensure the feature or enhancement has not already been suggested.

### Open a Pull Request (PR)

- Fork the repo, create a branch, implement your changes, add any relevant tests, and submit a PR when your changes are **tested** and ready for review.
- Fill in the [PR template](/.github/pull_request_template.md).

> [!NOTE]
> If you'd like to implement a new feature, please consider creating a [feature request issue](/.github/ISSUE_TEMPLATE/feature_request.yml) first to start a discussion about the feature.

#### F5 Contributor License Agreement (CLA)

F5 requires all contributors to agree to the terms of the F5 CLA (available [here](https://github.com/f5/f5-cla/blob/main/docs/f5_cla.md)) before any of their changes can be incorporated into an F5 Open Source repository (even contributions to the F5 CLA itself!).

If you have not yet agreed to the F5 CLA terms and submit a PR to this repository, a bot will prompt you to view and agree to the F5 CLA. You will have to agree to the F5 CLA terms through a comment in the PR before any of your changes can be merged. Your agreement signature will be safely stored by F5 and no longer be required in future PRs.

## Code Guidelines

<!-- ### Go/Python/Bash Guidelines (OPTIONAL) -->

### Git Guidelines

* Keep a clean, concise and meaningful git commit history on your branch (within reason), rebasing locally and squashing before submitting a PR.
* If possible and/or relevant, use the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) format when writing a commit message, so that changelogs can be automatically generated
* Follow the guidelines of writing a good commit message as described here <https://chris.beams.io/posts/git-commit/> and summarised in the next few points:
* In the subject line, use the present tense ("Add feature" not "Added feature").
* In the subject line, use the imperative mood ("Move cursor to..." not "Moves cursor to...").
* Limit the subject line to 67 characters or less.
* Limit the rest of the commit message to 72 characters or less.
* Reference issues and pull requests liberally after the subject line.
* Add more detailed description in the body of the git message (`git commit -a` to give you more space and time in your text editor to write a good message instead of `git commit -am`).
- Keep a clean, concise and meaningful git commit history on your branch (within reason), rebasing locally and squashing before submitting a PR.
- If possible and/or relevant, use the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) format when writing a commit message, so that changelogs can be automatically generated.
- Follow the guidelines of writing a good commit message as described here <https://chris.beams.io/posts/git-commit/> and summarized in the next few points:
- In the subject line, use the present tense ("Add feature" not "Added feature").
- In the subject line, use the imperative mood ("Move cursor to..." not "Moves cursor to...").
- Limit the subject line to 72 characters or less.
- Reference issues and pull requests liberally after the subject line.
- Add more detailed description in the body of the git message (`git commit -a` to give you more space and time in your text editor to write a good message instead of `git commit -am`).
31 changes: 14 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
<!-- markdownlint-disable-next-line first-line-h1 -->
[![CI](https://github.com/nginxinc/nginx-loadbalancer-kubernetes/actions/workflows/build-test.yml/badge.svg)](https://github.com/nginxinc/nginx-loadbalancer-kubernetess/actions/workflows/build-test.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/nginxinc/nginx-loadbalancer-kubernetes)](https://goreportcard.com/report/github.com/nginxinc/nginx-loadbalancer-kubernetes)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
Expand All @@ -8,16 +9,13 @@
[![Community Support](https://badgen.net/badge/support/community/cyan?icon=awesome)](https://github.com/nginxinc/nginx-loadbalancer-kubernetes/discussions)
[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)



<div style="margin-bottom: 5em;">
<span>
<img style="float: left;" src="nlk-logo.svg" width="124" />
<h2 style="padding: 1.5em">nginx-loadbalancer-kubernetes</h2>
</span>
</div>


The NGINX Loadbalancer for Kubernetes, or _NLK_, is a Kubernetes controller that provides TCP load balancing external to a Kubernetes cluster running on-premise.

## Requirements
Expand Down Expand Up @@ -70,7 +68,7 @@ NLK itself does not perform load balancing. Rather, NLK allows you to manage Ser
There are few bits of administrivia to get out of the way before you can start leveraging NLK for your load balancing needs.

As noted above, NLK is intended for when you have one or more Kubernetes clusters running on-premise. In addition to this,
you need to have at least one NGINX Plus host running outside your cluster (Please refer to the [Roadmap](#Roadmap) for information about other load balancer servers).
you need to have at least one NGINX Plus host running outside your cluster (Please refer to the [Roadmap](#roadmap) for information about other load balancer servers).

### Deployment

Expand Down Expand Up @@ -107,27 +105,27 @@ There is a much more detailed [Installation Reference](docs/README.md) available

1. Clone this repo (optional, you can simply copy the `deployments/` directory)

```git clone [email protected]:nginxinc/nginx-loadbalancer-kubernetes.git```
```git clone [email protected]:nginxinc/nginx-loadbalancer-kubernetes.git```

2. Apply the Namespace
1. Apply the Namespace

```kubectl apply -f deployments/deployment/namespace.yaml```
```kubectl apply -f deployments/deployment/namespace.yaml```

3. Apply the RBAC resources
1. Apply the RBAC resources

```./deployments/rbac/apply.sh```
```./deployments/rbac/apply.sh```

4. Update / Apply the ConfigMap (For best results update the `nginx-hosts` values first)
1. Update / Apply the ConfigMap (For best results update the `nginx-hosts` values first)

```kubectl apply -f deployments/deployment/configmap.yaml```
```kubectl apply -f deployments/deployment/configmap.yaml```

5. Apply the Deployment
1. Apply the Deployment

```kubectl apply -f deployments/deployment/deployment.yaml```
```kubectl apply -f deployments/deployment/deployment.yaml```

6. Check the logs
1. Check the logs

```kubectl -n nlk logs -f $(kubectl -n nlk get po -l "app=nlk" --no-headers -o custom-columns=":metadata.name")```
```kubectl -n nlk logs -f $(kubectl -n nlk get po -l "app=nlk" --no-headers -o custom-columns=":metadata.name")```

At this point NLK should be up and running. Now would be a great time to go over to the [Installation Reference](docs/README.md)
and follow the instructions to deploy a demo application.
Expand Down Expand Up @@ -164,11 +162,10 @@ diverse ideas will be key to NLK becoming a solution that is useful to the commu
when we are able to accept pull requests from the community.

## Authors

- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc.
- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc.

<br/>

## License

[Apache License, Version 2.0](https://github.com/nginxinc/nginx-loadbalancer-kubernetes/blob/main/LICENSE)
Expand Down
16 changes: 8 additions & 8 deletions SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,20 @@ Don't know how something in the nginx_loadbalancer_kubernetes works? Curious if

This isn't the right place to get support for NGINX specific questions, but the following resources are available below. Thanks for your understanding!

### Community Slack
### Community Forum

We have a community [Slack](https://nginxcommunity.slack.com/)!

If you are not a member click [here](https://join.slack.com/t/nginxcommunity/shared_invite/zt-1aaa22w80-~_~wSMNyPxLPLp5xunOC7w) to sign up (and let us know if the link does not seem to be working!)

Once you join, check out the `#beginner-questions` and `nginx-users` channels :)
We have a community [forum](https://community.nginx.org/)! If you have any questions and/or issues, try checking out the [`Troubleshooting`](https://community.nginx.org/c/troubleshooting/8) and [`How do I...?`](https://community.nginx.org/c/how-do-i/9) categories. Both fellow community members and NGINXers might be able to help you! :)

### Documentation

For a comprehensive list of all NGINX directives, check out <https://nginx.org>.

For a comprehensive list of admin and deployment guides for all NGINX products, check out <https://docs.nginx.com>.

### Mailing List
## Contributing

Please see the [contributing guide](/CONTRIBUTING.md) for guidelines on how to best contribute to this project.

## Community Support

Want to get in touch with the NGINX dev team directly? Try using the relevant mailing list found at <https://mailman.nginx.org/mailman3/lists/>!
This project does **not** offer commercial support. Community support is offered on a best effort basis through either GitHub issues/PRs/discussions or through any of our active communities.
2 changes: 1 addition & 1 deletion charts/nlk/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ nlk:
## trace,debug,info,warn,error,fatal,panic
logLevel: "info"

## the nginx hosts (comma-separated) to send upstream updates to
## the nginx hosts to send upstream updates to. For multiple hosts, use a sequence
nginxHosts: ""
## Sets the annotation value that NLK is looking for to watch a Service
# serviceAnnotationMatch: nginxaas
Expand Down
74 changes: 23 additions & 51 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,20 @@
# Overview

The nginx-loadbalancer-kubernetes runs in a Kubernetes Cluster and responds to changes in resources of interest, updating designated NGINX Plus hosts with the appropriate configuration.
The nginx-loadbalancer-kubernetes runs in a Kubernetes cluster and responds to changes in specified services by issuing requests to the designated NGINX Plus hosts to update the appropriate upstreams.

## Basic Architecture

The controller is deployed in a Kubernetes Cluster. Upon startup, it registers interest in changes to Service resources in the "nginx-ingress" namespace.
The Handler accepts the events raised by the Cluster and calls the Translator to convert the events into event definitions that are used to update NGINX Plus hosts.
Next, the Handler calls the Synchronizer with the list of translated events which are fanned-out for each NGINX host.
Lastly, the Synchronizer calls the [NGINX+ Configuration API](https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/) using the [NGINX Plus Go client](https://github.com/nginxinc/nginx-plus-go-client) to update the target NGINX Plus host(s).
The controller is deployed in a Kubernetes cluster. Upon startup, it registers interest in changes to services with the appropriate annotation.
The Watcher listens for events raised by the cluster and then adds these events to the Synchronizer's queue. The Synchronizer makes use of the Translator to transform the events into specific updates that will be sent to the NGINX Plus hosts.
Lastly, the Synchronizer calls the [NGINX+ Configuration API](https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/) using the [NGINX Plus Go client](https://github.com/nginxinc/nginx-plus-go-client) to update the target NGINX Plus host(s).

```mermaid
stateDiagram-v2
Controller --> Watcher
Controller --> Settings
Watcher --> Handler : "nlk-handler queue"
Handler --> Translator
Translator --> Handler
Handler --> Synchronizer : "nlk-synchronizer queue"
Watcher --> Synchronizer : "nlk-synchronizer queue"
Synchronizer --> Translator
Translator --> Synchronizer
Synchronizer --> BorderClient : "HttpBorderClient | TcpBorderClient"
BorderClient --> NGINXPlusLB1
BorderClient --> NGINXPlusLB2
Expand All @@ -26,38 +24,31 @@ stateDiagram-v2

### Settings

The Settings module is responsible for loading the configuration settings from the "nlk-config" ConfigMap resource in the "nlk" namespace.
The Settings are loaded when the controller starts and are reloaded when the "nlk-config" ConfigMap resource is updated.
The Settings module is responsible for loading the configuration settings when the controller starts. Any changes made to the settings through helm at runtime will cause nginx-loadbalancer-kubernetes to be restarted.

### Watcher

The Watcher is responsible for monitoring changes to Service resources in the "nginx-ingress" namespace.
It registers methods that handle each event type. Events are handled by creating a `core.Event` instance and adding it to the "nlk-handler" queue.
When adding the event to the queue, the Watcher also retrieves the list of Node IPs and adds the list to the event.
The master node ip is excluded from the list. (NOTE: This should be configurable.)

### Handler

The Handler is responsible for taking the `core.Event` instances from the "nlk-handler" queue and calling the Translator to convert the event into a `core.ServerUpdateEvent` instance,
adding each `core.ServerUpdateEvent` to the "nlk-synchronizer" queue.
The Watcher is responsible for monitoring changes to Service resources that are annotated by the user.
It registers methods that handle each event type. An event is created for a particular service and is added to the "nlk-synchronizer" queue.

### Translator

The Translator is responsible for converting the `core.Event` event into an `nginxClient.UpstreamServer` event.
This involves filtering out the `core.Event` instances that are not of interest to the controller by accepting only Port names starting with the NlkPrefix value (currently _nlk-_).
The event is then fanned-out based on the defined Ports, one event per defined Port. Each port is then augmented with the Ingress name (the name configured in the Port definition with the NlkPrefix value removed),
and the list of the Node's IP addresses.
The Translator is responsible for converting a `core.Event` event into an `nginxClient.UpstreamServer` event.

The Translator passes the list of events to the Synchronizer by calling the `AddEvents` method.
The Translator assembles a list of IP addresses for the Kubernetes service, routing traffic to the NodePorts, ClusterIPs or external load balancer, depending on the Kubernetes service type. If the service is of type NodePort, the Translator excludes the master node IP from the list of the addresses. (NOTE: This should be configurable.)

The Translator uses the port name to determine the protocol and the name of the user's upstream. For example, if the user has named their service port "http-tea", the translator will create an update event for the HTTP upstream named "tea".

**NOTE: It is important that the Port names match the name of the defined NGINX Plus Upstreams.**

In the following example the NGINX Plus Upstreams are named "nlk-nginx-lb-http" and "nlk-nginx-lb-https". These match the name in the NGINX Plus configuration.
In the following example the service uses the `nginx.com/nginxaas: nginxaas` annotation so that NLK knows to monitor the service. NGINX Plus Upstreams are named "ingress" and "ingress-tls". These match the upstream names specified in the NGINX configuration on the NGINX Plus host.

```yaml
apiVersion: v1
kind: Service
metadata:
annotations:
nginx.com/nginxaas: nginxaas
name: nginx-ingress
namespace: nginx-ingress
spec:
Expand All @@ -66,48 +57,29 @@ spec:
- port: 80
targetPort: 80
protocol: TCP
name: nlk-nginx-lb-http
name: http-ingress
- port: 443
targetPort: 443
protocol: TCP
name: nlk-nginx-lb-https
name: http-ingress-tls
selector:
app: nginx-ingress
```

### Synchronizer

The Synchronizer is responsible for fanning-out the given list of `core.ServerUpdateEvent` events, one for each configured NGINX Plus host.
The NGINX Plus hosts are configured using a ConfigMap resource named "nlk-config" in the "nlk" namespace. An example of the ConfigMap is shown below.
The NGINX Plus hosts are configured through the helm chart `nlk.config.nginxHosts` value. Use a string for a single host and a sequence for multiple hosts. See the comments in `charts/nlk/values.yaml` for details.

```yaml
apiVersion: v1
kind: ConfigMap
data:
nginx-hosts:
"http://10.1.1.4:9000/api,http://10.1.1.5:9000/api"
metadata:
name: nlk-config
namespace: nlk
```

This example includes two NGINX Plus hosts to support High Availability.

Additionally, the Synchronizer is responsible for taking the `core.ServerUpdateEvent` instances from the "nlk-synchronizer" queue and updating the target NGINX Plus host.
The Synchronizer uses the [NGINX Plus Go client](https://github.com/nginxinc/nginx-plus-go-client) to communicate with each NGINX Plus host.


#### Retry Mechanism

The Synchronizer uses a retry mechanism to handle failures when updating the NGINX Plus hosts.
The retry mechanism is implemented in the workqueue using the `workqueue.NewItemExponentialFailureRateLimiter`,
The Synchronizer uses a retry mechanism to handle failures when updating the NGINX Plus hosts.
The retry mechanism is implemented in the workqueue using the `workqueue.NewItemExponentialFailureRateLimiter`,
having defaults set to a base of 2 seconds, and a maximum of 60 seconds.

### Jitter

The Synchronizer uses a jitter mechanism to avoid thrashing the NGINX Plus hosts. Each `core.ServerUpdateEvent` instance
is added to the "nlk-synchronizer" queue with a random jitter value between 250 and 750 milliseconds.

## Authors

- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc.
- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc.
Loading