path: root/docs/quick_start.md

# Quick Start: load balancer

## Introduction

This quick start shows how to build a bare-bones load balancer using pingora and pingora-proxy.

The goal of the load balancer is for every incoming HTTP request, select one of the two backends: https://1.1.1.1 and https://1.0.0.1 in a round-robin fashion.

## Build a basic load balancer

Create a new cargo project for our load balancer. Let's call it `load_balancer`

```
cargo new load_balancer
```

### Include the Pingora Crate and Basic Dependencies

In your project's `cargo.toml` file add the following to your dependencies
```
async-trait="0.1"
pingora = { version = "0.1", features = [ "lb" ] }
```

### Create a pingora server
First, let's create a pingora server. A pingora `Server` is a process which can host one or many
services. The pingora `Server` takes care of configuration and CLI argument parsing, daemonization,
signal handling, and graceful restart or shutdown.

The preferred usage is to initialize the `Server` in the `main()` function and
use `run_forever()` to spawn all the runtime threads and block the main thread until the server is
ready to exit.


```rust
use async_trait::async_trait;
use pingora::prelude::*;
use std::sync::Arc;

fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();
    my_server.run_forever();
}
```

This will compile and run, but it doesn't do anything interesting.

### Create a load balancer proxy
Next let's create a load balancer. Our load balancer holds a static list of upstream IPs. The `pingora-load-balancing` crate already provides the `LoadBalancer` struct with common selection algorithms such as round robin and hashing. So let’s just use it. If the use case requires more sophisticated or customized server selection logic, users can simply implement it themselves in this function.


```rust
pub struct LB(Arc<LoadBalancer<RoundRobin>>);
```

In order to make the server a proxy, we need to implement the `ProxyHttp` trait for it.

Any object that implements the `ProxyHttp` trait essentially defines how a request is handled in
the proxy. The only required method in the `ProxyHttp` trait is `upstream_peer()` which returns
the address where the request should be proxied to.

In the body of the `upstream_peer()`, let's use the `select()` method for the `LoadBalancer` to round-robin across the upstream IPs. In this example we use HTTPS to connect to the backends, so we also need to specify to `use_tls` and set the SNI when constructing our [`Peer`](peer.md) object.

```rust
#[async_trait]
impl ProxyHttp for LB {

    /// For this small example, we don't need context storage
    type CTX = ();
    fn new_ctx(&self) -> () {
        ()
    }

    async fn upstream_peer(&self, _session: &mut Session, _ctx: &mut ()) -> Result<Box<HttpPeer>> {
        let upstream = self.0
            .select(b"", 256) // hash doesn't matter for round robin
            .unwrap();

        println!("upstream peer is: {upstream:?}");

        // Set SNI to one.one.one.one
        let peer = Box::new(HttpPeer::new(upstream, true, "one.one.one.one".to_string()));
        Ok(peer)
    }
}
```

In order for the 1.1.1.1 backends to accept our requests, a host header must be present. Adding this header
can be done by the `upstream_request_filter()` callback which modifies the request header after
the connection to the backends are established and before the request header is sent.

```rust
impl ProxyHttp for LB {
    // ...
    async fn upstream_request_filter(
        &self,
        _session: &mut Session,
        upstream_request: &mut RequestHeader,
        _ctx: &mut Self::CTX,
    ) -> Result<()> {
        upstream_request.insert_header("Host", "one.one.one.one").unwrap();
        Ok(())
    }
}
```


### Create a pingora-proxy service
Next, let's create a proxy service that follows the instructions of the load balancer above.

A pingora `Service` listens to one or multiple (TCP or Unix domain socket) endpoints. When a new connection is established
the `Service` hands the connection over to its "application." `pingora-proxy` is such an application
which proxies the HTTP request to the given backend as configured above.

In the example below, we create a `LB` instance with two backends `1.1.1.1:443` and `1.0.0.1:443`.
We put that `LB` instance to a proxy `Service` via the  `http_proxy_service()` call and then tell our
`Server` to host that proxy `Service`.

```rust
fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();

    let upstreams =
        LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443"]).unwrap();

    let mut lb = http_proxy_service(&my_server.configuration, LB(Arc::new(upstreams)));
        lb.add_tcp("0.0.0.0:6188");

    my_server.add_service(lb);

    my_server.run_forever();
}
```

### Run it

Now that we have added the load balancer to the service, we can run our new 
project with 

```cargo run```

To test it, simply send the server a few requests with the command:
```
curl 127.0.0.1:6188 -svo /dev/null
```

You can also navigate your browser to [http://localhost:6188](http://localhost:6188)

The following output shows that the load balancer is doing its job to balance across the two backends:
```
upstream peer is: Backend { addr: Inet(1.0.0.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.1.1.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.0.0.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.1.1.1:443), weight: 1 }
upstream peer is: Backend { addr: Inet(1.0.0.1:443), weight: 1 }
...
```

Well done! At this point you have a functional load balancer. It is a _very_ 
basic load balancer though, so the next section will walk you through how to
make it more robust with some built-in pingora tooling.

## Add functionality

Pingora provides several helpful features that can be enabled and configured 
with just a few lines of code. These range from simple peer health checks to 
the ability to seamlessly update running binary with zero service interruptions.

### Peer health checks

To make our load balancer more reliable, we would like to add some health checks 
to our upstream peers. That way if there is a peer that has gone down, we can 
quickly stop routing our traffic to that peer.

First let's see how our simple load balancer behaves when one of the peers is
down. To do this, we'll update the list of peers to include a peer that is 
guaranteed to be broken.

```rust
fn main() {
    // ...
    let upstreams =
        LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443", "127.0.0.1:343"]).unwrap();
    // ...
}
```

Now if we run our load balancer again with `cargo run`, and test it with 

```
curl 127.0.0.1:6188 -svo /dev/null
```

We can see that one in every 3 request fails with `502: Bad Gateway`. This is 
because our peer selection is strictly following the `RoundRobin` selection 
pattern we gave it with no consideration to whether that peer is healthy. We can
fix this by adding a basic health check service. 

```rust
fn main() {
    let mut my_server = Server::new(None).unwrap();
    my_server.bootstrap();

    // Note that upstreams needs to be declared as `mut` now
    let mut upstreams =
        LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443", "127.0.0.1:343"]).unwrap();

    let hc = TcpHealthCheck::new();
    upstreams.set_health_check(hc);
    upstreams.health_check_frequency = Some(std::time::Duration::from_secs(1));

    let background = background_service("health check", upstreams);
    let upstreams = background.task();

    // `upstreams` no longer need to be wrapped in an arc
    let mut lb = http_proxy_service(&my_server.configuration, LB(upstreams));
    lb.add_tcp("0.0.0.0:6188");

    my_server.add_service(background);

    my_server.add_service(lb);
    my_server.run_forever();
}
```

Now if we again run and test our load balancer, we see that all requests 
succeed and the broken peer is never used. Based on the configuration we used, 
if that peer were to become healthy again, it would be re-included in the round
robin again in within 1 second.

### Command line options

The pingora `Server` type provides a lot of built-in functionality that we can
take advantage of with single-line change. 

```rust
fn main() {
    let mut my_server = Server::new(Some(Opt::default())).unwrap();
    ...
}
```

With this change, the command-line arguments passed to our load balancer will be 
consumed by Pingora. We can test this by running:

```
cargo run -- -h
```

We should see a help menu with the list of arguments now available to us. We 
will take advantage of those in the next sections to do more with our load 
balancer for free

### Running in the background

Passing the parameter `-d` or `--daemon` will tell the program to run in the background.

```
cargo run -- -d
```

To stop this service, you can send `SIGTERM` signal to it for a graceful shutdown, in which the service will stop accepting new request but try to finish all ongoing requests before exiting.
```
pkill -SIGTERM load_balancer
```
 (`SIGTERM` is the default signal for `pkill`.)

### Configurations
Pingora configuration files help define how to run the service. Here is an 
example config file that defines how many threads the service can have, the 
location of the pid file, the error log file, and the upgrade coordination 
socket (which we will explain later). Copy the contents below and put them into
a file called `conf.yaml` in your `load_balancer` project directory.

```yaml
---
version: 1
threads: 2
pid_file: /tmp/load_balancer.pid
error_log: /tmp/load_balancer_err.log
upgrade_sock: /tmp/load_balancer.sock
```

To use this conf file:
```
RUST_LOG=INFO cargo run -- -c conf.yaml -d
```
`RUST_LOG=INFO` is here so that the service actually populate the error log.

Now you can find the pid of the service.
```
 cat /tmp/load_balancer.pid
```

### Gracefully upgrade the service
(Linux only)

Let's say we changed the code of the load balancer and recompiled the binary. Now we want to upgrade the service running in the background to this newer version.

If we simply stop the old service, then start the new one, some request arriving in between could be lost. Fortunately, Pingora provides a graceful way to upgrade the service.

This is done by, first, send `SIGQUIT` signal to the running server, and then start the new server with the parameter `-u` \ `--upgrade`.

```
pkill -SIGQUIT load_balancer &&\
RUST_LOG=INFO cargo run -- -c conf.yaml -d -u
```

In this process, The old running server will wait and hand over its listening sockets to the new server. Then the old server runs until all its ongoing requests finish.

From a client's perspective, the service is always running because the listening socket is never closed.

## Full examples

The full code for this example is available in this repository under

[pingora-proxy/examples/load_balancer.rs](../pingora-proxy/examples/load_balancer.rs)

Other examples that you may find helpful are also available here

[pingora-proxy/examples/](../pingora-proxy/examples/)
[pingora/examples](../pingora/examples/)