Some research on HA and proxies and mirrors
an idea by jordimassaguerpla
Updated about 7 years ago. No love. Has no hacker: grab it!

Description of the problem:

There is a website which we want it to be Available even in the following situations:

  • The servers are not available (i.e. DDoS attack)

  • The cluster is not available

  • The domain is not available (i.e. DNS is down, the domain is in a blacklist, etc.)

  • The website is broken. It is available but is returning the wrong content (i.e. because of some bug or issue with the database)

What we need is kind of an external load balancing even between alternative domains.

However, there is something to consider. Having an external balancer is also a point of failure. If that is taken down ... What could be done in this situation? An idea would be that this balancer is "private", so each user or community has his own, installed on their private cluster, network, or even installed on their laptops?, ... but then how is the list of servers updated? And how do we stablish the trust on the list of servers?

I don't have much experience on HA nor proxies, so this is more of a research project.

The tasks will be:

  • Try to configure HAProxy as an external load/mirror balancing (is this doable?) and configurable as a proxy

  • "think" of a way to update the configuration on a secure manner

  • Implement a Proof Of Concept of 2

No Hackers yet

Join this project

Looking for hackers with the skills:

haproxy blockchain dns

This project is part of:

Hack Week 16

Activity

  • about 7 years ago: jordimassaguerpla added keyword "dns" to this project.
  • about 7 years ago: jordimassaguerpla added keyword "blockchain" to this project.
  • about 7 years ago: jordimassaguerpla added keyword "haproxy" to this project.
  • about 7 years ago: jordimassaguerpla originated this project.


    • Comments

    • jordimassaguerpla
      about 7 years ago by jordimassaguerpla | Reply

      While looking into the problem of how to "distribute" a list of server names in a trusted way, I came across blockstack and namecoin, which, simplifying a lot, are decentralized DNS based on Blockchain technology (like bitcoin).

      At the same time, looking on load balancing solutions I came across DNS round-robin, which in short is the DNS returning multiple A records for a domain name, that is, multiple IPs for one single name.

      Searching a bit more, I found out that modern browsers, when they look for a name, that is, they try to resolve it, they accept a list of IP addresses. Then, they will choose one and, if the connection fails, they will use another one from the list. In practice, this is DNS round robin HA. It does not balance the load but it works for High Availability.

      Thus, if we connect those ideas, my proposal to solve that problem would be to use blockchain technology, either namecoin or blockstack, as decentralized DNS, which will store multiple A records for one name, and configure that DNS on our system, so our browsers will do the round robin HA.

      This will fix most of the problems, except for the one that the website is broken (it returns something but it is not what you would expect).

    • jordimassaguerpla
      about 7 years ago by jordimassaguerpla | Reply

      PR that adds multiple A records to the namecoin DNS.

      https://github.com/namecoin/nmcontrol/pull/121

    • jordimassaguerpla
      about 7 years ago by jordimassaguerpla | Reply

      By using the previous PR, I was able to setup a "Decentralized High Availability" by:

      • Using namecoins to have a decentralized domain name, in this example jordia65.bit, which had multiple A records (2 ip addresses). I bought this bit domain in peername.com. See: https://namecha.in/name/d/jordia65

      • Starting 2 webservers in digital ocean with the IP addresses that jordia65.bit domain "links to"

      • Installing namecoind and namecoin-qt in order to have a local copy of the namecoin blockchain history

      • Setting up nmcontrol to query the namecoind daemon

      Then, the test was to:

      1- Use nslookup to look for jordia65.bit, and effectively, it returned 2 ip addresses.

      2- Use wget to get http://jordia65.bit. It downloaded the index.html from the first server in digital ocean.

      3- Shutdown the server.

      4- Use wget again and see how wget tells you that the first server is unavailable and tries up with the second one.

      5- Do the same test with firefox.

      So, it is possible to setup a Decentralized HA by using namecoins.

    • jordimassaguerpla
      about 7 years ago by jordimassaguerpla | Reply

      Before doing all that, I tried setting up a proxy on a vm with apache2+mod_proxy. Then, in that same vm, I set up a DNS server with the jordia65.bit "zone" so that it returned 2 ip addresses.

      Then, I setup firefox to use that proxy and did the same tests as before, and I realized that this worked, meaning firefox was delegating the dns resolution to the proxy but if the first server was down, firefox was connecting to the second one.

    • jordimassaguerpla
      about 7 years ago by jordimassaguerpla | Reply

      My conclusion is that we should be able to setup a decentralized HA by setting up namecoin+nmcontrol in a proxy, and then configuring our browser to use that proxy.

    Similar Projects

    ddflare: (Dynamic)DNS management via Cloudflare API in Kubernetes by fgiudici

    Description

    ddflare is a project started a couple of weeks ago to provide DDNS management using v4 Cloudflare APIs: Cloudflare offers management via APIs and access tokens, so it is possible to register a domain and implement a DynDNS client without any other external service but their API.

    Since ddflare allows to set any IP to any domain name, one could manage multiple A and ALIAS domain records. Wouldn't be cool to allow full DNS control from the project and integrate it with your Kubernetes cluster?

    Goals

    Main goals are:

    1. add containerized image for ddflare
    2. extend ddflare to be able to add and remove DNS records (and not just update existing ones)
    3. add documentation, covering also a sample pod deployment for Kubernetes
    4. write a ddflare Kubernetes operator to enable domain management via Kubernetes resources (using kubebuilder)

    Available tasks and improvements tracked on ddflare github.

    Resources

    • https://github.com/fgiudici/ddflare
    • https://developers.cloudflare.com/api/
    • https://book.kubebuilder.io

    Learn enough Golang and hack on CoreDNS by jkuzilek

    Description

    I'm implementing a split-horizon DNS for my home Kubernetes cluster to be able to access my internal (and external) services over the local network through public domains. I managed to make a PoC with the k8s_gateway plugin for CoreDNS. However, I soon found out it responds with IPs for all Gateways assigned to HTTPRoutes, publishing public IPs as well as the internal Loadbalancer ones.

    To remedy this issue, a simple filtering mechanism has to be implemented.

    Goals

    • Learn an acceptable amount of Golang
    • Implement GatewayClass (and IngressClass) filtering for k8s_gateway
    • Deploy on homelab cluster
    • Profit?

    Resources

    EDIT: Feature mostly complete. An unfinished PR lies here. Successfully tested working on homelab cluster.