Work on kqlite (Lightweight remote SQLite with high availability and auto failover).
a project by epenchev
Updated about 1 month ago. 3 hacker ♥️. 3 followers.

Description

Continue the work on kqlite (Lightweight remote SQLite with high availability and auto failover).
It's a solution for applications that require High Availability but don't need all the features of a complete RDBMS and can fit SQLite in their use case.
Also kqlite can be considered to be used as a lightweight storage backend for K8s (https://docs.k3s.io/datastore) and the Edge, and allowing to have only 2 Nodes for HA.
Kqlite was somehow inspired from dqlite and rqlite, although they are using Raft to create a replication cluster.

Goals

  • Upgrade the client connection to TLS/SSL.
  • Add configuration.
  • Add SFTP service (for the replication).
  • Add replication and roles (Primary/Secondary).
  • Create HA logic with VIP management.
  • Create a 'Kine' layer for rewriting PostgreSQL queries to SQLite compatible ones.

Resources

https://github.com/benbjohnson/postlite
https://github.com/pganalyze/pg_query
https://github.com/benbjohnson/litestream
https://github.com/benbjohnson/litestream-library-example/tree/main
https://pkg.go.dev/github.com/pkg/sftp

HA Network split check.

  • Check secondary ( monitoring the replication link).
  • Check network interface.
  • Check default gateway (if present).
  • Check neighbors (ARP).
  • Check Internet connectivity (if present).

Join this project

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 24

Activity

  • about 1 month ago: jmeza liked this project.
  • about 1 month ago: epenchev liked this project.
  • about 1 month ago: agamez liked this project.
  • about 1 month ago: nchung joined this project.
  • about 1 month ago: epenchev disliked this project.
  • about 1 month ago: epenchev liked this project.
  • about 1 month ago: epenchev started this project.
  • about 1 month ago: epenchev originated this project.


    • Comments

    • epenchev
      20 days ago by epenchev | Reply

      As of now I've been experimenting mostly with the replication. Currently this is the current replication implementation https://github.com/kqlite/kqlite/pull/1 using Litestream and SFTP as in the initial plan. The cluster topology is Primary->Secondary with physical streaming replication between the nodes. Primary is the active one serving the database and replicating to Secondary and in case of failure takes over the database ownership.

    Similar Projects

    This project is one of its kind!