Polish filtra and move data collection to Postresql
a project by jochenbreuer
Updated about 5 years ago. 2 hacker ♥️. 2 followers.

Last hackweek filtra was created – a tool to extract information like lead and cycle times from Github repos for (but not limited to) projects that are doing Kanban. The collected metrics can then be visualized with Grafana.

Currently there are two problems with filtra:

  1. There are two branches. The master branch, that can query information from one project per instance and the multi-board branch, that can query from multiple projects. Sadly the metrics looks a bit different for both of the branches and this needs to be fixed. Also unittests are badly needed! See: Metrics are different #20 and Missing Tests #23
  2. Currently filtra is basically a Prometheus endpoint. But Prometheus is not a perfect fit for this use-case, since we are only collecting a few metrics per day but with a very high retention. This is usually different for data collected with Prometheus. So the conclusion was that Postgresql would be a better fit for storing that data.

Join this project

Looking for hackers with the skills:

go golang github metrics graphql grafana postgresql leadtime cycletime projectmanagement

This project is part of:

Hack Week 19

Activity

  • about 5 years ago: PSuarezHernandez liked this project.
  • about 5 years ago: jcavalheiro liked this project.
  • about 5 years ago: jcavalheiro joined this project.
  • about 5 years ago: jochenbreuer started this project.
  • about 5 years ago: jochenbreuer added keyword "go" to this project.
  • about 5 years ago: jochenbreuer added keyword "golang" to this project.
  • about 5 years ago: jochenbreuer added keyword "github" to this project.
  • about 5 years ago: jochenbreuer added keyword "metrics" to this project.
  • about 5 years ago: jochenbreuer added keyword "graphql" to this project.
  • about 5 years ago: jochenbreuer added keyword "grafana" to this project.
  • about 5 years ago: jochenbreuer added keyword "postgresql" to this project.
  • about 5 years ago: jochenbreuer added keyword "leadtime" to this project.
  • about 5 years ago: jochenbreuer added keyword "cycletime" to this project.
  • about 5 years ago: jochenbreuer added keyword "projectmanagement" to this project.
  • about 5 years ago: jochenbreuer originated this project.


    • Comments

    • jochenbreuer
      about 5 years ago by jochenbreuer | Reply

      First update: We'll go with Sqlite instead of Postgresql.

    Similar Projects

    Metrics Server viewer for Kubernetes by bkampen

    This project is finished please visit the github repo below for the tool.

    Description

    Build a CLI tools which can visualize Kubernetes metrics from the metrics-server, so you're able to watch these without installing Prometheus and Grafana on a cluster.

    Goals

    • Learn more about metrics-server
    • Learn more about the inner workings of Kubernetes.
    • Learn more about Go

    Resources

    https://github.com/bvankampen/metrics-viewer

    Contribute to terraform-provider-libvirt by pinvernizzi

    Description

    The SUSE Manager (SUMA) teams' main tool for infrastructure automation, Sumaform, largely relies on terraform-provider-libvirt. That provider is also widely used by other teams, both inside and outside SUSE.

    It would be good to help the maintainers of this project and give back to the community around it, after all the amazing work that has been already done.

    If you're interested in any of infrastructure automation, Terraform, virtualization, tooling development, Go (...) it is also a good chance to learn a bit about them all by putting your hands on an interesting, real-use-case and complex project.

    Goals

    • Get more familiar with Terraform provider development and libvirt bindings in Go
    • Solve some issues and/or implement some features
    • Get in touch with the community around the project

    Resources

    Cluster API Add-on Provider for Kubewarden by csalas

    Description

    Can we integrate Kubewarden with Cluster API provisioning?

    Cluster API is a Kubernetes project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters. TLDR; CAPI let's you define Kubernetes clusters in plain YAML, and CAPI providers (infrastructure, control plane/bootstrap, etc.) manage provisioning and configuration for you.

    What if we could create an add-on provider that automatically installs Kubewarden and deploys Policy Servers to CAPI clusters?

    Goals

    • As a user I'd like to set a cluster (or list of clusters) and have the provider install Kubewarden for me.
    • As a user I'd like to set what policies must be enforced for a cluster (or list of clusters).

    Resources

    • Cluster API: https://cluster-api.sigs.k8s.io/
    • Kubewarden: https://docs.kubewarden.io/

    ddflare: (Dynamic)DNS management via Cloudflare API in Kubernetes by fgiudici

    Description

    ddflare is a project started a couple of weeks ago to provide DDNS management using v4 Cloudflare APIs: Cloudflare offers management via APIs and access tokens, so it is possible to register a domain and implement a DynDNS client without any other external service but their API.

    Since ddflare allows to set any IP to any domain name, one could manage multiple A and ALIAS domain records. Wouldn't be cool to allow full DNS control from the project and integrate it with your Kubernetes cluster?

    Goals

    Main goals are:

    1. add containerized image for ddflare
    2. extend ddflare to be able to add and remove DNS records (and not just update existing ones)
    3. add documentation, covering also a sample pod deployment for Kubernetes
    4. write a ddflare Kubernetes operator to enable domain management via Kubernetes resources (using kubebuilder)

    Available tasks and improvements tracked on ddflare github.

    Resources

    • https://github.com/fgiudici/ddflare
    • https://developers.cloudflare.com/api/
    • https://book.kubebuilder.io

    Jenny Static Site Generator by adam.pickering

    Description

    For my personal site I have been using hugo. It works, but I am not satisfied: every time I want to make a change (which is infrequently) I have to read through the documentation again to understand how hugo works. I don't find the documentation easy to use, and the structure of the repository that hugo requires is unintuitive/more complex than what I need. So, I have decided to write my own simple static site generator in Go. It is named Jenny, after my wife.

    Goals

    • Pages can be written in markdown (which is automatically converted to HTML), but other file types are also allowed
    • Easy to understand and use
      • Intuitive, simple design
      • Clear documentation
      • Hot reloading
      • Binaries provided for download
    • Future maintenance is easy
      • Automated releases

    Resources

    https://github.com/adamkpickering/jenny

    Learn enough Golang and hack on CoreDNS by jkuzilek

    Description

    I'm implementing a split-horizon DNS for my home Kubernetes cluster to be able to access my internal (and external) services over the local network through public domains. I managed to make a PoC with the k8s_gateway plugin for CoreDNS. However, I soon found out it responds with IPs for all Gateways assigned to HTTPRoutes, publishing public IPs as well as the internal Loadbalancer ones.

    To remedy this issue, a simple filtering mechanism has to be implemented.

    Goals

    • Learn an acceptable amount of Golang
    • Implement GatewayClass (and IngressClass) filtering for k8s_gateway
    • Deploy on homelab cluster
    • Profit?

    Resources

    EDIT: Feature mostly complete. An unfinished PR lies here. Successfully tested working on homelab cluster.

    iSCSI integration in Warewulf by ncuralli

    Description

    This Hackweek project aims to enhance Warewulf’s capabilities by adding iSCSI support, enabling both remote boot and flexible mounting of iSCSI devices within the filesystem. The project, which already handles NFS, DHCP, and iPXE, will be extended to offer iSCSI services as well, centralizing all necessary services for provisioning and booting cluster nodes.

    Goals

    • iSCSI Boot Option: Enable nodes to boot directly from iSCSI volumes
    • Mounting iSCSI Volumes within the Filesystem: Implement support for mounting iSCSI devices at various points within the filesystem

    Resources

    https://warewulf.org/

    Steps

    • add generic framework to handle remote ressource/filesystems to wwctl [ ]
    • add iSCSI handling to wwctl configure [ ]
    • add iSCSI to dracut files [ ]
    • test it [ ]

    suse-rancher-supportconfig by eminguez

    Description

    Update: Live at https://github.com/e-minguez/suse-rancher-supportconfig I finally didn't used golang but used gum instead add-emoji

    SUSE's supportconfig support tool collects data from the SUSE Operating system. Rancher's rancher2_logs_collector.sh support tool does the same for RKE2/K3s.

    Wouldn't be nice to have a way to run both and collect all data for SUSE based RKE2/K3s clusters? Wouldn't be even better with a fancy TUI tool like bubbletea?

    Ideally the output should be an html page where you can see the logs/data directly from the browser.

    Goals

    • Familiarize myself with both supportconfig and rancher2_logs_collector.sh tools
    • Refresh my golang knowledge
    • Have something that works at the end of the hackweek ("works" may vary add-emoji )
    • Be better in naming things

    Resources

    All links provided above as well as huh

    WebUI for your data by avicenzi

    A single place to view every bit of data you have.

    Problem

    You have too much data and you are a data hoarder.

    • Family photos and videos.
    • Lots of eBooks, TV Shows, Movies, and else.
    • Boxes full of papers (taxes, invoices, IDs, certificates, exams, and else).
    • Bank account statements (multiple currencies, countries, and people).

    Maybe you have some data on S3, some on your NAS, and some on your local PC.

    • How do you get it all together?
    • How do you link a bank transaction to a product invoice?
    • How to tag any object type and create a collection out of it (mix videos, photos, PDFs, transactions)?
    • How to store this? file/folder structure does not work, everything is linked together

    Project Description

    The idea is a place where you can throw all your data, photos, videos, documents, binaries, and else.

    Create photo albums, document collections, add tags across multiple file-formats, link content, and else.

    The UI should be easy to use, where the data is not important for now (could be all S3 or local drive).

    Similar proposals

    The closest I found so far is https://perkeep.org/, but this is not what I'm looking for.

    Goal for this Hackweek

    Create a web UI, in Svelte ideally, perhaps React.

    It should be able to show photos and videos at least.

    Resources

    None so far, this is just an idea.

    Hack on rich terminal user interfaces by amanzini

    Description

    TUIs (Textual User Interface) are a big classic of our daily workflow. Many linux users 'live' in the terminal and modern implementations have a lot to offer : unicode fonts, 24 bit colors etc.

    Goals

    • Explore the current available solution on modern languages and implement a PoC , for example a small maze generator, porting of a classic game or just display the HackWeek cute logo.
    • Practice some Go / Rust coding and programming patterns
    • Fiddle around, hack, learn, have fun
    • keep a development diary, practice on project documentation

    Follow this link for source code repository

    Some ideas for inspiration:

    Related projects:

    Resources