Why

Knowledge is only useful if it is accessible.

What

A tool to search the SUSE world and help you find the things that really matter to you.

How

Index all Wiki pages (Github, MF Wiki, etc) and other R&D relevant sources of knowledge. Search interface should be available in both Web and command line formats and support full-text search.

Implementation ideas

  • Lucene-based search engine (ex: ElasticSearch)
  • Indexing daemon (Golang or Python)
  • REST Api
  • HTML / Javascript frontend (ex: React.js)
  • Command line search tool (Golang and Python)

Looking for hackers with the skills:

golang python3

This project is part of:

Hack Week 16

Activity

  • about 7 years ago: bfilho liked this project.
  • about 7 years ago: sven15 liked this project.
  • about 7 years ago: sven15 started this project.
  • about 7 years ago: hennevogel liked this project.
  • about 7 years ago: mwilck liked this project.
  • about 7 years ago: whdu liked this project.
  • about 7 years ago: slahl liked this project.
  • about 7 years ago: dmaiocchi liked this project.
  • about 7 years ago: dmaiocchi liked this project.
  • about 7 years ago: jcavalheiro added keyword "golang" to this project.
  • about 7 years ago: jcavalheiro added keyword "python3" to this project.
  • about 7 years ago: jcavalheiro originated this project.

  • Comments

    • dmacvicar
      about 7 years ago by dmacvicar | Reply

      https://hackweek.suse.com/projects/intranet-search-engine

    • moio
      about 7 years ago by moio | Reply

      http://fastzilla.suse.de/ is another initiative to have a fast index at least on Bugzilla

    • gsanso
      about 7 years ago by gsanso | Reply

      IRC logs would also be a good source of data.

      • jcavalheiro
        about 7 years ago by jcavalheiro | Reply

        Definitely. Do we have a server copy of channel logs somewhere?

    • sven15
      about 7 years ago by sven15 | Reply

      Everybody interested in this project: let's talk on friday! We're already working on something like that. Or maybe exactly the same: Elasticsearch, Python crawler, web UI (docserv.nue.suse.com).

    • sven15
      about 7 years ago by sven15 | Reply

      Please join #search on IRC

    Similar Projects

    suse-rancher-supportconfig by eminguez

    Description

    Update: Live at https://github.com/e-minguez/suse-rancher-supportconfig I finally didn't used golang but used gum instead add-emoji

    SUSE's supportconfig support tool collects data from the SUSE Operating system. Rancher's rancher2_logs_collector.sh support tool does the same for RKE2/K3s.

    Wouldn't be nice to have a way to run both and collect all data for SUSE based RKE2/K3s clusters? Wouldn't be even better with a fancy TUI tool like bubbletea?

    Ideally the output should be an html page where you can see the logs/data directly from the browser.

    Goals

    • Familiarize myself with both supportconfig and rancher2_logs_collector.sh tools
    • Refresh my golang knowledge
    • Have something that works at the end of the hackweek ("works" may vary add-emoji )
    • Be better in naming things

    Resources

    All links provided above as well as huh


    file-organizer: A CLI Tool for Efficient File Management by okhatavkar

    Description

    Create a Go-based CLI tool that helps organize files in a specified folder by sorting them into subdirectories based on defined criteria, such as file type or creation date. Users will pass a folder path as an argument, and the tool will process and organize the files within it.

    Goals

    • Develop Go skills by building a practical command-line application.
    • Learn to manage and manipulate files and directories in Go using standard libraries.
    • Create a tool that simplifies file management, making it easier to organize and maintain directories.

    Resources

    • Go Standard Libraries: Utilize os, filepath, and time for file operations.
    • CLI Development: Use flag for basic argument parsing or consider cobra for enhanced functionality.
    • Go Learning Material: Go by Example and The Go Programming Language Documentation.

    Features

    • File Type Sorting: Automatically move files into subdirectories based on their extensions (e.g., documents, images, videos).
    • Date-Based Organization: Add an option to organize files by creation date into year/month folders.
    • User-Friendly CLI: Build intuitive commands and clear outputs for ease of use. This version maintains the core idea of organizing files efficiently while focusing on Go development and practical file management.


    Jenny Static Site Generator by adam.pickering

    Description

    For my personal site I have been using hugo. It works, but I am not satisfied: every time I want to make a change (which is infrequently) I have to read through the documentation again to understand how hugo works. I don't find the documentation easy to use, and the structure of the repository that hugo requires is unintuitive/more complex than what I need. So, I have decided to write my own simple static site generator in Go. It is named Jenny, after my wife.

    Goals

    • Pages can be written in markdown (which is automatically converted to HTML), but other file types are also allowed
    • Easy to understand and use
      • Intuitive, simple design
      • Clear documentation
      • Hot reloading
      • Binaries provided for download
    • Future maintenance is easy
      • Automated releases

    Resources

    https://github.com/adamkpickering/jenny


    ClusterOps - Easily install and manage your personal kubernetes cluster by andreabenini

    Description

    ClusterOps is a Kubernetes installer and operator designed to streamline the initial configuration and ongoing maintenance of kubernetes clusters. The focus of this project is primarily on personal or local installations. However, the goal is to expand its use to encompass all installations of Kubernetes for local development purposes.
    It simplifies cluster management by automating tasks and providing just one user-friendly YAML-based configuration config.yml.

    Overview

    • Simplified Configuration: Define your desired cluster state in a simple YAML file, and ClusterOps will handle the rest.
    • Automated Setup: Automates initial cluster configuration, including network settings, storage provisioning, special requirements (for example GPUs) and essential components installation.
    • Ongoing Maintenance: Performs routine maintenance tasks such as upgrades, security updates, and resource monitoring.
    • Extensibility: Easily extend functionality with custom plugins and configurations.
    • Self-Healing: Detects and recovers from common cluster issues, ensuring stability, idempotence and reliability. Same operation can be performed multiple times without changing the result.
    • Discreet: It works only on what it knows, if you are manually configuring parts of your kubernetes and this configuration does not interfere with it you can happily continue to work on several parts and use this tool only for what is needed.

    Features

    • distribution and engine independence. Install your favorite kubernetes engine with your package manager, execute one script and you'll have a complete working environment at your disposal.
    • Basic config approach. One single config.yml file with configuration requirements (add/remove features): human readable, plain and simple. All fancy configs managed automatically (ingress, balancers, services, proxy, ...).
    • Local Builtin ContainerHub. The default installation provides a fully configured ContainerHub available locally along with the kubernetes installation. This configuration allows the user to build, upload and deploy custom container images as they were provided from external sources. Internet public sources are still available but local development can be kept in this localhost server. Builtin ClusterOps operator will be fetched from this ContainerHub registry too.
    • Kubernetes official dashboard installed as a plugin, others planned too (k9s for example).
    • Kubevirt plugin installed and properly configured. Unleash the power of classic virtualization (KVM+QEMU) on top of Kubernetes and manage your entire system from there, libvirtd and virsh libs are required.
    • One operator to rule them all. The installation script configures your machine automatically during installation and adds one kubernetes operator to manage your local cluster. From there the operator takes care of the cluster on your behalf.
    • Clean installation and removal. Just test it, when you are done just use the same program to uninstall everything without leaving configs (or pods) behind.

    Planned features (Wishlist / TODOs)

    • Containerized Data Importer (CDI). Persistent storage management add-on for Kubernetes to provide a declarative way of building and importing Virtual Machine Disks on PVCs for


    WebUI for your data by avicenzi

    A single place to view every bit of data you have.

    Problem

    You have too much data and you are a data hoarder.

    • Family photos and videos.
    • Lots of eBooks, TV Shows, Movies, and else.
    • Boxes full of papers (taxes, invoices, IDs, certificates, exams, and else).
    • Bank account statements (multiple currencies, countries, and people).

    Maybe you have some data on S3, some on your NAS, and some on your local PC.

    • How do you get it all together?
    • How do you link a bank transaction to a product invoice?
    • How to tag any object type and create a collection out of it (mix videos, photos, PDFs, transactions)?
    • How to store this? file/folder structure does not work, everything is linked together

    Project Description

    The idea is a place where you can throw all your data, photos, videos, documents, binaries, and else.

    Create photo albums, document collections, add tags across multiple file-formats, link content, and else.

    The UI should be easy to use, where the data is not important for now (could be all S3 or local drive).

    Similar proposals

    The closest I found so far is https://perkeep.org/, but this is not what I'm looking for.

    Goal for this Hackweek

    Create a web UI, in Svelte ideally, perhaps React.

    It should be able to show photos and videos at least.

    Resources

    None so far, this is just an idea.


    Selenium with Python by xguo

    Description

    Try to create test case about Selenium base on Python

    Goals

    • Knowledge about Selenium with Python
    • Create new test case about Selenium

    Resources

    https://selenium-python.readthedocs.io/ https://www.selenium.dev/


    Small healthcheck tool for Longhorn by mbrookhuis

    Project Description

    We have often problems (e.g. pods not starting) that are related to PVCs not running, cluster (nodes) not all up or deployments not running or completely running. This all prevents administration activities. Having something that can regular be run to validate the status of the cluster would be helpful, and not as of today do a lot of manual tasks.

    As addition (read enough time), we could add changing reservation, adding new disks, etc. --> This didn't made it. But the scripts can easily be adopted.

    This tool would decrease troubleshooting time, giving admins rights to the rancher GUI and could be used in automation.

    Goal for this Hackweek

    At the end we should have a small python tool that is doing a (very) basic health check on nodes, deployments and PVCs. First attempt was to make it in golang, but that was taking to much time.

    Overview

    This tool will run a simple healthcheck on a kubernetes cluster. It will perform the following actions:

    • node check: This will check all nodes, and display the status and the k3s version. If the status of the nodes is not "Ready" (this should be only reported), the cluster will be reported as having problems

    • deployment check: This check will list all deployments, and display the number of expected replicas and the used replica. If there are unused replicas this will be displayed. The cluster will be reported as having problems.

    • pvc check: This check will list of all pvc's, and display the status and the robustness. If the robustness is not "Healthy", the cluster will be reported as having problems.

    If there is a problem registered in the checks, there will be a warning that the cluster is not healthy and the program will exit with 1.

    The script has 1 mandatory parameter and that is the kubeconf of the cluster or of a node off the cluster.

    The code is writen for Python 3.11, but will also work on 3.6 (the default with SLES15.x). There is a venv present that will contain all needed packages. Also, the script can be run on the cluster itself or any other linux server.

    Installation

    To install this project, perform the following steps:

    • Create the directory /opt/k8s-check

    mkdir /opt/k8s-check

    • Copy all the file to this directory and make the following changes:

    chmod +x k8s-check.py


    Enhance UV openQA helper script by mdonis

    Description

    A couple months ago an UV openQA helper script was created to help/automate the searching phase inside openQA for a given MU to test. The script searches inside all our openQA job groups (qam-sle) related with a given MU and generates an output suitable to add (copy & paste) inside the update log.

    This is still a WIP and could use some enhancements.

    Goals

    • Move script from bash to python: this would be useful in case we want to include this into MTUI in the future. The script will be separate from MTUI for now. The idea is to have this as a CLI tool using the click library or something similar.
    • Add option to look for jobs in other sections inside aggregated updates: right now, when looking for regression tests under aggregated updates for a given MU, the script only looks inside the Core MU job group. This is where most of the regression tests we need are located, but some MUs have their regression tests under the YaST/Containers/Security MU job groups. We should keep the Core MU group as a default, but add an option to be able to look into other job groups under aggregated updates.
    • Remove the -a option: this option is used to indicate the update ID and is mandatory right now. This is a bit weird and goes against posix stardards. It was developed this way in order to avoid using positional parameters. This problem should be fixed if we move the script to python.

    Some other ideas to consider:

    • Look into the QAM dashboard API. This has more info on each MU, could use this to link general openQA build results, whether the related RR is approved or not, etc
    • Make it easier to see if there's regression tests for a package in an openQA test build. Check if there's a possibility to search for tests that have the package name in them inside each testsuite.
    • Unit testing?

    More ideas TBD

    Resources

    https://github.com/os-autoinst/scripts/blob/master/openqa-search-maintenance-core-jobs

    https://confluence.suse.com/display/maintenanceqa/Guide+on+how+to+test+Updates

    Post-Hackweek update

    All major features were implemented. Unit tests are still in progress, and project will be moved to the SUSE github org once everything's done. https://github.com/mjdonis/oqa-search


    Symbol Relations by hli

    Description

    There are tools to build function call graphs based on parsing source code, for example, cscope.

    This project aims to achieve a similar goal by directly parsing the disasembly (i.e. objdump) of a compiled binary. The assembly code is what the CPU sees, therefore more "direct". This may be useful in certain scenarios, such as gdb/crash debugging.

    Detailed description and Demos can be found in the README file:

    Supports x86 for now (because my customers only use x86 machines), but support for other architectures can be added easily.

    Tested with python3.6

    Goals

    Any comments are welcome.

    Resources

    https://github.com/lhb-cafe/SymbolRelations

    symrellib.py: mplements the symbol relation graph and the disassembly parser

    symrel_tracer*.py: implements tracing (-t option)

    symrel.py: "cli parser"