L0 Supportconfig Monitoring and Analysis using ML
a project by andavis
Updated over 3 years ago. 10 hacker ♥️. 6 followers.

Project Description

  • Create an automated L0-support-like analytics solution for supportconfig data that is tiered across a customer's environment and SUSE environment (similar to a very modular AIOps Edge-Core approach). A pictorial overview of the ecosystem SupportConfigAnalytics
  • Monitor aspect (lower right) - Start data pipeline with trigger/hint/clue based collection of system's information (like supportconfig) plus some change-centric metadata, then aggregate for step-wise run analysis tool(s) to provide time-series observations and potential/suggested actions, all within the local environment.
  • Analyzer aspect (middle and upper left): Create models from existing supplied data to provide "Nearest Neighbor" analysis to determine whether a submitted/collected dataset (supportconfig) looks like a current cert, SR, or bug.

Goals for this Hackweek

  • Continue to collect more example data submissions for testing
  • Iterate (and further automate) model and training dataset creation
  • Iterate data/model version control and regression testing
  • Augment more hint/clue modules to trigger data collection
  • Iterate (and multi-package as RPM/containers) all components
  • Iterate (and test) the customer side of data pipeline, adding time-series checks
  • Be able to PoC demo an end-to-end usage of the above ecosystem

Resources

  • Monitoring (mostly bash-centric)
  • Analyzer (mostly python-centric)
  • Peer review (customer, partner, support) of PoC demo

Join this project

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 20

Activity

  • over 2 years ago: nicoladm liked this project.
  • over 3 years ago: johnmpugh liked this project.
  • almost 4 years ago: acho liked this project.
  • almost 4 years ago: vliaskovitis liked this project.
  • almost 4 years ago: SLindoMansilla liked this project.
  • almost 4 years ago: mkoutny liked this project.
  • almost 4 years ago: anthidote joined this project.
  • almost 4 years ago: mlnoga liked this project.
  • almost 4 years ago: xgonzo liked this project.
  • almost 4 years ago: gboiko liked this project.
  • almost 4 years ago: moio liked this project.
  • almost 4 years ago: bwgartner joined this project.
  • almost 4 years ago: andavis started this project.
  • almost 4 years ago: andavis originated this project.


    • Comments

    • mlnoga
      almost 4 years ago by mlnoga | Reply

      @dirkmueller had a similar idea for topic clustering and topic extraction of L3 support tickets with NLP toolkits. You might want to link up.

      @puzel and team have started manually tagging root causes of L3 tickets, too.

    • anthidote
      almost 4 years ago by anthidote | Reply

      Hi,

      There is an existing supportconfig analysis project maintained by Jason Record. It might be interesting to combine effort, maybe make this a plugin to the existing python framework?

      GitHub link:

      https://github.com/openSUSE/sca-server-report

      • bwgartner
        almost 4 years ago by bwgartner | Reply

        Understand and had reached out to Jason some time back on this topic to see about the planned path forward for that project. Was hoping to address some gaps with this project and maybe synergistically improve both

    • andavis
      almost 4 years ago by andavis | Reply

      Thanks for the pointers (just downloaded sca-server-report...)! I think there are several efforts going on wrt supportconfig analysis, so definitely agree that we should figure out how to combine efforts. The piece that I'm most interested in (and what I've mostly been doing) is trying to relate new supportconfigs to existing issues (SRs/bugs) using vectorized supportconfig data and nearest-neighbor analysis. Original idea was to create datasets for each supportconfig file (and that's what I'm currently doing w/ the messages.txt files). But I think a more interesting/technically valuable idea might be to align datasets with pre-determined topics (e.g., bugzilla "components").

      • anthidote
        over 3 years ago by anthidote | Reply

        Certainly an interesting idea, I'm [Anthony Stalker] in L1 support and I'm trying to develop Jason's sca-server-report tool.

        A "fuzzy matching" to other cases based on supportconfig data could certainly be interesting to avoid duplication of efforts and streamline efforts in resolving (not just) defects. The data generated by support engineers confirming or rejecting that match could then potentially improve and train the model.

    Similar Projects

    This project is one of its kind!