Description

AI has the potential to help with something many of us spend a lot of time doing which is making sense of openQA logs when a job fails.

User Story

Allison Average has a puzzled look on their face while staring at log files that seem to make little sense. Is this a known issue, something completely new or maybe related to infrastructure changes?

Goals

  • Leverage a chat interface to help Allison
  • Create a model from scratch based on data from openQA
  • Proof of concept for automated analysis of openQA test results

Bonus

  • Use AI to suggest solutions to merge conflicts
    • This would need a merge conflict editor that can suggest solving the conflict
  • Use image recognition for needles

Resources

Timeline

Day 1

  • Conversing with open-webui to teach me how to create a model based on openQA test results

Day 2

Highlights

  • I briefly tested compared models to see if they would make me more productive. Between llama, gemma and mistral there was no amazing difference in the results for my case.
  • Convincing the chat interface to produce code specific to my use case required very explicit instructions.
  • Asking for advice on how to use open-webui itself better was frustratingly unfruitful both in trivial and more advanced regards.
  • Documentation on source materials used by LLM's and tools for this purpose seems virtually non-existent - specifically if a logo can be generated based on particular licenses

Outcomes

  • Chat interface-supported development is providing good starting points and open-webui being open source is more flexible than Gemini. Although currently some fancy features such as grounding and generated podcasts are missing.
  • Allison still has to be very experienced with openQA to use a chat interface for test review. Publicly available system prompts would make that easier, though.
  • The proof of concept for a model based on test results (Testimony) looks promising, although for real-world use more effort needs to be put into improving the dataset and selecting relevant features.

Looking for hackers with the skills:

ai openqa tensorflow testing python

This project is part of:

Hack Week 24

Activity

  • 6 months ago: livdywan added keyword "python" to this project.
  • 6 months ago: livdywan added keyword "testing" to this project.
  • 6 months ago: livdywan started this project.
  • 6 months ago: livdywan added keyword "ai" to this project.
  • 6 months ago: livdywan added keyword "openqa" to this project.
  • 6 months ago: livdywan added keyword "tensorflow" to this project.
  • 6 months ago: livdywan originated this project.

  • Comments

    Be the first to comment!

    Similar Projects

    This project is one of its kind!