Project Description

Currently openQA requires a reference image to be stored to do OCR based comparisons. It is not possible to pass a character string to openQA which should be compared to the text in the screenshot. This project is about allowing to just store character strings in the corresponding JSON file of the needle and to get rid of any reference images in case of OCR needles.

Status

Research about possible tools was done. The result was that the current implementation based on Tesseract appears to be too inaccurate on short character strings. The program GOCR seems to do more classical recognition by shape which seems to work reasonably accurate on well shaped characters. The accuracy of the matched strings could be calculated using the library perl-Text-Levenshtein.

Goal for this Hackweek

  • Create draft implementation of OCR in os-autoinst.
  • Optional: Create easy handling of text based OCR needles in openQA web frontend (e.g. providing live preview of recognized text)

Resources

  • This project is tracked here: https://progress.opensuse.org/issues/121354
  • openQA frontend repo: https://github.com/os-autoinst/openQA
  • openQA backend repo: https://github.com/os-autoinst/os-autoinst
  • GOCR: https://wasd.urz.uni-magdeburg.de/jschulen/ocr/
  • Perl-Text-Levenshtein: https://github.com/neilb/Text-Levenshtein

Looking for hackers with the skills:

openqa mojolicious perl ocr os-autoinst

This project is part of:

Hack Week 22

Activity

  • almost 2 years ago: okurz liked this project.
  • almost 2 years ago: jzerebecki liked this project.
  • almost 2 years ago: pdostal liked this project.
  • almost 2 years ago: mkoutny liked this project.
  • almost 2 years ago: ggardet_arm left this project.
  • almost 2 years ago: ggardet_arm joined this project.
  • almost 2 years ago: ggardet_arm liked this project.
  • almost 2 years ago: robert.richardson liked this project.
  • almost 2 years ago: dancermak liked this project.
  • almost 2 years ago: ybonatakis liked this project.
  • almost 2 years ago: clanig started this project.
  • almost 2 years ago: clanig added keyword "openqa" to this project.
  • almost 2 years ago: clanig added keyword "mojolicious" to this project.
  • almost 2 years ago: clanig added keyword "perl" to this project.
  • almost 2 years ago: clanig added keyword "ocr" to this project.
  • almost 2 years ago: clanig added keyword "os-autoinst" to this project.
  • almost 2 years ago: clanig originated this project.

  • Comments

    • okurz
      almost 2 years ago by okurz | Reply

      There is very basic support for OCR in os-autoinst with https://github.com/os-autoinst/os-autoinst/blob/master/ocr.pm which might give you some good ideas and a starting base. https://github.com/os-autoinst/os-autoinst/blob/master/t/02-test_ocr.t shows its usage

    • clanig
      almost 2 years ago by clanig | Reply

      Created draft PR: https://github.com/os-autoinst/os-autoinst/pull/2276

    Similar Projects

    Enhance UV openQA helper script by mdonis

    Description

    A couple months ago an UV openQA helper script was created to help/automate the searching phase inside openQA for a given MU to test. The script searches inside all our openQA job groups (qam-sle) related with a given MU and generates an output suitable to add (copy & paste) inside the update log.

    This is still a WIP and could use some enhancements.

    Goals

    • Move script from bash to python: this would be useful in case we want to include this into MTUI in the future. The script will be separate from MTUI for now. The idea is to have this as a CLI tool using the click library or something similar.
    • Add option to look for jobs in other sections inside aggregated updates: right now, when looking for regression tests under aggregated updates for a given MU, the script only looks inside the Core MU job group. This is where most of the regression tests we need are located, but some MUs have their regression tests under the YaST/Containers/Security MU job groups. We should keep the Core MU group as a default, but add an option to be able to look into other job groups under aggregated updates.
    • Remove the -a option: this option is used to indicate the update ID and is mandatory right now. This is a bit weird and goes against posix stardards. It was developed this way in order to avoid using positional parameters. This problem should be fixed if we move the script to python.

    Some other ideas to consider:

    • Look into the QAM dashboard API. This has more info on each MU, could use this to link general openQA build results, whether the related RR is approved or not, etc
    • Make it easier to see if there's regression tests for a package in an openQA test build. Check if there's a possibility to search for tests that have the package name in them inside each testsuite.
    • Unit testing?

    More ideas TBD

    Resources

    https://github.com/os-autoinst/scripts/blob/master/openqa-search-maintenance-core-jobs

    https://confluence.suse.com/display/maintenanceqa/Guide+on+how+to+test+Updates

    Post-Hackweek update

    All major features were implemented. Unit tests are still in progress, and project will be moved to the SUSE github org once everything's done. https://github.com/mjdonis/oqa-search


    Hack on isotest-ng - a rust port of isotovideo (os-autoinst aka testrunner of openQA) by szarate

    Description

    Some time ago, I managed to convince ByteOtter to hack something that resembles isotovideo but in Rust, not because I believe that Perl is dead, but more because there are certain limitations in the perl code (how it was written), and its always hard to add new functionalities when they are about implementing a new backend, or fixing bugs (Along with people complaining that Perl is dead, and that they don't like it)

    In reality, I wanted to see if this could be done, and ByteOtter proved that it could be, while doing an amazing job at hacking a vnc console, and helping me understand better what RuPerl needs to work.

    I plan to keep working on this for the next few years, and while I don't aim for feature completion or replacing isotovideo tih isotest-ng (name in progress), I do plan to be able to use it on a daily basis, using specialized tooling with interfaces, instead of reimplementing everything in the backend

    Todo

    • Add make targets for testability, e.g "spawn qemu and type"
    • Add image search matching algorithm
    • Add a Null test distribution provider
    • Add a Perl Test Distribution Provider
    • Fix unittests https://github.com/os-autoinst/isotest-ng/issues/5
    • Research OpenTofu how to add new hypervisors/baremetal to OpenTofu
    • Add an interface to openQA cli

    Goals

    • Implement at least one of the above, prepare proposals for GSoC
    • Boot a system via it's BMC

    Resources

    See https://github.com/os-autoinst/isotest-ng


    OpenQA Golang api client by hilchev

    Description

    I would like to make a simple cli tool to communicate with the OpenQA API

    Goals

    • OpenQA has a ton of information that is hard to get via the UI. A tool like this would make my life easier :)
    • Would potentially make it easier in the future to make UI changes without Perl.
    • Improve my Golang skills

    Resources

    • https://go.dev/doc/
    • https://openqa.opensuse.org/api


    Make more sense of openQA test results using AI by livdywan

    Description

    AI has the potential to help with something many of us spend a lot of time doing which is making sense of openQA logs when a job fails.

    User Story

    Allison Average has a puzzled look on their face while staring at log files that seem to make little sense. Is this a known issue, something completely new or maybe related to infrastructure changes?

    Goals

    • Leverage a chat interface to help Allison
    • Create a model from scratch based on data from openQA
    • Proof of concept for automated analysis of openQA test results

    Bonus

    • Use AI to suggest solutions to merge conflicts
      • This would need a merge conflict editor that can suggest solving the conflict
    • Use image recognition for needles

    Resources

    Timeline

    Day 1

    • Conversing with open-webui to teach me how to create a model based on openQA test results

    Day 2

    Highlights

    • I briefly tested compared models to see if they would make me more productive. Between llama, gemma and mistral there was no amazing difference in the results for my case.
    • Convincing the chat interface to produce code specific to my use case required very explicit instructions.
    • Asking for advice on how to use open-webui itself better was frustratingly unfruitful both in trivial and more advanced regards.
    • Documentation on source materials used by LLM's and tools for this purpose seems virtually non-existent - specifically if a logo can be generated based on particular licenses

    Outcomes

    • Chat interface-supported development is providing good starting points and open-webui being open source is more flexible than Gemini. Although currently some fancy features such as grounding and generated podcasts are missing.
    • Allison still has to be very experienced with openQA to use a chat interface for test review. Publicly available system prompts would make that easier, though.


    Learn obs/ibs sync tool by xlai

    Description

    Once images/repo are built from IBS/OBS, there is a tool to sync the image from IBS/OBS to openqa asset directory and trigger openqa jobs accordingly.

    Goals

    Check how the tool is implemented, and be capable to add/modify our needed images/repo in future by ourselves.

    Resources

    • https://github.com/os-autoinst/openqa-trigger-from-obs
    • https://gitlab.suse.de/openqa/openqa-trigger-from-ibs-plugin/-/tree/master?ref_type=heads


    Create object oriented API for perl's YAML::XS module, with YAML 1.2 Support by tinita

    Description

    YAML::XS is a binding to libyaml and already quite old, but the most popular YAML module for perl. There are two main issues:

    • It uses global package variables to influence behaviour.
    • It didn't implement the loading of types like numbers and booleans according to the YAML spec (neither 1.1 nor 1.2).

    Goals

    Create a new interface which works object oriented. Currently YAML::XS exports a list of functions.

    • The new API will allow to create a YAML::XS object containing configuration influencing the behaviour of loading and dumping.
      • It keeps the libyaml parser and emitter structs in memory, so repeated calls can save the creation of those structs
    • It will by default implement the YAML 1.2 Core Schema, so it is compatible to other YAML processors in perl and in other languages
    • If I have time, I would like to add the merge << key feature as an option. We could then use it in openQA as a replacement for YAML::PP to be faster.

    I already created a proof of concept with a minimal functionality some weeks before this HackWeek.

    Resources