Description

TL;DR: I'd like to have something like this for YaST log.

YaST logs all actions it does into the /var/log/YaST2/y2log file. The log is very detailed and contains every detail of the YaST run. And that's a problem! Even a default openSUSE Leap 15.4 installation where you basically just press Next, Next,... produces about a 13MB log file (uncompressed) with more than 80k lines!

Navigating in such log is very difficult, you can see every tiny detail what was done, but it is difficult to find the big picture, why was that happening, what was the goal of that part.

Also some code parts repeat during the installation. For example evaluating the base product is done several times during installation, in the self-update step, when displaying the base product license, when registering the system...

So if you see in the log the base product evaluation you cannot be sure to which step it belongs, you have to scroll up and down to find it. This also means you have to start reading the log from the very beginning and go on so you know at which point you currently are. And that is terrible.

The Idea

Let's introduce some grouping in the log, to define some structure so you can see what was happening, where it started, where it finished and what was the result.

I got inspired by the GitHub Action log grouping:

See more details in the GitHub documentation.

GitHub Actions do not allow nested groups, only one grouping level is possible. The YaST grouping should allow unlimited nested groups. That should allow easy navigation in the log, you should easily get into the point you are interested in, or the other way round, you could easily know where exactly you are.

Something like this:

▼ Registration step
  ▶ Enter registration data
  ▶ Register the system
  ▶ Register the "SLES-15-SP4" base product
  ▶ Select modules and extensions
  ▼ Register modules and extensions
    ▶ Register "Basesystem" module
    ▼ Register "Server Applications" module
      ▶ Contacting SCC
      ▶ Adding "Pool" repository
      ▶ Adding "Updates" repository
    ▶ Register "Desktop Applications" module
    ...

You can see an example y2log with manually added group markers to get an idea how it could look like.

Improving the YaST Logger

First we need to enhance the YaST logger to add special log markers which will mark each group. We could use the same tags as GitHub Actions (::group:: and ::endgroup::) logged as usual log messages. This would keep the backward compatibility with the current log format.

Example code:

log.group "Registering the system" do
  registration.register(...)
end

The result of the block would be also the result of the logging group. If the result is a failure (like false, :abort, Exception) we could display the log group header in red, maybe with some error icon ( ).

Adding Result Data

It would be nice to have some result summary for each group so you not have to go into the details.

log.group "Adding online repositories" do |g|
  ...
  g.details = "added #{repos.size} repositories"
  :next
end

This would be displayed like this:

▶ Adding online repositories (added 5 repositories)

Custom Failure Reporting

The predefined error values might not be enough in some cases, allow reporting an error explicitly:

log.group "Adding online repositories" do |g|
  ...
  if ret.empty?
    g.failure = true
    g.details = "No repository added"
  end

  :skip
end

This should be displayed in the log view like this (in red)

▶ ⚠️ Adding online repositories (No repository added)

The Log Browser

Of course, the added markers allow to easily find the start and end of a block, but it would be nice to have some browser which would visualize the structure of the log.

Writing a desktop application would not be bad but generating an HTML page with the structured log would be easier. That will also allow us to easily create a web service for that so you could upload the log and directly see the result in the web browser. We could also containerize it so you could easily run it locally.

And because YaST is written in Ruby and Ruby is also used for web applications let's write the HTML generator in Ruby.

Goal for this Hackweek

• Add grouping support to the YaST logger
• Use it at some places in YaST (at least in the installation workflow manager so we know where each step starts/end)
• Create a script which can parse a single YaST y2log file and render a resulting HTML page
• Support multiple YaST runs in the log
• Colorized output (errors in red, ....)
• Display nested log groups, allow expanding/collapsing groups with a ▶button
• Basic filtering (show/hide debug messages)
• Generate a single document HTML (with JS and CSS embedded) for easier sharing the generated document or HTML with separate JS and CSS files

Future Enhancements

• Advanced filtering (component based: show/hide libzypp messages, libyui messages, YaST agents,...)
• Allow reading compressed logs (like y2log-1.gz)
• Allow reading whole tarballs created by save_y2log script
• Create a web application, instead of running the script allow uploading it to a server running locally and directly display the rendered page
• Extract some high level data from the log as a description of whole process (Upgrading from SLES12-SP3 to SLES15-SP4) or to detect some unusual situations (Using custom registration server http://rmt.example.com). This would be displayed at the top as the log summary
• Containerize the script and the web application
• Host the web application at some free hosting service

Needed Skills

• Ruby (text parsing and processing, generating HTML, ERB)
• Web Development (HTML, CSS, JavaScript)
• YaST basics (optional, the y2log structure is trivial)