Hackweek 18 Update

What Has Been Done During HackWeek 18

• The REST API plugin (libyui-rest-api RPM) has been split into two independent parts (for Qt and Ncurses frontends). Now it is possible to install the REST API plugin into a minimal system without installing the whole X11 stack, just install the libyui-ncurses-rest-api plugin for the Ncurses UI.
• It turned out that some widgets with the :notify property set do not send the change event when the value was changed via the REST API. Several widgets have been fixed but there still might be some broken widgets. We should later test all widgets and make sure it works everywhere.
• Additionally the REST API now allows using the newer IPv6 protocol. There is one quite interesting feature in IPv6 - autoconfiguration. The link local address is automatically created from the MAC address from the network card. This means you can know the IPv6 address of a testing virtual machine even before starting it. That might be useful for automated tests.
• Now it's also possible to use the simple HTTP Basic Authentication. If you run the REST API you really do not want to allow the access for everybody, especially for YaST which runs as root. Now you can set the credentials via YUI_AUTH_USER and YUI_AUTH_PASSWD environment variables.
  
  • This means that only one pair user/password can be set
  • ⚠️ Warning ⚠️ The password is still transferred in clear text! Use that feature only in a trusted network! Anybody on the way between the client and the REST API server can read that. To make it really secure we need to enable the SSL encryption.

Implementation Details

This is the complete list of the changes:

• https://github.com/libyui/libyui-ncurses/pull/77
• https://github.com/libyui/libyui/pull/143
• https://github.com/libyui/libyui-rest-api/pull/2
• https://github.com/libyui/libyui-ncurses-rest-api/pull/1
• https://github.com/libyui/libyui-qt-rest-api/pull/1
• https://github.com/libyui/ci-libyui-container/pull/1

Hackweek 17 Update

During Hackweek 17 I did these improvements to the project:

• Moved the HTTP server handling, now works better and it not blocked by UI calls
• Added support for the text mode (ncurses Ui)
• Added more details in the response, supports more UI actions
• Built testing packages for openSUSE Leap 15.0

See more details in the blog post (more examples) and in the Git repository (how to install the testing packages or how to patch the openSUSE Leap 15.0 installer).

=================================================================================================

Remote UI Control

This is actually a continuation of my previous Hackweek project YaST Integration Tests Using Cucumber. That project worked but in the end it turned out to be too big step forward. And was bound to one specific testing framework (cucumber).

Better Approach - REST API

So I was thinking about it how to make it more generic and my idea is: provide a REST API (using HTTP + JSON) for querying the UI and sending the actions.

API Examples

Here are some examples how the API could look. They are not final, it is just a proposal for starting the discussion:

Read (GET):

# dump whole dialog content
curl http://localhost:<port>/dialog
# dump only specific widget(s):
# by internal ID
curl http://localhost:<port>/widget?id=`checkbox
# by label (as displayed = translated!)
curl http://localhost:<port>/widget?label=Enabled
# by type (e.g. check all check box states)
curl http://localhost:<port>/widget?type=YCheckBox

Write, run actions (POST):

# by label (similar to read above)
curl -X POST http://localhost:<port>/widget?label=Enabled&checked=true
curl -X POST http://localhost:<port>/widget?label=Next&action=click

The read output could look like:

{
    "class": "YDialog",
    "children": [
        {
            "class": "YWizard",
            "id": "`wizard",
            "children": [
                 ...
            ]
         }
     ]
}

Advantages

• Not specific to any testing framework, you can write wrappers for RSpec, Cucumber,... adapting the openQA should be easier than adding the Cucumber support
• HTTP + JSON is easy to process (even from shell using curl and jq)
• Can be used remotely when running inside a virtual machine (important for openQA), the only complication is network-less machine, in that case the HTTP requests could be sent from another console via a curl command locally
• Allows using HTTP Basic/Digest Auth for authentication
• Switching to secure HTTPS should be relatively easy, then you could send your root password securely over network

Implementation Details

• Use GNU libmicrohttpd - embedded HTTP server, a C library
• Use Boost Property Tree - supports JSON parsing and serializing, a C++ library

Existing Proof of Concept

I already have a proof of concept code:

• Compile libyui from my fork, use the http_server branch
• Compile libyui-qt from my fork, use the http_server branch
• Recompile/reinstall the yast2-ycp-ui-bindings package (because of the ABI changes in libyui)
• Run YUI_HTTP_PORT=8888 yast2 <module> as root
• Run curl http://localhost:8888, you will see the dump of the current dialog in the JSON format (not complete yet, some values might be missing)

It does not support the paths mentioned in the proposal above or POST actions yet, but at least it shows that embedding an HTTP server and processing JSON is not that difficult in C++ as it could look at the first sight.

TODO

• Propose the REST API endpoints and parameters
• Improve the JSON dumper
• Implement the proposed API paths
• Error handling (widget not found, action not possible)
• Make the feature optional (e.g. enable/disable by a cmake option, users outside (open)SUSE might not be interested in this feature, it also adds some build and runtime dependencies)
• Implement the support also in the ncurses UI
• Implement support also for the packager widget (libyui-qt-pkg and libyui-ncurses-pkg)
• Write some simple wrappers/helpers for RSpec or Cucumber (see the previous project for the Cucumber step definitions)
• Document the API
• As this is a generic libyui solution add also some plain C++ examples (to demonstrate the usage outside YaST)
• Announce the feature so it can be really used

Related Project

There is the Make YaST Testing Independent of Keyboard Shortcuts hackweek project which is a bit related to this project. It's advantage is that it is simpler and can be possibly easier integrated into the current openQA.

=================================================================================================