Project Description

SUSE IT needs help from fellow geekos with release engineering skills to define the requirements, process, infrastructure, and tools for building an openSUSE-based distribution bundled with SUSE IT-supported application stack. The resulting OS build will be offered as a standard distribution for new SUSE employees in addition to the existing Operating System library.

Goal for this Hackweek

  1. Define requirements (and name) for the build.
  2. Selected features implemented for the MVP.
  3. Define a process for updating the image with the future releases of openSUSE.
  4. Identify and deploy required infrastructure and repositories.
  5. Produce an installable OS image that can be used by volunteers across the company.
  6. Document everything.
  7. Have a lot of fun in the process.

MVP requirements

  • the deployment do the end-user machine is unattended
  • machine has disk encrypted
  • once the machine is provisioned, it is added to asset DB (serial number + asset owner)
  • productivity tools are installed

Details are tracked here: https://en.opensuse.org/Portal:Leap:CSBRequirements

Teams bridge

This project is part of:

Hack Week 21

Activity

  • over 1 year ago: dsterba liked this project.
  • almost 2 years ago: wfrisch joined this project.
  • almost 2 years ago: toe joined this project.
  • almost 2 years ago: fbonazzi liked this project.
  • almost 2 years ago: crameleon liked this project.
  • almost 2 years ago: rkissiov joined this project.
  • almost 2 years ago: lkocman liked this project.
  • almost 2 years ago: fos liked this project.
  • almost 2 years ago: balmeida joined this project.
  • almost 2 years ago: wstephenson joined this project.
  • almost 2 years ago: wstephenson liked this project.
  • almost 2 years ago: mkoutny liked this project.
  • almost 2 years ago: QuaTran liked this project.
  • almost 2 years ago: IGonzalezSosa liked this project.
  • almost 2 years ago: bfilho liked this project.
  • almost 2 years ago: pperego liked this project.
  • almost 2 years ago: wfrisch liked this project.
  • almost 2 years ago: huizhizhao liked this project.
  • almost 2 years ago: sydsb liked this project.
  • almost 2 years ago: anthidote joined this project.
  • almost 2 years ago: anthidote liked this project.
  • almost 2 years ago: ancorgs liked this project.
  • almost 2 years ago: gkurel_suse joined this project.
  • almost 2 years ago: ddemaio joined this project.
  • almost 2 years ago: ddemaio liked this project.
  • All Activity

    Comments

    • lkocman
      almost 2 years ago by lkocman | Reply

      I think we're talking about a spin of openSUSE Leap 15.X with probably some autoyast config to meet requirements from the SUSE IT Policy (encrypted home etc) and some tool initialized on the first boot (Or perhaps SUMA could do that ?) that can pull additional "new" software and deploys configuration changes to clients as needed.

      The first-boot experience should probably connect to suse-guest (or other corporate wifi), pull VPN client and up2date configs, enable local printer discovery, certificates, enable multimedia and preferred browser that works with our web tools, preinstall tools used by SUSE Engineering (and other departments) teams, slack, Evolution with EWS config, etc.

      I know both RH with (CentOS 7.X based CSB) and IBM (with their open client) used to have 3rd party tools that were monitoring installed software, deploy new config changes etc. Perhaps that's something we could consider as well.

      Non-technical users had an option "not to have root password' any sort of management was done from the IT side. Knowing root password meant that you manage the host by yourself.

      The question is also if IT will want to have control about what updates go in as that would probably require some sort of RMT in between.

      • lrupp
        almost 2 years ago by lrupp | Reply

        Looks like you want to explore kiwi's feature set and OBS ability to constantly rebuild the image to me. add-emoji

        Beside the encryption, you might want to have a look at existing images (like the one of the openSUSE heroes) to get a starting point.

        ...I'm just wondering what you plan to do with the rest of the week add-emoji

        • lkocman
          almost 2 years ago by lkocman | Reply

          I personally wanted to play with container images for a quick home mirror enablement :-)

          • lrupp
            almost 2 years ago by lrupp | Reply

            JFYI: as we are testing our Salt Setup via Gitlab Runners, there are already some specialized Container images available, that you might want to check for some ideas.

        • lkocman
          almost 2 years ago by lkocman | Reply

          The problem is more with the management after the first-boot. There is a bunch of software in flatpaks etc. What about rancher tools with perhaps custom installers etc :-) There is unfortunatelly more to it than just kiwi tweaks.

      • vgrinco
        almost 2 years ago by vgrinco | Reply

        For encryption I'd make use of the device TPM chip for storing the encryption key, to not rely on people remembering passwords.

        We might also explore options to allow users to select the profile for their machine. E.G. sales, engineering, marketing, etc. and have ansible profiles packaged for each that would apply only specific set of rules, deploy the apps they use, create shortcuts relevant for that particular role, and so on. It would result in a cleaner experience.

        For the bootstrap process, we can prepare a PXE server in given VLANs, which will also guarantee some sort of connectivity for the machine that's being bootstrapped. During the bootstrap process, we might want to consider updating the asset management system with the machine serial number and owner login.

        Agree with the root/no root option, and a mechanism to reset the password to gain full system access for the ones who want to self-manage. We must assume that there will be part of the population who will rely on IT entirely to manage their OS (HR, for example, we want to have as diverse community as possible). Not sure if we'll be able to create a secure, user-initiated mechanism for remote support - we can add that to the backlog to be implemented at a latter stage.

        • lrupp
          almost 2 years ago by lrupp | Reply

          For PXE, please get in touch with our "Core Components" Team. They provide PXE + AutoYaST + some kind of Post-Boostrap process now since >15 years. It's called 'Orthos' and allows also to inventorize a machine automatically.

          For E&I, there are standard PXE services up and running in Nuremberg and Prague as well, including the latest development versions of our distribution. IMHO this should exist in Provo and Prague as well, but I'm not 100% sure there. EngInfra is providing this service.

      • bmwiedemann
        almost 2 years ago by bmwiedemann | Reply

        enable multimedia

        depends on the openh264 work with Cisco, if we want something that can pass legal review.

        Or we need to convince Microsoft & friends to re-design their tools to use open codecs such as VP9/webm or AV1 in Teams.

        We certainly do not want to include those 3rd party Packman repos.

    • lkocman
      almost 2 years ago by lkocman | Reply

      We already build a Corporate Standard build for Datto in OBS (it's Ubuntu based but the requirements would be similar), we might want to add that to our starting points next to image for Heroes.

      https://mysuse.sharepoint.com/sites/OpenSourceCommunityCitizens/SitePages/Past-Events-and-meetings.aspx

    • anthidote
      almost 2 years ago by anthidote | Reply

      I was helping a new joiner who made the masochistic choice of a Thinkpad with an i9 and Intel Onboard GFX + Dedicated NVIDIA GFX. It does not well work on Tumbleweed. It works ok with Ubuntu, so I told him to stick with it and maybe keep on openSUSE as dual boot if he's interested in hacking it.

      Perhaps the most important part of this would be to have IT "certify" or "reccomend" certain laptops for openSUSE. Once a new joiner has made a hw choice like that, there's very little we can realisitically do.

      The same Thinkpad model with a Ryzen + Vega GFX is much, much more stable with TW. In fact it works flawlessly.

      I can share a pdf guide, that I am building for new joiners. Perhaps there are some things that could be used for the image. One thing that could be useful is preloading it with SUSE CA certificates. That would make accessing the LDAP directory and setting up a thunderbird addressbook a bit easier.

      • anthidote
      • pperego
        almost 2 years ago by pperego | Reply

        It seems my setup and yes TW works very bad. Today I had panic moments when zypper dup crashed and system was in a corrupted state after an update. I had the same setup: thinkpad i9, onboard chard + nvidia T100 quadro

    • vgrinco
      almost 2 years ago by vgrinco | Reply

      We could also consider leapmicro[1]. Might be cool to offer a system unbreakable by updates. Heard there was a desktop spin for it.

      [1] https://get.opensuse.org/leapmicro/5.2/

      • vgrinco
      • vgrinco
        almost 2 years ago by vgrinco | Reply

        Leapmicro doesn't have a desktop manager - so no go. MicroOS is beta, and worked with moderate success - plus it's tumbleweed based. Sounds like a cool tiny os, but is not stable enough to be used in prod. Am working on leape 15.4 RC now -so far the best experience.

    • lkocman
      almost 2 years ago by lkocman | Reply

      Quick update

      I do currently experiment here https://build.opensuse.org/project/show/home:lkocman:suse-it-infra

      It's based on GNOME Live image package set, however I'm aiming for selfinstall media with gnome-initial-setup for firstboot

      I'd like to submit Proof Of Concept to https://build.opensuse.org/project/show/home:mpiala:suse-it-infra quite soon

      Official space where we'll build images long term is here (SUSE IT has admin permissions) https://build.opensuse.org/project/show/isv:SUSE:suse-it-infra

    • lkocman
      almost 2 years ago by lkocman | Reply

      Right now a simple task for volunteer could be to fix the rename in kiwipostrun (see project description, it contains the build error) More complext task will be to decide where to take teams, slack, element (irc/openSUSE) from

    • lnussel
      almost 2 years ago by lnussel | Reply

      We are the actual OS vendor, what's wrong with the official images we already produce? :-) We can basically change all aspects of them to suite our needs. Forking and maintaining a specific image in OBS over time is a lot of effort. Maybe SUSE specifics aren't so specific and could be integrated in a way that makes it possible to be reused by other companies? Upstream first is always a good idea :-)

      For example, IIRC the fedora installer asks whether the installation at hand is a corporate one and offers AD integration then. That's probably not exactly what we need here. Instead the installer could offer something similar to the community repos dialog, allowing to specify a custom domain name. Ie one would enter suse.com, the installer downloads a config file from there that tells the installer eg what VPN technology is used, what extra repos or packages are needed or whether eg full disk encryption should be enabled by default.

      • lkocman
        almost 2 years ago by lkocman | Reply

        We want self-install with first-boot and mdm integration. We can make spin of official images + use non-oss repo for everything else. We'll still need to store vpn configs and similar somewhere (it repo)

    • wstephenson
      almost 2 years ago by wstephenson | Reply

      Is there a way for an extension (once activated early in the installation) to hook in to the installer and add the features we require to the base openSUSE images, as Ludwig suggests? Or can this be done with the venerable AutoYast?

    • lkocman
      almost 2 years ago by lkocman | Reply

      do we have the (open)vpn configuration packaged somewhere?

    • gkurel_suse
      almost 2 years ago by gkurel_suse | Reply

      Here's the short description of the corporate openSUSE image seen by SUSE IT. Please note that this is still a draft, based on comments posted in this project along with the security and business requirements. We believe it is a good starting point. The content is kept and updated on Confluence. File below is the latest snapshot in PDF format.

      Link: openSUSE Corporate Image

    • lkocman
      almost 2 years ago by lkocman | Reply

      I did converted article to wiki, so it's accessible outside SUSE network, and also in case that confluence would be again not available for a week or so https://en.opensuse.org/Portal:Leap:CSBRequirements

    • rkissiov
    • lkocman
      almost 2 years ago by lkocman | Reply

      The call is open to everyone, feel free to connect. The meeting is still open, we'll most likely keep it open for others to join for rest of the day.

    • mpiala
      almost 2 years ago by mpiala | Reply

      Day 1 kick-off meeting

      Attendees: lkocman, vgrinco, mpiala, rkissiov, wfrisch, toepkes, gkurel

      Initially defined streams: A. deployment B. packaging C. MDM enrollment (former configuration management + asset management integration)

      Actions for today: A1: gnome-initial-setup is enabled, disk encrypted by default password (later can be changed) - lkocman A2: create dummy rpm from IBS B1: deploy package for VPN configuration (for corp + Eng VPN; openconnect + OpenVPN are dependency for this package, should be already in Leap) - wfrisch B2: deploy package for MDM enrollment (/opt/vmware/ws1-hub/bin/ws1HubUtil enroll -s ds1678.awmdm.com -g m3115296 -u openSUSE -p openSUSE ; workspaceone-intelligent-hub is dependency for this package) - gkurel C1: enroll device into our Workspace One tenant (currently default credentials, goal is to use Okta)

      • mpiala
        almost 2 years ago by mpiala | Reply

        better readable format:

        Initially defined streams: A. deployment B. packaging C. MDM enrollment (former configuration management + asset management integration)

        Actions for today: A1: gnome-initial-setup is enabled, disk encrypted by default password (later can be changed) - lkocman A2: create dummy rpm from IBS - mpiala B1: deploy package for VPN configuration (for corp + Eng VPN; openconnect + OpenVPN are dependency for this package, should be already in Leap) - wfrisch B2: deploy package for MDM enrollment (/opt/vmware/ws1-hub/bin/ws1HubUtil enroll -s ds1678.awmdm.com -g m3115296 -u openSUSE -p openSUSE ; workspaceone-intelligent-hub is dependency for this package) - gkurel C1: enroll device into our Workspace One tenant (currently default credentials, goal is to use Okta) - rkissiov

    • mpiala
      almost 2 years ago by mpiala | Reply

      Day 1 summary:

      Deployment

      • A1: build has enabled gnome-initial-setup + disk encryption. Still failing, lkocman will continue on it

      Packaging

      • B1: Deploy package for openconnect - done by wfrisch. Deploy package for OpenVPN - item for tomorrow for gkurel/wfrisch
      • B2: Deploy package for MDM enrollment - done by vgrinco, using only default credentials. Once C1 is completed, we will change enrollment to use Okta.

      MDM enrollment

      • C1: Enrollment working with default credentials works. Call with Workspace One happened (Okta authentication), we are expecting some answers tomorrow.

      Thank you guys for a nice work today! Let's have a sync tomorrow at 9:30 CEST

    • lkocman
      almost 2 years ago by lkocman | Reply

      I have encrypted image by using "suseit" password https://download.opensuse.org/repositories/home:/lkocman:/suse-it-infra/images/

      Together with gnome-branding which should install basic flatpaks after gnome-initial-setup (thank to Richard's work in MicroOS).

      However, I'm now debugging why my image end up with gdm, instead of gnome-intial-setup (I suppose systems service conflicts).

    • mpiala
      over 1 year ago by mpiala | Reply

      Day 2

      Deployment

      • The encryption works. We might consider to disable the encryption for few days, to lower the build time
      • Some issue with GDM, lkocman working on it

      Packaging

      • OpenVPN package in progress by wfrisch
      • Teams, Slack to be installed using flatpack - gkurel is working it

      MDM enrollment

      • Currently, we can enroll (all) linux machines under one user and to move them to different user is not working. Waiting for replies from Workspace One, how to properly integrate it with Okta.

    • lkocman
      over 1 year ago by lkocman | Reply

      So we do have an btrfs encrypted Leap 15.4 self-install image that triggers gnome-initial-setup. Once the first-boot is done gnome-branding-Leap's mod-firstboot (Big thanks to Richard) will pull remaining software as flatpaks. Deployment on real HW was under 3 minutes. VM in file/gnome-boxes more likely 5-7.

      https://ibb.co/yPHvV2g (see pic from my tuxedo laptop)

      • Issue1: I had to disable ignition as it was failing the boot (seems like caused by some update in SLE Micro 5.3)

      • Issue2: We've had to fork gdm, as the gnome-initial-setup feature is disabled in SLE/Leap.

      • Issue3: I see that the gnome-initial-setup is either buggy (e.g. nothing happens if you click on connect to wifi) or there is something wrong with the modified GNOME livecd base that was used for the self-install image and seems like no user is created after finishing the first-boot wizard. This currently blocks the gnome-branding mod-firstboot and any other progress.

      I could use somebody who could debug it and fix the remaining issues. Unfortunately the latest version from the factory will not build on top of 15.4

      The image is built by these two files (livecd-openSUSE as I started with GNOME livecd as a base) https://build.opensuse.org/package/view_file/home:lkocman:suse-it-infra/livecd-openSUSE/selfinstall-leap-gnome-csb.kiwi?expand=1 https://build.opensuse.org/package/view_file/home:lkocman:suse-it-infra/livecd-openSUSE/config.sh?expand=1

      I did create https://build.opensuse.org/package/show/isv:SUSE:suse-it-infra/suse-csb-release, that will enable the isv:SUSE:suse-it-infra repo and adds recommends for suse-it software (OpenVPN configs etc). This one could use some love as well.

    • lkocman
      over 1 year ago by lkocman | Reply

      I did fork a newer version of gnome-initial-setup from factory (r 14) which doesn't seem to have sle patches. Let's see if that works better

    • lkocman
      over 1 year ago by lkocman | Reply

      OK, so I've fixed the gnome-initial-setup it was really stupid issue, as we were building the image with readonly root (copied from livecd). I did notice this before, but simply remounting it as r/w before reastarting display-manager did not fix the issue. Full boot with /r/w / simply worked. I got to the point where I had Leap 15.4 self-installed image with GNOME, that triggered gnome-branding-Leap which installed remaining software as flatpaks. Fixed in https://build.opensuse.org/package/rdiff/home:lkocman:suse-it-infra/livecd-openSUSE?linkrev=base&rev=36

      The current image has ignition-dracut enabled, so we can set a root password, otherwise, it's a pain. Seems like self-install built itself erases root password, even when it was originally set in kiwi.

    • lkocman
      over 1 year ago by lkocman | Reply

      Oky so we did syncup with mpiala, put our changes together and all was submitted and accepted into https://build.opensuse.org/project/show/isv:SUSE:suse-it-infra

      My remaining next steps for this hackweek is to ensure that ignition/combustion can change root password (otherwise you have to use rescue media or similar), as the initial build on Monday was failing on ignition related things.

    • lkocman
      over 1 year ago by lkocman | Reply

      The ideal next steps, in general, would be to make self-install Leap media with updated gnome-initial-setup (and gdm without SLEs patches to disable gnome-initial-setup) part of official Leap 15.4 or 15.5 :Images.

      We'd still have to figure out a way for suse-it to manage the gnome-initial-setup and gnome-branding (mod-firstboot), without rebuilding the image, in any case at least we'd have a fork of the official image to overwrite defaults).

    • lkocman
      over 1 year ago by lkocman | Reply

      I've played a bit with combustion (tested in gnome-boxes) and depends if you first-boot with "ignition usb device" or not, the it either has control over root or not https://github.com/SUSE/suse-csb-release/pull/2

      The changeLuksKey for btrfs doesn't seem to work, so that will require further investigation.

      Otherwise, the custom sudoers, where the user has to enter the root password works ... keep in mind that as long as the person knows the luks password, they can boot with init=/bin/sh and get a root shell...

    • lkocman
      over 1 year ago by lkocman | Reply

      Regarding root access, datto has a secondary "ansible" user, with IT pubkey on it. The status of machine is being reported even if it's connected through VPN.

      Perhaps that's an approach we could consider as well.

    • lkocman
      over 1 year ago by lkocman | Reply

      Just additional info, the user is datto-ansible, we could be using suse-ansible (or -salt, I've heard that we want to prefer Ansible over salt in Alp, not sure what SUSE IT uses ...). Also, the user has to be a system user (have uid < 1000), so it's not displayed in GDM, and gnome-initial-setup starts.

    • cdywan
      about 1 year ago by cdywan | Reply

      Is this project being continued in Hackweek 22?

    Similar Projects

    A CLI for Harvester by mohamed.belgaied

    [comment]: # Harvester does not officially come...


    Tumbleweed support for Raspberry Pi 4 with Quad SATA HAT by jbaier_cz

    [comment]: # (Please use the project descriptio...


    LSP server for SPEC files by mcepl

    Has anybody ever heard about RPM Spec LSP serve...


    A CLI for Harvester by mohamed.belgaied

    [comment]: # Harvester does not officially come...


    Investigate zypper/openSUSE repository refresh optimisations by dirkmueller

    [comment]: # (Please use the project descriptio...


    Switch software-o-o to parse repomd data by hennevogel

    Currently software.opensuse.org search is using...


    openSUSE on ZoL from OpenZFS project by jkohoutek

    Idea is to have SUSE system with OpenZFS as roo...


    SSH key distribution solution by vgrinco

    Project Description

    SSH key distribution so...


    Update Haskell ecosystem in Tumbleweed to GHC-9.6.x by psimons

    [comment]: # (Please use the project descriptio...


    A set of utilities to produce a "from scratch" OCI/Docker container using Opensuse/SLE rpms by ldragon

    [comment]: # (Please use the project descriptio...


    openSUSE on ZoL from OpenZFS project by jkohoutek

    Idea is to have SUSE system with OpenZFS as roo...


    LSP server for SPEC files by mcepl

    Has anybody ever heard about RPM Spec LSP serve...