Project Description

Python modules version updates in Factory.

Goal for this Hackweek

A mass update in devel:languages:python. Create few script that could help to automate this.

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 23


  • 7 months ago: kstreitova liked this project.
  • 8 months ago: hennevogel liked this project.
  • 8 months ago: dgarcia liked this project.
  • 8 months ago: mcalabkova started this project.
  • 8 months ago: pgajdos originated this project.

  • Comments

    • pgajdos
      8 months ago by pgajdos | Reply

      Currently, in home:pgajdos:python, there are ~550 (out of ~700) packages building sucessfuly and perhaps also with fewer rpmlint warnings (I will share the list below). This is with joint script and man power. It was meant as a demo how many packages can be updated quickly with not much going into python code.

      There are two questions: 1. what to do with the building packages in home:pgajdos:python processed with this round 2. what do do with the process in the future

      Both are obviously blocked by the question 'is the upstream changelog in rpm changelog neccessary?': Markéta: its somewhat sloppy Michal Švec: nice to have, but in this case can be overkill Matěj: nice to have, but for automatic updates can be tolerated Steven: while submitting to SLE, autobuild team can have objections Dirk Mueller: links the master changes file can be a problem

      I see also other issue: runtime dependencies. There will be few errors for sure, question is, how many. By the script, I have updated packages to contain requires what rpmlint suggests, but will not be 100%. If I remember correctly, there was something like 80 packages updated in this regard.

      Below is the list of rpmlint issues. Are there any we should really concentrate on (again, with script or with manual intervention)? [python-missing-require should be solved partially soon with the script; that are packages that now builds, so another script run is needed]

      And what with the process in the future? As I already expressed to some of you, I can imagine a project, e. g. devel:languages:python:{version-updates,pmmu,or whatever}, which would basically link all modules listed in Matěj's Monday summary of available updates. Script would link it there, checkout, try to do a version update and run some clean ups and automatic improvements and then check in. There would not be any manual work. The script could run e. g. monthly and always would regenerate whole project.

      Question then is, what could be automatically submitted from this project to devel:languages:python:*. I guess some metrics could be introduced, e. g. 0 errors, 0 warnings from rpmlint. But until changelog anabasis is not solved, I guess it is too soon to consider.

    • pgajdos
      8 months ago by pgajdos | Reply

      Some very rough statistics:

      There were 285 manual changes, from that 40 because of build failure in %files, 53 in %check, 10 in %install, 40 in %build, 136 in %prep and and few others or wrongly documented.

      There were ~3200 automatic changes. This can be misleading though, I tried to do small gradual changes like 'version update to', 'remove redundant python_module define', 'use %autosetup', 'run time deps', and so on.

      Some are not counted as they become broken meanwhile in the project, thus were removed.

    • pgajdos
      6 months ago by pgajdos | Reply

      A communication with Dirk Müller.

      Hi Dirk,

      so I continued in a bit different way, for me more logical than the one proposed by my surrounding. I have now finished first prototype of pmmu script, which run over night today. Even if you have stolen a lot testing material during last few weeks, I can probably safely say that in the current state of the script, it brings slightly more than 50% packages to successful build (plus it tries to lower rpmlint warnings and errors down). In current run, it tooks cca 4 hours to finish and it alone created home:pgajdos:python current content (thus, this can be regenerated in any time). The current result is:

      broken: 9 failed: 131 succeeded: 189 unresolvable: 29.

      To your remarks:

      1. I obviously considered two ways even before hackweek: A. regenerating the spec from scratch by something like py2pack and B. making small changes massively. B. won here.

      2. Shape of upstream changelogs in rpm changelog, aside it is a complete NONSENSE (and you know I spare upper caps for shouting) as a global policy, is not well defined, thus not scriptable. Everytime can come someone and tell 'this is too verbose' or 'this is too short' or 'this is too <whatever>' and he will have true and not true at the same time. Interesting is, that most people say the same as you .. I am not fan but we have to do as someone said we have to. Life is to short to deal with such stupid things.

      3. On the opposite side, correct versioned dependencies would be nice to have, but was obviously not the most important thing to solve. I am considering to borrow py2pack functionality to fix or regenerate dependencies. I am probably doing that partially already, because I am trying to add runtime dependencies which rpmlint reports to be missing.

      4. Automatic updates will of course never of such quality as the human one and if people will not accept it, then it is void. But perhaps the result can eventually be usable for my upcoming submissions.

    Similar Projects

    This project is one of its kind!