SUSE Hack Week: Reimplementation of "quilt setup"

# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

* Use asterisks
* for lists

This is an [example link](http://example.com/)

This is an ![example image](http://paste.opensuse.org/view/raw/68957446)

This is a user link @hans

This is a project link hw#some-cool-title

More Complex Markdown Help

Formatting Help

# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

* Use asterisks
* for lists

This is an [example link](http://example.com/)

This is an ![example image](http://paste.opensuse.org/view/raw/68957446)

This is a user link @hans

This is a project link hw#some-cool-title

More Complex Markdown Help

Formatting Help

Edit
Preview

The new backend for "quilt setup" is working, and the performance improvement is very nice. For example, the *kernel-default package* (which was the slowest one I was aware of) took *1 min 48 s* to setup before, while with the new --fast option, it only takes *25 s*. So I think the project can be called a success :-)

There are a few technical implementation details which still need to be sorted out, I'll discuss that with upstream. There are some possible improvements left on my to-do list as well.

It took a few tries to get things right. For example, I originally wanted to get rid of the md5sum step altogether, before I remembered that patches are typically passed to us through stdin, so we still need a checksum of all patch files in order to resolve the contents to a filename. I was still able to save time by skipping md5sum on archive files.

Another problem I hit is that my original implementation used "quilt import" to add each patch to the series file. This created a copy of each patch, instead of linking back to the original files in the source directory. That made it impossible to refresh the patches directly as "quilt setup" currently allows. I first worked around that by deleting the copies and linking back afterward, but that was both inefficient and fragile. The proper fix was to manually add the file to the series file instead of calling "quilt import". That required some work so that all the paths were correct both when rpmbuild is applying the patches and later when the working tree is available to the user. The trickiest part was to make it all work also when options *-d* and/or *--sourcedir* are used.

Finally, contrary to my original intention, "quilt push" isn't called on every patch. This turned out to be a bad idea from a performance point of view, because individual calls to "quilt push" are much slower than a single call to "quilt push -a". So, it is still up to the user to apply the patches with "quilt push", same as before.

Two user-visible differences exist between the original implementation of "quilt setup" and the new, faster one:

* Headers in the generated series file are incomplete. This prevents reusing the series file for a future call to "quilt setup". I think most users don't care about this feature though, so they won't notice. This could be fixed, but at the price of some of the performance gain, so I don't want to do it by default. Maybe this can be implemented as an option later, but that only really makes sense if we get rid of the original implementation of "quilt setup" (which my patches do not.)
* Patch failures are no longer reported, the user will see them on "quilt push". A great side effect of this is that all patches are added to the series even if one fails to apply in the middle. So I consider this change a feature, as this avoids the "quilt setup" / "rm -rf" cycle the user had to go through before when some patches do not apply.

Also a nice side effect for this project is that I came up with a few bug fixes, cleanups, and performance improvement patches for quilt along the way. 6 of them are already [upstream](http://git.savannah.gnu.org/cgit/quilt.git) and 6 more have been [posted for review](http://lists.nongnu.org/archive/html/quilt-dev/2014-10/index.html).

The new backend for "quilt setup" is working, and the performance improvement is very nice. For example, the kernel-default package (which was the slowest one I was aware of) took 1 min 48 s to setup before, while with the new --fast option, it only takes 25 s. So I think the project can be called a success :-)

There are a few technical implementation details which still need to be sorted out, I'll discuss that with upstream. There are some possible improvements left on my to-do list as well.

It took a few tries to get things right. For example, I originally wanted to get rid of the md5sum step altogether, before I remembered that patches are typically passed to us through stdin, so we still need a checksum of all patch files in order to resolve the contents to a filename. I was still able to save time by skipping md5sum on archive files.

Another problem I hit is that my original implementation used "quilt import" to add each patch to the series file. This created a copy of each patch, instead of linking back to the original files in the source directory. That made it impossible to refresh the patches directly as "quilt setup" currently allows. I first worked around that by deleting the copies and linking back afterward, but that was both inefficient and fragile. The proper fix was to manually add the file to the series file instead of calling "quilt import". That required some work so that all the paths were correct both when rpmbuild is applying the patches and later when the working tree is available to the user. The trickiest part was to make it all work also when options -d and/or --sourcedir are used.

Finally, contrary to my original intention, "quilt push" isn't called on every patch. This turned out to be a bad idea from a performance point of view, because individual calls to "quilt push" are much slower than a single call to "quilt push -a". So, it is still up to the user to apply the patches with "quilt push", same as before.

Two user-visible differences exist between the original implementation of "quilt setup" and the new, faster one:

Headers in the generated series file are incomplete. This prevents reusing the series file for a future call to "quilt setup". I think most users don't care about this feature though, so they won't notice. This could be fixed, but at the price of some of the performance gain, so I don't want to do it by default. Maybe this can be implemented as an option later, but that only really makes sense if we get rid of the original implementation of "quilt setup" (which my patches do not.)
Patch failures are no longer reported, the user will see them on "quilt push". A great side effect of this is that all patches are added to the series even if one fails to apply in the middle. So I consider this change a feature, as this avoids the "quilt setup" / "rm -rf" cycle the user had to go through before when some patches do not apply.

Also a nice side effect for this project is that I came up with a few bug fixes, cleanups, and performance improvement patches for quilt along the way. 6 of them are already upstream and 6 more have been posted for review.

# A First Level Header
## A Second Level Header

Use one asterisk to *emphasize*

Use two asterisks for **strong emphasis**

* Use asterisks
* for lists

This is an [example link](http://example.com/)

This is an ![example image](http://paste.opensuse.org/view/raw/68957446)

This is a user link @hans

This is a project link hw#some-cool-title

More Complex Markdown Help

Formatting Help

Looking for hackers with the skills:

This project is part of:

Activity

Comments

almost 10 years ago by matejcik | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by rwill | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by jdelvare | Reply

almost 10 years ago by jdelvare | Reply

over 9 years ago by jdelvare | Reply

about 9 years ago by jdelvare | Reply

Similar Projects