.png){kind=small}
L3: Improve crash-setup, develop a core-setup
The current crash-setup source is located here.
Pretty much is working nicely but it doesn't care for the debug source making it impossible to use the crash> gdb list * command right away. This is bug 997558 which should be worked on.
A further script would be really useful to build up the analysis environment on l3slave for core dumps of crashed user-space applications. The MF Open Enterprise Server (OES) has a tool called NetIQ GetCore Utility which prepares such an environment with files opencore.sh, opencore.ini, the binaries of the executable and libs, and the core dump itself. The files it generates can be used as a base and prove that it is possible to build up a core dump analysis environment decoupled from the host system.
What it is missing is setting up the debug symbols and the debug source as well. I've done that manually already. So this is possible and should be automated.
Both these topics are efficiency improvement tasks for the L3 work.
This project is part of:
Hack Week 17
Activity
Comments
-
over 6 years ago by sparschauer | Reply
The core-setup project should not depend on l3slave and should work with openSUSE for non-employees as well. It should have different modes for:
- analysis on the system which collected the dump:
- auto-get debuginfo and debugsource packages with zypper - packages are latest? (recommended for reproducible segfaults)
- include paths to custom code like e.g. scanmem which I compile from source as an upstream maintainer
- analysis on another system
- auto-get and extract RPMs from SUSE servers like e.g.
http://download.opensuse.org/update/leap/42.3/oss/x86_64/,http://download.opensuse.org/debug/update/leap/42.3/oss/x86_64/
- auto-get and extract RPMs from SUSE servers like e.g.
- SLES
- get packages from NFS mounts
- get packages from *.suse.de servers
- openSUSE
- get packages from download.opensuse.org or mirrors
Key features:
- read coredump, detect crashed process and libs
- convert build-id to package name + version with the help of SUSE repos
- maybe use rpm.txt from supportconfig if exact package info cannot be gathered from the dump
- get and extract the required binary, debuginfo, and debugsource packages
- build up opencore.sh and opencore.ini - maybe with support for custom settings
Initial tasks:
- check for, evaluate, and analyze similar tools
- analyze coredump structure - What is included and can be used?
Main goals:
- prevent gdb from picking up wrong code/source/debuginfo or wrong packages
- take needed code parts from other FOSS projects to avoid duplicate efforts
- analysis on the system which collected the dump:
-
over 6 years ago by sparschauer | Reply
For
crash-setup -d vmcore, crash-setup uses kdumpid which uses libkdumpfile.Example output of
kdumpid:Format: compressed kdump
Arch: x86_64
Version: 4.4.138-94.39-default-
over 6 years ago by sparschauer | Reply
crash-setupuses kernel-source.git to convert the version into a git tag, then to a commit, and then to the oldest git branch containing it. This step is extremely slow as it is done by comparing the commit id in the git log of all the configured git branches from old to new. The SLE release is gathered from this. I'm sure this can be sped up.Example: 4.4.138-94.39 -> rpm-4.4.138-94.39 -> baa07f9df91b -> SLE12-SP3
-
over 6 years ago by sparschauer | Reply
git describeis even slower. Only configuring major.minor per branch would help.
-
-
-
over 6 years ago by sparschauer | Reply
I've introduced a new crash-setup option '-s' or '--source' to get the debugsource package automatically as well. With the crash
cdcommand I can get to the right directory so that thegdb listcommand will work. The idea is to create a fileopencrash.shcontainingcrash -i ./opencrash.ini vmlinux.gz vmlinux.debug vmcoreand a fileopencrash.inicontaining e.g.cd ./root/usr/src/debug/kernel-default-4.4.138/linux-4.4/kernelautomatically.-
over 6 years ago by sparschauer | Reply
Autocreation of
opencrash.sh/.iniimplemented. Since SLE12 there are different paths to source files. Socd rootis used for SLE11 and before.Commits can be found here.
-
-
-
over 6 years ago by sparschauer | Reply
I worked on the core-setup already.
gdb -ex "quit" -c corecan get the build id of the main program running. Then we can look intorepodata/*primary.xml.gzand search for it. The next "location href" line above shows the name of the debuginfo package.Example:
$ ./core-setup ./core.hald.2341
main build id from core.hald.2341: da45f02baff8ef519840d5a17fd15926f2c802e2
looking up main build id...
x86_64/hal-debuginfo-0.5.12-23.76.1.x86_64.rpmNow we can remove "-debuginfo" from the RPM name and get the standard package name "hal-0.5.12-23.76.1.x86_64.rpm". With these two packages it is possible to run gdb regularly and to gather the build ids of the libraries.
-
over 6 years ago by sparschauer | Reply
I made a lot of progress with core-setup.
It handles the hald example core dump pretty well already. The CPU arch, the crashed executable, and the main build id are gathered from gdb -batch -ex "info auxv" -c core. Besides the packages for the crashed executable, the debug and standard packages for 20 of 22 libraries are fetched and extracted.Unfortunately, just removing "-debuginfo" from the package name is not enough. There are more standard packages required for the same debuginfo package. The biggest issue is that the build id is not stored in the coredump and only reflects the standard packages of the host system. So the wrong package versions are gathered.
Looks like I have to parse rpm.txt from the supportconfig for the right versions and check which package provides the library file which has to be extracted first and then make sure that really the correct files are picked up by gdb.
-
over 6 years ago by sparschauer | Reply
At least this method works pretty well with the
NetIQ GetCore Utilitywhich gathers all loaded ELF files from the crashed system. This way the correct build ids and therefore debuginfo packages are picked up.
-
-
-
almost 6 years ago by sparschauer | Reply
@alnovak ignored my merge request for crash-setup. Using the change as custom enhancement if nobody else is interested in it. The core-setup idea has proven working when I picked up the rpms from the supportconfig
rpm.txtmanually for L3 bug 1105883.
Reply to sparschauer
I made a lot of progress with core-setup. It handles the hald example core dump pretty well already. The CPU arch, the crashed executable, and the main build id are gathered from gdb -batch -ex "info auxv" -c core. Besides the packages for the crashed executable, the debug and standard packages for 20 of 22 libraries are fetched and extracted.
Unfortunately, just removing "-debuginfo" from the package name is not enough. There are more standard packages required for the same debuginfo package. The biggest issue is that the build id is not stored in the coredump and only reflects the standard packages of the host system. So the wrong package versions are gathered.
Looks like I have to parse rpm.txt from the supportconfig for the right versions and check which package provides the library file which has to be extracted first and then make sure that really the correct files are picked up by gdb.
# A First Level Header ## A Second Level Header Use one asterisk to *emphasize* Use two asterisks for **strong emphasis** - Use hyphens - for unordereed - lists This is an [link to example.com](http://example.com/) This is an image  This is a user link @hans This is a project link hw#some-cool-title
Similar Projects
Port git-fixup to POSIX shell script and submit to git/git by mcepl
Description
https://github.com/keis/git-fixup is an exceedingly useful program, which I use daily, and I would love to every git user could bask in its awesomeness. Alas, it is a bash script, so it is not appropriate for the inclusion in git proper.
Goals
Port the script to plain POSIX shell and submit for consideration to git@vger.kernel.org
Resources
FizzBuzz OS by mssola
Project Description
FizzBuzz OS (or just fbos) is an idea I've had in order to better grasp the fundamentals of the low level of a RISC-V machine. In practice, I'd like to build a small Operating System kernel that is able to launch three processes: one that simply prints "Fizz", another that prints "Buzz", and the third which prints "FizzBuzz". These processes are unaware of each other and it's up to the kernel to schedule them by using the timer interrupts as given on openSBI (fizz on % 3 seconds, buzz on % 5 seconds, and fizzbuzz on % 15 seconds).
This kernel provides just one system call, write, which allows any program to pass the string to be written into stdout.
This project is free software and you can find it here.
Goal for this Hackweek
- Better understand the RISC-V SBI interface.
- Better understand RISC-V in privileged mode.
- Have fun.
Resources
Add a machine-readable output to dmidecode by jdelvare
Description
There have been repeated requests for a machine-friendly dmidecode output over the last decade. During Hack Week 19, 5 years ago, I prepared the code to support alternative output formats, but didn't have the time to go further. Last year, Jiri Hnidek from Red Hat Linux posted a proof-of-concept implementation to add JSON output support. This is a fairly large pull request which needs to be carefully reviewed and tested.
Goals
Review Jiri's work and provide constructive feedback. Merge the code if acceptable. Evaluate the costs and benefits of using a library such as json-c.