From 0800455fb6619c45f5313b3dbc74db88ec4922a4 Mon Sep 17 00:00:00 2001 From: Xi Ruoyao Date: Sun, 18 Feb 2024 15:34:58 +0800 Subject: [PATCH 1/2] Glibc: Re-organize test failures and mention how to rule out timed out tests Glibc tests occansionally fail due to a timeout because: 1. The hardware is slower than the developers expected. 2. Some tests use multiple or even all CPU cores internally, for e.g. with 8 active CPU cores we may end up running 8 tests (due to -j8) each of them uses 8 cores in the worst case, resulting a severe congestion. I'm almost sure nptl/tst-thread-affinity* are cases of 2. Let's document how to rule out the timed out tests instead of making the list of known failures longer and longer. --- chapter08/glibc.xml | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/chapter08/glibc.xml b/chapter08/glibc.xml index e8432fe19..10fccb3a9 100644 --- a/chapter08/glibc.xml +++ b/chapter08/glibc.xml @@ -163,34 +163,34 @@ esac is known to fail in the LFS chroot environment. - - Three nptl/tst-thread-affinity* - tests are known to fail. - - - - - - - The stdlib/tst-arc4random-thread - test is known to fail if the host kernel is relatively old. - - Some tests, for example - nss/tst-nss-files-hosts-multi, - are known to fail on relatively slow systems due to an internal - timeout. + nss/tst-nss-files-hosts-multi and + nptl/tst-thread-affinity* + are known to fail due to a timeout (especially when the system is + relatively slow and/or running the test suite with multiple + parallel make jobs). These tests can be identified with: + + + grep "Timed out" -l $(find -name \*.out) + + It's possible to re-run a single test with enlarged timeout + with + TIMEOUTFACTOR=<factor> + make test t=<test name>. + For example, TIMEOUTFACTOR=10 make test + t=nss/tst-nss-files-hosts-multi will re-run + nss/tst-nss-files-hosts-multi with ten times + the original timeout. Additionally, some tests may fail with a relatively old CPU - model or host kernel version. + model (for example + elf/tst-cpu-features-cpuinfo) or host kernel + version (for example + stdlib/tst-arc4random-thread). From 5f2ba8ce99b9e189615f9ae975e9ea1b902173de Mon Sep 17 00:00:00 2001 From: Xi Ruoyao Date: Sun, 18 Feb 2024 22:09:17 +0800 Subject: [PATCH 2/2] gcc: Decorate test failures, remove outdated comments, ... move comments for upstream PR links to correct location. BTW data-model-4.c is also in analyzer directory, so we can just say 7 analyzer tests. --- chapter08/gcc.xml | 44 +++++++++++++------------------------------- 1 file changed, 13 insertions(+), 31 deletions(-) diff --git a/chapter08/gcc.xml b/chapter08/gcc.xml index b97b0b8cc..66f803273 100644 --- a/chapter08/gcc.xml +++ b/chapter08/gcc.xml @@ -172,48 +172,30 @@ su tester -c "PATH=$PATH make -k check" url="&test-results;"/> and . - - - Eight gcc tests (out of over 185,000), data-model-4.c, pr56837.c, - and six "analyzer" tests are known to fail. + + Eight gcc tests (out of over 185,000): + pr56837.c + and seven tests in the analyzer + directory are known to fail. - One libstdc++ test (out of over 15000), copy.cc, is known to fail. + + One libstdc++ test (out of over 15000), copy.cc, is + known to fail. - For g++, 21 tests (out of approximately 250,000), 14 "AddressSanitizer*" - tests and 7 interception-malloc-test-1.C tests, are known to fail. + For g++, 21 tests (out of approximately 250,000): 14 + AddressSanitizer* + tests and 7 interception-malloc-test-1.C tests, are + known to fail. Additionally, several tests in the vect directory are known to fail if the hardware does not support AVX. - + A few unexpected failures cannot always be avoided. The GCC developers are usually aware of these issues, but have not resolved them yet. Unless the test results are vastly different from those at the above URL, it is safe to continue. - - Install the package: make install