Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runsuite_wrapper.pl is erroneously attributing test failures to i#1938 #2534

Open
xdje42 opened this issue Jan 14, 2025 · 3 comments
Open

runsuite_wrapper.pl is erroneously attributing test failures to i#1938 #2534

xdje42 opened this issue Jan 14, 2025 · 3 comments

Comments

@xdje42
Copy link
Contributor

xdje42 commented Jan 14, 2025

Describe the bug

The test wrapper runsuite_wrapper.pl apparently hardwires all expected failures as being due to i#1938.

ref: https://github.com/DynamoRIO/drmemory/blob/release_2.6.0/tests/runsuite_wrapper.pl#L281

To Reproduce

Submit a PR and study the ci-clang workflow test output.

Expected behavior

Each expected test failure should be attributed to the proper bug.
Otherwise developers have to spend unnecessary time auditing test failures to ensure new bugs aren't missed.

Screenshots or Pasted Text

From the ci-clang workflow here: #2532

2025-01-13T22:50:33.8979853Z ====> FAILURE in drmemory-dbg-32 <====
2025-01-13T22:50:33.8980200Z drmemory-dbg-32: 113 tests passed, **** 11 tests failed, but ignoring 6 for i1938: ****
2025-01-13T22:50:33.8980541Z    selfmod 
2025-01-13T22:50:33.8980685Z    syscalls_unix 
2025-01-13T22:50:33.8980844Z    clone 
2025-01-13T22:50:33.8980991Z    pthread_test 
2025-01-13T22:50:33.8981155Z    (ignore: i#1938)        wrap_cs2bug 
2025-01-13T22:50:33.8981366Z    (ignore: i#1938)        wrap_operators 
2025-01-13T22:50:33.8981584Z    (ignore: i#1938)        pcache-use 
2025-01-13T22:50:33.8981772Z    realloc 
2025-01-13T22:50:33.8981929Z    (ignore: i#1938)        app_suite 
2025-01-13T22:50:33.8982148Z    (ignore: i#1938)        fuzz_threads 
2025-01-13T22:50:33.8982359Z    (ignore: i#1938)        app_suite.pattern

Versions

HEAD as of commit f62daf1
OS: Test output reports Ubuntu 20.04.6

@derekbruening
Copy link
Contributor

You can see the actual issue in the comments in that file:

https://github.com/DynamoRIO/drmemory/blob/release_2.6.0/tests/runsuite_wrapper.pl#L203

              # TODO i#2342: These are hitting a DR encoding assert.  Maybe we
                # should just drop wrap_ support anyway.  Also xref i#1741.
                'wrap_malloc' => 1,
                'wrap_cs2bug' => 1,
                'wrap_operators' => 1,

#2342 is the culprit for the wrap_ tests in 32-bit debug.

@xdje42
Copy link
Contributor Author

xdje42 commented Jan 14, 2025

Re: reading comments in runsuite_wrapper.pl:
Indeed but developers should have be required to do all that digging, reading the test output should be sufficient.

@xdje42
Copy link
Contributor Author

xdje42 commented Jan 14, 2025

Data point re: #2342: I'm seeing a different failure for wrap_operators: #2535

derekbruening pushed a commit that referenced this issue Jan 15, 2025
…2526)

The definition of strcasestr is

char *strcasestr(const char *text, const char *pattern);

but drmemory uses

const char *strcasestr(const char *text, const char *pattern);

This causes build errors when both /usr/include/string.h and
common/utils.h are included in linux. The definition for strcasestr for
C has always not had the `const` in the result type (going back to its
original addition to glibc in 1997). Thus we're pretty safe in not
breaking anything on *nix. The other main use case of drmemory's
strcasestr is on Windows, which does not have strcasestr.

Tested:
$ cmake && make
Also, there are failures in the clang and x86 workflows, but comparing
the logs before/after
shows all failures pre-exist this patch, no new failures.

Here is the complete audit, all failures are pre-existing.

Recorded (errantly) as i#1938 failures:
  - wrap_operators: "internal crash" (Received SIGSEGV)
    - filed as #2535
  - wrap_cs2bug: CHECK_TRUNCATE_TYPE_sbyte(new_offs) (#2342)
- pcache-use: "Usage error: meta-instr faulted? must set translation
field and handle fault!" (#2202)
  - app_suite: vsyscall incorrect assumption (#2491)
- app_suite.pattern: assert failure: save->ignore_next_delete == 0
(premature deletion)
    - filed as i#2537
  - fuzz_threads: vsyscall incorrect assumption (#2491)

Note: The above are marked as expected failures but [runsuite_wrapper.pl
hardwires the addition of "i#1938" to test
output](https://github.com/DynamoRIO/drmemory/blob/master/tests/runsuite_wrapper.pl#L281).
#2534

Not currently recorded as expected failures (all #2491 - vsyscall
incorrect assumption):
  - selfmod
  - syscalls_unix
  - clone
  - pthread_test
  - realloc

Fixes #2522
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants