Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

busybox: enable testcase logging #69

Merged
merged 1 commit into from
Nov 16, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions busybox/19-enable-testcase-logging.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
--- busybox-1.27.2/testsuite/runtest 2023-10-27 17:17:52.404370639 +0200
+++ busybox-1.27.2.phoenix/testsuite/runtest 2023-10-27 17:17:16.635573994 +0200
@@ -40,7 +40,7 @@
mkdir -p ".tmpdir.$applet"
cd ".tmpdir.$applet" || return 1

-# echo "Running testcase $testcase"
+ echo "---Testcase '$testname' starting---"
d="$tsdir" \
sh -x -e "$testcase" >"$testname.stdout.txt" 2>&1 || status=$?
if [ $status -ne 0 ]; then

--- busybox-1.27.2/testsuite/testing.sh 2023-10-27 17:17:52.404370639 +0200
+++ busybox-1.27.2.phoenix/testsuite/testing.sh 2023-10-27 17:17:16.635573994 +0200
@@ -87,7 +87,7 @@

$ECHO -ne "$3" > expected
$ECHO -ne "$4" > input
- [ -z "$VERBOSE" ] || echo ======================
+ echo "---Testcase '$NAME' starting---"
[ -z "$VERBOSE" ] || echo "echo -ne '$4' >input"
[ -z "$VERBOSE" ] || echo "echo -ne '$5' | $2"
$ECHO -ne "$5" | eval "$2" > actual
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that there are also other places where test cases start:
Screenshot from 2023-11-08 12-35-25

There you also don't know when you should start parsing output.

Please correct it, run all the tests and make sure that all the output is formatted correctly.

Copy link
Contributor Author

@mateusz-bloch mateusz-bloch Nov 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it turns out that in the new style of BusyBox tests, some may include checks within the test itself. Currently, this is only the case for bunzip2.tests and bzcat.tests.

After the recent changes:
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, one more nitpick: maybe it would be good to do ---Testcase 'testcase name' starting---

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean that?
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it the same?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that what it was about?:
image

I didn't understand, previously I changed only that the testcase was printed and not the entire path in one case of printning logs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, exactly


--- busybox-1.27.2/testsuite/bunzip2.tests 2023-10-27 17:17:52.404370639 +0200
+++ busybox-1.27.2.phoenix/testsuite/bunzip2.tests 2023-10-27 17:17:16.635573994 +0200
@@ -488,6 +488,7 @@
}

check() {
+ echo "---Testcase '$1' starting---"
eval $2 >t_actual 2>&1
if $ECHO -ne "$expected" | cmp - t_actual; then
echo "PASS: $1"
@@ -535,6 +536,7 @@

# This test is only for bunzip2
if test "${0##*/}" = "bunzip2.tests"; then
+ echo "---Testcase 'test_bz2 file' starting---"
if test1_bz2 | ${bb}bunzip2 >/dev/null \
&& test "`test1_bz2 | ${bb}bunzip2 | md5sum`" = "61bbeee4be9c6f110a71447f584fda7b -"
then
@@ -544,6 +546,7 @@
FAILCOUNT=$((FAILCOUNT + 1))
fi

+ echo "---Testcase 'pbzip_4m_zeros file' starting---"
if pbzip_4m_zeros | ${bb}bunzip2 >/dev/null \
&& test "`pbzip_4m_zeros | ${bb}bunzip2 | md5sum`" = "b5cfa9d6c8febd618f91ac2843d50a1c -"
then

--- busybox-1.27.2/testsuite/bzcat.tests 2023-10-27 17:17:52.404370639 +0200
+++ busybox-1.27.2.phoenix/testsuite/bzcat.tests 2023-10-27 17:17:16.635573994 +0200
@@ -40,6 +40,7 @@
}

check() {
+ echo "---Testcase '$1' starting---"
eval $2 >t_actual 2>&1
if $ECHO -ne "$expected" | cmp - t_actual; then
echo "PASS: $1"
Loading