Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibibility with Continuous Integration services #48

Closed
Overdrivr opened this issue Feb 23, 2016 · 5 comments
Closed

Compatibibility with Continuous Integration services #48

Overdrivr opened this issue Feb 23, 2016 · 5 comments

Comments

@Overdrivr
Copy link

Is greatest compatible with CI services like appveyor ?

I belive it would be compliant as long as test errors are outputed on stderr instead of stdout. Looking at the code, this is not the case for now isn't it ?

@jibsen
Copy link
Contributor

jibsen commented Feb 23, 2016

I may be wrong, but I think many CI services expect xUnit style XML output in order to get fancy test result pages.

If you just want to run your unit tests as part of the build, then that works fine with greatest (success depends on the return value of the unit test).

As an example, I am using this AppVeyor script to run unit tests using greatest. If one of the builds fail, I can go into the console log and see the output from the unit test there.

@Overdrivr
Copy link
Author

How can Appveyor know that the test passed or failed unless errors are outputed on stderr instead of stdout ?

@jibsen Could you try to make a minor modification (in a new branch) on your program to voluntarily make the test fail to see if Appveyor reports a failed build ? Because I have the feeling that despite the test will fail Appveyor will report build success.

@jibsen
Copy link
Contributor

jibsen commented Feb 23, 2016

Sure, I pushed a branch that fails one of the unit tests (pack_numbers), the AppVeyor build is here.

As you can see the builds are failing, and if you go into one of the builds and check the bottom of the log you can see that the pack_numbers test is failing.

I think the test_script section (like the build_script) fails if the return value from the command indicates failure.

@Overdrivr
Copy link
Author

Ah yes you are right, the return code is enough. Awesome :)
Thank you @jibsen

@silentbicycle
Copy link
Owner

For what it's worth, greatest prints almost all output to GREATEST_STDOUT, which can be defined to stderr. It only defaults to stdout: https://github.com/silentbicycle/greatest/blob/master/greatest.h#L110

The only thing that explicitly prints to stdout is the greatest_printf_cb callback, which is used by ASSERT_EQUAL_T, ASSERT_STR(N)_EQ, and ASSERT_MEM_EQ. This will be fixed, but requires a major version bump because it's a breaking API change. #39

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants