How to do go test coverage for system testing

Hello,

I’d like to ask how to do go test coverage for system level testing. I read several go test coverage article and all those are talking about generate report when we have xx.go file
and associated xx_test.go file under same package.

I now have all my functional testing file located in one folder which is not specifically test
one particular package but when test case run, it will execute the binary or go file, how to
do test coverage in this case? The situation is similar as in C++, we instrument a binary file
via bullseye and run functional testing to see how is our functional test case coverage

The articles you mention describe Go’s unit testing system where each test usually focuses on one specific function of the package to be tested.

I think that unit testing is where code coverage analysis makes sense; on the other hand, when doing a system test with a ready-to-use binary, you would rather want to test the services that this binary provides to the outside (in other words, you do a real black box testing to verify if the binary works as documented). I don’t think that code coverage analysis is relevant at this level of testing; it should have been done during unit testing already.

3 Likes

Hi christophberger,

Thanks a lot for the reply. I would say when say unit testing is where code coverage analysis make sense might apply well if the system is relatively small and new? For a large system which might be also relatively old, many issues would be captured at integration level rather than at unit testing level so functional/system level testing usually are the main ones people rely on to capture the problem. There are tools in C++ which does the code coverage for system level testing which people use a lot.

Regardless of the previous discussion, I would like to understand whether Go has such tool or effective way to do the code coverage for this type of system/functional testing?

Maybe if you formally write your system tests as unit tests, like, for example,

// file: yourapp_test.go

package main

func TestSystem() {
    // run all the system tests here
}

then I believe you can use the coverpkg flag to enable coverage on all the packages you want to analyze. (Without that, you would only analyze the main package.) For example,

go test -run System -coverpkg=pkg1,pkg2,pkg3,... -coverprofile=cover.out
go tool cover -html=cover.out

(If all packages to be covered reside beneath the current dir, -coverpkg=./... should also work.)

This is just a rough idea but maybe you can take it as a starting point.

I see. Cool! I get my report with this way. Thanks very much for your time and help!

You’re welcome!

Hello christophberger,

I have another question. If I have tests located under different folders- unit and function test folders. Is there a way to combine the test coverage reports generated from tests of these two folders? As I’d like to see what is the overall coverage for all the tests, no matter they are function or unit tests.

Thanks,
Vera

Indeed it seems that coverprofile files can be merged to get the overall coverage. I have not tested this myself, but I found this open issue where the OP describes these manual steps:

If I manually construct a coverage.out file from multiple packages, it seems to work
fine:

  1. go test -coverprofile a.part ./a
  2. go test -coverprofile b.part ./b
  3. echo “mode: set” >coverage.out
  4. grep -h -v “mode: set” *.part >>coverage.out
  5. go tool cover -html=coverage.out

The very last comment (at this moment) of this issue announces a utility that merges coverage profiles, which might be what you are looking for.

Ah, cool! Let me try it out and see. Thanks very much for your help!

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.