Having trouble testing SQLite functions

Getting Help Code Review
1

Hey, gophers! I’m writing because I need a substantial amount of help with my first Go project, good. Basically, it’s a mood tracker that interfaces with an SQLite database.

My problem: I’ve been working on sqlite_test.go, and seeing my tests break in different manners, depending on whether I use TestMain or not (I’ll come back to that later).

For starters, here’s sqlite.go — it’s the same in branches test1 and test2. Now, here’s me testing it with test1, in which each test is relatively isolated (if also messy).

=== RUN   TestInsertEvents
=== RUN   TestInsertEvents/2006-01-02_15:04:05+00:00
=== RUN   TestInsertEvents/2019-06-27_19:25:09-07:00
--- PASS: TestInsertEvents (0.00s)
    --- PASS: TestInsertEvents/2006-01-02_15:04:05+00:00 (0.00s)
    --- PASS: TestInsertEvents/2019-06-27_19:25:09-07:00 (0.00s)
=== RUN   TestEvents
=== RUN   TestEvents/reviewing_1_day_back
DEBU[0000] Getting all events starting from 2019-06-26.
DEBU[0000] Querying activities for event 1.
--- FAIL: TestEvents (0.00s)
    --- FAIL: TestEvents/reviewing_1_day_back (0.00s)
        sqlite_test.go:90: Error upon querying event: no such table: activities
FAIL
exit status 1
FAIL    github.com/Goorzhel/good        0.004s

Weird, innit? Maybe I could chalk this up to each test running in its own goroutine and…well, honestly, my mental model of what’s going on here isn’t honed enough to tell exactly what it is. But I’m damn sure an activities table is supposed to be there.

But hang on, it gets weirder. Here’s test2, wherein I use a TestMain.

=== RUN   TestInsertEvents/2006-01-02_15:04:05+00:00
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x61d31a]

goroutine 8 [running]:
testing.tRunner.func1(0xc00010e200)
        /usr/lib/go/src/testing/testing.go:830 +0x392
panic(0x754c60, 0xa09f00)
        /usr/lib/go/src/runtime/panic.go:522 +0x1b5
database/sql.(*DB).conn(0x0, 0x801b80, 0xc0000140a8, 0x1, 0x0, 0x0, 0x0)
        /usr/lib/go/src/database/sql/sql.go:1080 +0x3a
database/sql.(*DB).query(0x0, 0x801b80, 0xc0000140a8, 0xc00001e1b0, 0x29, 0x0, 0x0, 0x0, 0xa28101, 0x7f0f35936008, ...)
        /usr/lib/go/src/database/sql/sql.go:1513 +0x66
database/sql.(*DB).QueryContext(0x0, 0x801b80, 0xc0000140a8, 0xc00001e1b0, 0x29, 0x0, 0x0, 0x0, 0x40, 0xc00001e1b0, ...)
        /usr/lib/go/src/database/sql/sql.go:1495 +0xd1
database/sql.(*DB).QueryRowContext(...)
        /usr/lib/go/src/database/sql/sql.go:1596
database/sql.(*DB).QueryRow(0x0, 0xc00001e1b0, 0x29, 0x0, 0x0, 0x0, 0x29)
        /usr/lib/go/src/database/sql/sql.go:1607 +0x8d
github.com/Goorzhel/good.(*dbConn).IDByName(0xa264d8, 0x7a52a8, 0x5, 0x7a5302, 0x5, 0x539, 0x877c38, 0x9eb564)
        /home/ca/projects/good/sqlite.go:80 +0x1ed
github.com/Goorzhel/good.(*dbConn).InsertEvent(0xa264d8, 0xa0e840, 0x5d157c36, 0xc000048f98)
        /home/ca/projects/good/sqlite.go:159 +0x72
github.com/Goorzhel/good.testInsertEventFunc.func1(0xc00010e200)
        /home/ca/projects/good/sqlite_test.go:73 +0x37
testing.tRunner(0xc00010e200, 0xc00000e800)
        /usr/lib/go/src/testing/testing.go:865 +0xc0
created by testing.(*T).Run
        /usr/lib/go/src/testing/testing.go:916 +0x35a
exit status 2
FAIL    github.com/Goorzhel/good        0.005s

Whee! It segfaults! I knew good.dbConn.IDByName was trouble. But, other than idle poking with go-delve/delve, I can’t figure out what’s wrong. (I’d chalk that up to, again, being a Go newbie, and also completely inexperienced at debugging stack traces. And at writing unit tests, but I digress.)

Weirdest of all, it’s only on go test that I see breakage. Right now you can clone the source, go build it, and run the program normally without seeing a single issue.

I have a small suspicion that the issue might be my complete neglect of the database/sql.Tx feature, but I’d rather not be wrong and find myself barking up the wrong tree again.

Anyway. Gophers, can you help me dig myself out of this hole?

P.S.: It’s likely that, after I solve this problem, I’ll rebase away the changes and nuke these branches — let me know if you’d like me to put the relevant parts in a Gist, or something similar.

1 Like
3

I appreciate the link, but as I said in the post, my problem is specific to testing the Go code:

Weirdest of all, it’s only on go test that I see breakage. Right now you can clone the source, go build it, and run the program normally without seeing a single issue.

Rudimentary reads and writes have presented no issue in a monothreaded setting (the compiled binary). But, from my cursory reading, the tests are each run in their own goroutine.

So, I might have some kind of race condition on my hands, but I’m currently out of ideas about how to work around it.

(I could forge ahead without fixing this, but I’d like to get these tests working so that I don’t have to rely on manually running the tool.)

1 Like
4

Tests functions in go are not running default in sequentially.

You need to put some effort to run them sequentially.

The Run methods of T and B allow defining subtests and sub-benchmarks, without having to define separate functions for each. This enables uses like table-driven benchmarks and creating hierarchical tests. It also provides a way to share common setup and tear-down code:

func TestFoo(t *testing.T) {
//
t.Run(“A=1”, func(t *testing.T) { … })
t.Run(“A=2”, func(t *testing.T) { … })
t.Run(“B=1”, func(t *testing.T) { … })
//
}

Each subtest and sub-benchmark has a unique name: the combination of the name of the top-level test and the sequence of names passed to Run, separated by slashes, with an optional trailing sequence number for disambiguation.

The argument to the -run and -bench command-line flags is an unanchored regular expression that matches the test’s name. For tests with multiple slash-separated elements, such as subtests, the argument is itself slash-separated, with expressions matching each name element in turn. Because it is unanchored, an empty expression matches any string. For example, using “matching” to mean “whose name contains”:

go test -run ‘’ # Run all tests.
go test -run Foo # Run top-level tests matching “Foo”, such as “TestFooBar”.
go test -run Foo/A= # For top-level tests matching “Foo”, run subtests matching “A=”.
go test -run /A=1 # For all top-level tests, run subtests matching “A=1”.

https://tip.golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks

1 Like
5

@kync had a point: I can’t, speaking logically, test db.Events() as a unit without first inserting events.

This was the kick I needed to remove some rakes I had left for myself to step on repeatedly, among them being my insistence on in-memory SQLite DBs. I settled for a file that’s os.Removed later.

So now, all my tests pass and I can get back to freaking building out this thing already:

% go test
PASS
ok      github.com/Goorzhel/good        0.125s

Thanks, everybody!

2 Likes