Problems using database/sql with denisenkom/go-mssqldb

macOS 10.13.1 High Sierra
Golang 1.9.2
Docker Toolbox
microsoft/mssql-server-linux:2017-latest

Hello, I am playing around with database/sql with the denisenkom/go-mssqldb driver. I’ve successfully managed to connect to SQL Server and to execute TSQL statements, but I am having difficulties, I think due to shortcomings in my understanding of the way SQL Server works, rather than any Golang difficulties.
This is my client code:

func main() {
	db := q.Connect()
	defer db.Close()
	// tx := db.NewTx()
	db.Exec("DROP DATABASE IF EXISTS Company")
	db.Exec(`CREATE TABLE Company
		(ProductID int PRIMARY KEY NOT NULL,
		ProductName varchar(25) NOT NULL,
		Price money NULL,
		ProductDescription text NULL)`)
	// tx.EndTx()
}

The problem is that the CREATE TABLE command is failing:

2017/12/02 13:22:08 exec: CREATE TABLE Company
                (ProductID int PRIMARY KEY NOT NULL,
                ProductName varchar(25) NOT NULL,
                Price money NULL,
                ProductDescription text NULL) failed: mssql: There is already an object named 'Company' in the database.

What am I missing here? Thanks.

You drop a database Company and create a table Company. I guess you must drop the table Company.

Thanks very much George! You know sometimes when you look and look and cannot see the problem!
Now to get the transactions working…!

Corrected version:

func main() {
	db := q.Connect()
	defer db.Close()
	// tx := db.NewTx()
	db.Exec("DROP DATABASE IF EXISTS Company")
	db.Exec("DROP TABLE IF EXISTS Company")
	db.Exec(`CREATE TABLE Company
		(ProductID int PRIMARY KEY NOT NULL,
		ProductName varchar(25) NOT NULL,
		Price money NULL,
		ProductDescription text NULL)`)
	// tx.EndTx()
}

Mind you, surely dropping the database Company would automatically drop the enclosed table Company…

OK. I’ve made some progress, My client code is now:

package main

import (
	q "github.com/carlca/bigdata/sqlserver"
	_ "github.com/denisenkom/go-mssqldb"
)

func main() {
	db := q.Connect()
	defer db.Close()
	tx := db.NewTx()
	tx.Exec("DROP TABLE IF EXISTS Company")
	tx.Exec(`CREATE TABLE Company
		(ProductID int PRIMARY KEY NOT NULL,
		ProductName varchar(25) NOT NULL,
		Price money NULL,
		Price2 money NULL,		
		ProductDescription text NULL)`)
	tx.CommitTx()
}

I also have this library code:

package sqlserver

import (
	"database/sql"
	"flag"
	"fmt"

	e "github.com/carlca/utils/essentials"
)

var (
	debug    = flag.Bool("debug", false, "enable debugging")
	password = flag.String("password", "", "the database password")
	port     = flag.Int("port", 1433, "the database port")
	server   = flag.String("server", "", "the database server")
	user     = flag.String("user", "", "the database user")
)

// DB inherits from sql.DB
type DB struct {
	*sql.DB
}

// Exec combines Prepare and Exec methods
func (db *DB) Exec(cmd string) {
	stmt, err := db.Prepare(cmd)
	e.CheckError("prepare: "+cmd+" failed", err)
	_, err = stmt.Exec()
	e.CheckError("exec: "+cmd+" failed", err)
	if *debug {
		fmt.Printf("exec: %v succeeded\n", cmd)
	}
}

// NewTx wraps the *DB.Begin func
func (db *DB) NewTx() *Tx {
	tnx, err := db.Begin()
	e.CheckError("BeginTx failed: ", err)
	if *debug {
		fmt.Printf("NewTx: succeeded\n")
	}
	return &Tx{*tnx}
}

// Tx inherits from sql.Tx
type Tx struct {
	sql.Tx
}

// Exec wraps *Tx.Exec
func (tx *Tx) Exec(cmd string) {
	stmt, err := tx.Prepare(cmd)
	e.CheckError("prepare: "+cmd+" failed", err)
	_, err = stmt.Exec()
	e.CheckError("exec: "+cmd+" failed", err)
	if *debug {
		fmt.Printf("exec: %v succeeded\n", cmd)
	}
}

// CommitTx wraps the *Tx.Commit func
func (tx *Tx) CommitTx() {
	err := tx.Commit()
	e.CheckError("tx.CommitTx failed: ", err)
	if *debug {
		fmt.Printf("tx.CommitTx succeeded\n")
	}
}

// Connect establishes contact with an SQL Server
func Connect() *DB {
	// parse command line flags
	flag.Parse()
	// dump flags if debug
	if *debug {
		fmt.Printf("password: %s\n", *password)
		fmt.Printf("port: %d\n", *port)
		fmt.Printf("server: %s\n", *server)
		fmt.Printf("user: %s\n", *user)
	}
	// build connection string
	connString := fmt.Sprintf("server=%s;user id=%s;password=%s;port=%d", *server, *user, *password, *port)
	// if debug dump connection string
	if *debug {
		fmt.Printf("connString: %s\n", connString)
	}
	// create an SQL Server connection
	dbx, err := sql.Open("mssql", connString)
	e.CheckError("Open DB failed: ", err)
	if *debug {
		fmt.Printf("open mssql: succeeded\n")
	}
	err = dbx.Ping()
	e.CheckError("db.Ping failed", err)
	return &DB{dbx}
}

The problem is that I’m getting intermittant failure at the end of the client code. It says “panic: sql: connection returned that was never out”. When this occurs, however, if I wait a couple of seconds and run the client program again, it runs without failure.

The error message comes from a func *DB.putConn in sql.go, and seems to be a state dependent issue that occurs depending on timing.

I’ve had a search on Google but, apart from some mentions from github.com/go from 2012 or thereabouts, I cannot find any relevant references.

Can anyone help? Thanks.

func (db *DB) putConn(dc *driverConn, err error) {
	db.mu.Lock()
	if !dc.inUse {
		if debugGetPut {
			fmt.Printf("putConn(%v) DUPLICATE was: %s\n\nPREVIOUS was: %s", dc, stack(), db.lastPut[dc])
		}
		panic("sql: connection returned that was never out")
	}
	if debugGetPut {
		db.lastPut[dc] = stack()
	}
	dc.inUse = false

	for _, fn := range dc.onPut {
		fn()
	}
	dc.onPut = nil

	if err == driver.ErrBadConn {
		// Don't reuse bad connections.
		// Since the conn is considered bad and is being discarded, treat it
		// as closed. Don't decrement the open count here, finalClose will
		// take care of that.
		db.maybeOpenNewConnections()
		db.mu.Unlock()
		dc.Close()
		return
	}
	if putConnHook != nil {
		putConnHook(db, dc)
	}
	added := db.putConnDBLocked(dc, nil)
	db.mu.Unlock()

	if !added {
		dc.Close()
	}
}

It’s worth pointing out that when the code fails, it is failing at a point after the occurrence of the very last item in the client code, namely the call to tx.CommitTx. The debug info and error message is:

go run simple.go -debug=true -server=192.168.99.100 -password=<MyPassword> -user=sa
password: <MyPassword>
port: 1433
server: 192.168.99.100
user: sa
connString: server=192.168.99.100;user id=sa;password=<MyPassword>;port=1433
open mssql: succeeded
NewTx: succeeded
exec: DROP TABLE IF EXISTS Company succeeded
exec: CREATE TABLE Company
                (ProductID int PRIMARY KEY NOT NULL,
                ProductName varchar(25) NOT NULL,
                Price money NULL,
                Price2 money NULL,
                ProductDescription text NULL) succeeded
tx.CommitTx succeeded
panic: sql: connection returned that was never out

goroutine 20 [running]:
database/sql.(*DB).putConn(0xc420098140, 0xc4202ae000, 0x14008a0, 0xc42004a1d0)
        /usr/local/go/src/database/sql/sql.go:1054 +0x202
database/sql.(*driverConn).releaseConn(0xc4202ae000, 0x14008a0, 0xc42004a1d0)
        /usr/local/go/src/database/sql/sql.go:380 +0x47
database/sql.(*driverConn).(database/sql.releaseConn)-fm(0x14008a0, 0xc42004a1d0)
        /usr/local/go/src/database/sql/sql.go:643 +0x3e
database/sql.(*Tx).close(0xc4202ca000, 0x14008a0, 0xc42004a1d0)
        /usr/local/go/src/database/sql/sql.go:1720 +0x91
database/sql.(*Tx).rollback(0xc4202ca000, 0x1, 0x0, 0x1057b91)
        /usr/local/go/src/database/sql/sql.go:1807 +0xef
database/sql.(*Tx).awaitDone(0xc4202ca000)
        /usr/local/go/src/database/sql/sql.go:1701 +0x5e
created by database/sql.(*DB).beginDC
        /usr/local/go/src/database/sql/sql.go:1451 +0x216
exit status 2

Also, I tried rearranging the client code with defer tx.CommitTx like so:

func main() {
	db := q.Connect()
	defer db.Close()
	tx := db.NewTx()
	defer tx.CommitTx()
	tx.Exec("DROP TABLE IF EXISTS Company")
	tx.Exec(`CREATE TABLE Company
		(ProductID int PRIMARY KEY NOT NULL,
		ProductName varchar(25) NOT NULL,
		Price money NULL,
		Price2 money NULL,		
		ProductDescription text NULL)`)
}

When I ran it. It failed twice in quick succession and then ran without error the third and subsequent times.

I suggest you to put a small delay (around 200ms) between two successive SQL commands.

time.Sleep(200 * time.Millisecond)

Thanks George, I’ll try that out. What do you think is causing the problem and is this a safe solution that you’re suggesting - it seems a bit hit and miss…

Hi again George, I tied the sleep command in a variety of places and it didn’t improve things. I’m about to try out PostGres because I’m not contractually obliged to use any particular DBMS. And beside, although I use VSCode for coding Golang, I don’t particularly want to be reliant on too many MS products!

Cheers,
Carl

Hey Carl,
I used in a few cases denisenkom driver on MSSQL server (2014 i guess) but the things went well. I used this kind of initialization

db, err = sql.Open("mssql", "server="+c.Mssql.Ip+";user id="+c.Mssql.User+";password="+c.Mssql.Password+";database="+c.Mssql.Db+";encrypt=disable")
if err != nil {
	log.Fatal(err.Error())
}

and this kind of exec

	_, err := db.Exec("sql...")
	if err != nil {
		log.Print(err.Error())
	}

Indeed when i made a loop of exec instructions i observed that a small delay to complete the comand was needed (around above value). Also i didn’ t need to use locks.

I understand that you try to use the Linux version of MSSQL. I did’n tested SQL server on Linux.

Hi George,

Thanks a lot for taking the time to reply like this, I really appreciate it!

In the past I’ve done production work using SQL Server and I remember that I always had to use a series of delays and that to arrive at a combo of delays that actually worked, usually involved a Faustian pact or two. There was always this nagging sense of insecurity with the whole business, not what you want when dealing with databases!

The fact that this is a combo of Linux with SQL Server and Docker, which is obviously a young grouping of technologies without much time to settle, bothers me as well.

As I said, I am going to investigate PostGresSQL because that has a native Mac installer and I believe it it is very well established with a good reputation.

Cheers,
Carl

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.