Help with unicode conversion

I am reading data out of SAP Hana and if the type is decimal, it comes in as follows:

If the field type is decimal: ACWP_HRS_EXCESS_RUNTIME:"\ufffd\ufffd\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000080"

If the field type is double:
DBL_ACWP_HRS_EXCESS_RUNTIME:6.392

How can I get the decimal value to show up like the double?

I have tried so many iterations of using the unicode.SimpleFold function and the utf8.DecodeRuneInString function that I cannot see straight. Coming to Go from PHP is a challenge. :slight_smile:

Thank you for helping a newbie!

Sincerely,
Scott

Odds are that the data you’ve got there has already been mangled beyond recognition. I don’t know how a “decimal” is represented in the registry (and couldn’t find anything by a quick Google), but the string

"\ufffd\ufffd\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000080"

is 16 bytes of data:

[]byte{0xff, 0xfd, 0xff, 0xfd, 0x00, ... 0x00, 0x80}

which we could have tried to interpret, if we knew the format. But the two initial “characters” \ufffd are the unicode “replacement character”, which gets inserted when something which was not a valid unicode codepoint is interpreted as unicode. This indicates we’ve probably already lost whatever data was in those positions, before producing the string you’re showing here.

So, back up a little. What does your code look like? Do you have any specs or documents on the expected data format?

1 Like

Ain’t they Oracle DB under the hood?
If yes, try DUMP and maybe github.com/rana/ora/num

Thanks for the fast response. Here is my code that gets the data:

`
rows, err := db.Query(sqlString)
if err != nil {
return “”, err
}
defer rows.Close()
columns, err := rows.Columns()
if err != nil {
return “”, err
}
count := len(columns)
tableData := make([]map[string]interface{}, 0)
values := make([]interface{}, count)
valuePtrs := make([]interface{}, count)
for rows.Next() {
for i := 0; i < count; i++ {
valuePtrs[i] = &values[i]
}
rows.Scan(valuePtrs…)
entry := make(map[string]interface{})
for i, col := range columns {
var v interface{}
val := values[i]
b, ok := val.([]byte)
if ok {
v = string(b)
} else {
v = val
}
entry[col] = v

	}
	tableData = append(tableData, entry)
}

`
I looked at the docs in the SAP driver, but they are beyond my understanding since I am just starting in Go. They state on the page for the driver (https://github.com/SAP/go-hdb) that it has: Built-in support of HANA decimals as Go rational numbers http://golang.org/pkg/math/big.

But I am not sure how to translate this into code.

It is crazy that they do not have an example showing how to do this required conversion.

Any help is appreciated.

Thanks again,
Scott

I don’t think they are Oracle under the hood since they are big competitors in the in-memory database space. I will look into the libs you suggested though. Thanks!

Scott

You have a lot of empty interfaces, which obscures what you get back. Maybe try just selecting the relevant column and scanning into a *math/big.Rat, which is supposedly what the driver will use for decimals then.

Jakob,

I need a generic pull because I will not know what field is getting pulled or if the field is decimal ahead of time. That is why I am using the slices of interfaces. This is powering a rest based way to access json data. So, the url will have the table and the rest interface (which I am trying to build) will pull back the data in json. I do not know enough to check which fields are big.Rat in order to use that type to translate the field. Does that make any sense?

Scott

Sure, but you could test that the Rat works as you expect. When you know that it does, you can grab it from an empty interface using a type switch or whatever.

Or do something else. I’m just offering a debugging step.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.