Nanosecond timestamp precision not playing well in containers

(Cross-posted from here)

Hey! I’ve got a test suite that’s failing inside containers but passing on the host machine.

It’s written in Go and uses time.Now() to get nanosecond precision timestamps. The tests are basically doing a custom json.Marshal and json.Unmarshal in order to convert the timestamp to an integer and back again - it all works nicely in our system (if anyone’s interested why, it’s to save bytes on large BSON payloads with tons of time-series data, timestamps are just 8 bytes whereas a string representation can be up to 55 bytes!)

So the weird thing here is that time.Now() is actually returning nanosecond precision because it uses it in the test case to set the expected value (as you’ll see in the diff) but the conversion code doesn’t appear to be functioning the same inside the container.

DataStructure{Time:time.Time{wall:0xbe680503db9047f2, ext:3924085, loc:(*time.Location)(0x9fe260)}, <other fields...>} (expected)

DataStructure{Time:time.Time{wall:0x1b9047f2, ext:63641326479, loc:(*time.Location)(0x9fe260)}, <other fields...>} (actual)

You can see the expected value contains the full 64 bit timestamp bytes, no zeroes,
And here’s a larger breakdown from the awesome Testify library:

Diff:
--- Expected
+++ Actual
@@ -1,3 +1,3 @@
    (shared.TaskPosition) {
- Time: (time.Time) 2017-09-18 10:14:39.462440434 +0000 UTC m=+0.003924085,
+ Time: (time.Time) 2017-09-18 10:14:39.462440434 +0000 UTC,
    TaskID: (shared.TaskID) (len=2) "t1",

So in the test, the expected value is being loaded correctly but somewhere in the conversion code, it’s not kicking out the same value again - and this only happens inside docker. I actually have no idea what m=+0.003924085 is, I assumed the .462440434 contains the granularity for nanoseconds. Why is this m value missing from the actual result?

The conversion code looks like this:

func (tp *DataStructure) UnmarshalJSON(b []byte) error {

	// create a dummy data structure of the expected format, all other fields are composed in and
	// the `Time` field is overwritten with an int64 instead of a time.Time as in the alias
	type alias DataStructure
	tmp := struct {
		Time int64 `json:"t"`
		*alias
	}{}

	// Unmarshal the fake structure, processing Time as an int64
	err := json.Unmarshal(b, &tmp)
	if err != nil {
		return err
	}

	// convert the Time in the temporary structure to a time.Time and assign it to the real one
	tp.Time := time.Unix(0, tmp.Time)

	// assign all other fields to `tp`
	tp.OtherFields = tmp.OtherFields
	// etc...

	return nil
}

I understand this may be a Go question but the fact that it only happens in a container is odd to me. Hopefully someone can help find a solution!

Thanks!

The +m... stuff is the monotonic clock added in Go 1.9. It will be set in times created by time.Now(), but not when you create a time from parsing or otherwise. Perhaps this explains it, depending on what the test does. Perhaps it doesn’t but the difference is that a monotonic clock is not available in the container. Perhaps the non-container tests are running on an older version of Go.

Ah, I completely forgot about that!

My machine has Go version 1.8 and the container is running 1.9 - that’s the root of the problem.

Now I just have to figure out how to properly do these tests in 1.9 :thinking: any ideas? It must be the lack of the monotonic clock being added to the structure during the time.Unix(0, tmp.Time) call - I suppose I could remove it from the time.Now call that sets up the original expected value…

A bit pedantic but surely this “breaks” the “Go 1.0 promise” of non-breaking changes, even though it’s only a test case that broke it’s still a break…

From the page above,

The canonical way to strip a monotonic clock reading is to use t = t.Round(0).

so I’d do that on the value when comparing to what you deserialize.

1 Like

Ah, that’s what I get for not reading! Thanks :slight_smile:

About the compatibility, https://golang.org/pkg/time/#Time.String says

The returned string is meant for debugging; for a stable serialized representation, use t.MarshalText, t.MarshalBinary, or t.Format with an explicit format string.

I haven’t checked if that was added with 1.9 or not, though. :slight_smile:

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.