The easiest way to solve easy problems tend to be hard to test: procedural approach needs some testing love

Hi.

We all use “divide and conquer” approach when writing our code. The actual “divide” may vary depend on a task, yet in many circumstances it is just we extract a “smaller” method from one we need to implement.

This is quite a typical approach when implementing some gRPC service, for instance, where we need to implement some interface.

So, how it may looks like:

  1. We have some method to write. Let it be GetByIDs
  2. We need to use different subsystems (dbs, other services, queues, etc) to get these entities by their ids.
  3. At this time we are limited by able to get only one item by its id. This is just an example, to express an idea. So the clearest solution is to introduce a getByID method which will be called for any of ids given to GetByIDs.

Imagine we need to

  1. As if an entity with given id is presented in db1. Convert and return it then.
  2. If an entity is presented in db2 retrieve, convert and return it just like before.
  3. Try to get entity from another service if both methods above got nothing.

Need to send notification about success getting an entity.

This may looks like this:

type Struct struct {
    db1 repo1.Repo
    db2 repo2.Repo
    srv srv.Service

    mq queue.Queue
}

func (s *Struct) getByID(ctx context.Context, id string) (res *Entity, err error) {
    defer func() {
        if res == nil {
            return
        }

        if err = s.mq.Send(ctx, constSubject, id, res.Source); err != nil {
            err = fmt.Errorf("queue an item got from the source %s: %w", res.Source, err)
        }
    }()

    v1, err := s.db1.GetByID(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("try get entity from db1: %w", err)
    }
    if v1 != nil {
        return convert1(v1), nil
    }

    v2, err := s.db1.GetByID(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("try get entity from db2: %w", err)
    }
    if v2 != nil {
        return convert2(v2), nil
    }

    v3, err := s.srv.GetByID(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("try get entity from service: %w", err)
    }
    if v3 != nil {
        return convertSrv(v3), nil
    }

    return nil, nil
}

GetByIDs will look like this then:

func (s *Struct) GetByIDs(ctx context.Context, ids []string) ([]*Entity, error) {
    var res []*Entity
    for _, id := range ids {
        ent, err := s.getByID(ctx, id)
        if err != nil {
            return fmt.Errorf("get entity by id=%s: %w", id, err)
        }

        if ent == nil {
            logger.Ctx(ctx).Warn().EntID(id).Msg("no entity with such id")
            continue
        }
        res = append(res, ent)
    }

    return res, nil
}

So, how we would test it? We divided an implementation, so why not to divide testing too? OK, we generate/write manually mocks for these repos and service, write a good test for getByID and now write a test for GetByIDs.

And it turns out we cannot divide testing. Need to test getByID again and again when testing GetByIDs. Getting more annoying the more complicated “smaller” getByID is.

Know about interfaces. Just don’t want to use them here because it would be an abstraction for the sake of it without a real need.

I guess this easy and straightforward approach needs a special testing magic where we could make a kind of “literal inheritance”, i.e. get a type with embedded Struct whose getByID calls will be redirected to a substitutes. I mean something like that

//go:testing-redirect Struct.getByID testingStruct.getByID
type testingStruct struct {
    Struct

    mock *StructMock
}

// this will be called from within testingStruct.GetByIDs
func (s *tesitngStruct) getByID(ctx context.Context, id string) (*Entity, error) {
    return s.mock.getByID(ctx, id)
}

What do you think about it?

You lost me here. If you’ve set up mocks for getByID, Why can’t you test GetByIDs?

(*Struct).GetByIDs calls (*Struct).getByID, not (*testingStruct).getByID. The nested Struct doesn’t know about the type wrapping it; it’s not actual inheritance. Go uses static function dispatch for concrete types and only uses virtual dispatch for interfaces.

If I needed the functionality GetByIDs provided, I would use interfaces so that I could implement it like this:

type multiGetByID struct {
    gbidFuncs []interface{ getByID(context.Context, string) (*Entity, error) }
}

func (m multiGetByID) GetByIDs(ctx context.Context, ids []string) (es []*Entity, err error) {
    es = make([]*Entity, 0, len(ids))
    for _, id := range ids {
        for _, gbidFunc := range m.gbidFuncs {
            e, err := gbidFunc(ctx, id)
            if err != nil {
                return nil, fmt.Errorf("get entity by id=%s: %w", id, err)
            }
            if e == nil {
                logger.Ctx(ctx).Warn().EntID(id).Msg("no entity with such id")
                continue
            }
            es = append(es, e)
        }
    }
    return
}

You lost me here. If you’ve set up mocks for getByID , Why can’t you test GetByIDs ?

Because getByID is just a method and cannot be mocked

(*Struct).GetByIDs calls (*Struct).getByID , not (*testingStruct).getByID . The nested Struct doesn’t know about the type wrapping it; it’s not actual inheritance. Go uses static function dispatch for concrete types and only uses virtual dispatch for interfaces.

I know. That’s why I would love to have a special testing magic to emulate this.

If I needed the functionality GetByIDs provided, I would use interfaces so that I could implement it like this:

Introduce interface just for testing. This is exactly what I would love to avoid: there’s straightforward solution and it would be great if the language would provide a support for easy testing of it as well.

Now we have either easy solution + overcomplicated testing or needlessly complicated solution with easy testing.

In fact I can make “easy” testing without interfaces but this needs two test runs with different build tags and a bit of supplementary code.

I’m not sure I understand the issue: Based on your initial post describing “divide and conquer,” I’m under the impression that GetByIDs used to be a single function and that you factored it out into GetByIDs and getByID. If that’s true, then I don’t understand why you’re testing getByID itself because it’s an implementation detail of GetByIDs.

Your current implementation loops through the IDs and then conceptually “loops” through the underlying DBs/services/queues/etc., and does so serially. Let’s say you refactor to essentially flip the loops: Instead of iterating through the IDs and trying to get the ID with each underlying implementation, you instead loop over the implementations and pass in all the IDs with each implementation (maybe it’s faster because some of those implementations support some sort of IN or .contains() operator(s)). Now, even though the results of GetByIDs should be the same, you have to change your tests because you don’t have a getByID function any more; it’s getByIDs and receives the same ids []string parameter as GetByIDs.

It sounds like instead of using interfaces (an existing, heavily used and understood concept in Go and one that seems like it would solve your problem), you want a new language feature to enable inheritance (but perhaps only for testing?). Am I correct? If so, I find it extremely unlikely that this feature request will be implemented.

I will say that I’m just another Go user and have no affiliation with the official Go language team, so my opinion means nothing. But I suggest that if you would like this feature to be implemented and added to the Go language specification, that you come up with a more concrete reason why this feature is necessary. I don’t think they will accept interfaces being too complicated.