Using different forks of same dependency

[byte-bandit (Christian Lohr)]

Hi.

I am currently faced with a problem that seems to be pretty unique. Hopefully someone will be able to help me out.

Essentially, I am trying to support two different projects A and O, both of which have been built on top of E. There is a sizeable go library for E, let’s call it lib-E, but in order to support the other projects, they have released their own libraries, which are direct forks of lib-E.

Notably, lib-A and lib-O do not have any import paths across the code base, and the go.mod project definition is identical across all three libraries: github.com/maintainer/lib-E. I assume this is done in order to minimise maintenance and allow for a more easy plug-and-play approach with consumers of the libs.

Maybe you’re already starting to see my issue now. Let’s assume I have two packages in my code base, one of them should be built against lib-A, one against lib-O. Both packages are identical in code, but as their underlaying dependency libs differ, so does the compiled output.

I have tried several approaches so far, all without luck:

• Extracting both packages into their own repositories, along with their own go.mod, each containing replace statements of lib-E to either lib-A or lib-O. This fails, since my main go.mod declares a direct dependency to lib-E which will override the module replacements.

• Keeping the code in one repository, but using named imports in both packages (i.e. import libA "github.com/maintainer/libE"), then using two replace directives in the main go.modthat point to the individual repositories forlibAandlibO. This fails since both modules are still declared as github.com/maintainer/libE`, so I end up with:

module declares its path as: github.com/maintainer/libE
	       but was required as: libA

As it stands now, I am starting to think that what I’m trying to do is impossible in Go, with the only solution being building two small binaries (each directly against libA or libO) and then use RPC between those small adapter binaries and my main application.

That’s obviously a lot of overhead I’d like to avoid if somehow possible. Can anyone think of a better solution for this problem?

[christophberger]

Hi @byte-bandit, welcome to the forum.

I am uncertain if I understand the problem fully, but to me, this seems to the root of the problem:

@byte-bandit

Notably, lib-A and lib-O do not have any import paths across the code base, and the go.mod project definition is identical across all three libraries: github.com/maintainer/lib-E

If lib-A and lib-O are forks of lib-E, their go.mod files should reflect this, as well as the import paths in A and O, respectively.

If changing the go.mod files and the import paths is not an option, you can try using a Go workspace to replace the import paths locally. (Tutorial, reference)

[byte-bandit (Christian Lohr)]

Hey @christophberger,

thanks for getting back to me, really appreciate the input. Let me be a bit more clear in my motivations.

Take a look at https://github.com/ethereum-optimism/op-geth and https://github.com/OffchainLabs/go-ethereum/ - They’re both forks of github.com/ethereum/go-ethereum (sorry, I can only post two links). Note that neither of them adapt their go.mod , neither their import paths. I can only assume this is so that any consumer may simply use either of them as a drop in replacement for the original one by using a replace keyword in their own go.mod file.

I’m maintaining an existing code base that uses op-geth and now wishes to introduce support for Arbitrum by creating a module using the OfficialLabs implementation. Note how they’re all pointing to github.com/ethereum/go-ethereum. While only supporting op-geth, this wasn’t an issue, thanks to replace. But having to pull in a second dependency that uses the same import paths has so far proven impossible.

I have tried various options I could think of, like extracting the code into individual modules, use work spaces, use named imports, I can’t even remember what else at this stage.

What I need is a reliable way of module level redirects, that is not overridden by the main module. That doesn’t seem to be allowed with go workspaces.

At this stage, I’m not convinced this is even possible in go, but it’s likely I’m simply not seeing the obvious here.