I have been trying for a couple days now to do something that I thought would be simple. I simply want to modify a specific key value (many layers deep) in a docker compose yaml file but leave the rest of the file intact (including comments). Docker compose files to not have a well defined structure, so each file may have a different schema.
As far as I can tell, the standard golang yaml libraries do not support anything like what I am looking for. I have never been able to unmarshal / marshal a docker compose yaml file using map[string]interface{} and have it look anything like the original. Many elements of the file are missing when I try this. Using unmarshal / marshal with yaml.Node generates errors from the marshal call. In short, I cannot even enmarshal / marshal a docker compose yaml file much less modify it.
I’m not using it yet. I still use gopkg.in/yaml.v2 which uses map[interface{}]interface{} and does not support comments.
Are there missing elements besides comments when you unmarshal/marshal? I don’t see how you can preserve comments with Unmarshal/Marshal, especially when you use generic map[string]any. Where would the comments go? You’d have to use some kind of Decoder/Encoder.
Have you thought of editing the file without using yaml, using regular expressions to find the value you want to modify?
Preserving comments and possibly formatting does not seem extremely simple to me.
I think you want yaml.Node. Let’s create a file called test.yaml with the following content:
services: # These are the services
# OK new line comment
web: # web and ports and stuff
build: .
ports:
- "8000:5000"
redis:
image: "redis:alpine"
Then try the following go code:
func main() {
b, err := os.ReadFile("test.yaml")
if err != nil {
log.Fatalf("Problem opening file: %v", err)
}
var dockerCompose yaml.Node
yaml.Unmarshal(b, &dockerCompose)
// Swap out redis:alpine for redis:golang
imageNode := findChildNode("redis:alpine", &dockerCompose)
if imageNode != nil {
imageNode.SetString("redis:golang")
}
// Create a modified yaml file
f, err := os.Create("modified.yaml")
if err != nil {
log.Fatalf("Problem creating file: %v", err)
}
defer f.Close()
yaml.NewEncoder(f).Encode(dockerCompose.Content[0])
}
// Recusive function to find the child node by value that we care about.
// Probably needs tweaking so use with caution.
func findChildNode(value string, node *yaml.Node) *yaml.Node {
for _, v := range node.Content {
// If we found the value we are looking for, return it.
if v.Value == value {
return v
}
// Otherwise recursively look more
if child := findChildNode(value, v); child != nil {
return child
}
}
return nil
}
Which produces the following in modified.yaml:
services: # These are the services
# OK new line comment
web: # web and ports and stuff
build: .
ports:
- "8000:5000"
redis:
image: "redis:golang"
This should at least get you on the path to what you’re looking for. My only other thought is: since Docker is written in Go and open source, you could just see what they’re doing to parse the yaml files in question and copy that. However, many yaml parsers don’t consider comments to be part of the data serialization/deserialization and thus they are probably ignoring the comments. Useful reading:
Please see my original question. None of the standard yaml packages (including yaml.v3 / yaml.Node work if you do not know the schema up front. They are all fairly useless.
I believe that statement is incorrect. See my example code where I recursively loop over the yaml as a yaml.Node object with yaml.Node children. That code will work regardless of schema and you don’t need to know the schema up front. The difference is that vmware-labs/go-yaml-edit is using a text/transform.Transformer to transform the text.
It’s in their labs project, which isn’t intended for production use. From that org, emphasis mine:
This organization contains experimental open source projects.
And again, there are absolutely solutions for arbitrary yaml files. The code above is generic and knows nothing of the schema it is iterating over. And it would work with any schema.
You can file a bug report or a feature request against your favorite yaml package, if you have one.
Do you have an example yaml file that fails to unmarshal/marshal, or decoded/encoded correctly?