I have some Golang codes which look like the following
package main
type MyStruct struct {
field1 string
field2 float64
field3 int
field4 bool
}
func main() {
names := getNames()
myStruct := getMyStruct(names)
writeToCsv(myStruct)
}
func getNames() []string {
// get list of names then return
}
func getMyStruct(names []string) []Mystruct {
myStruct := []MyStruct{}
for i := range names {
// do something then assign the computed values
myStruct = append(myStruct, MyStruct{
field1: value1,
field2: value2,
field3: value3,
field4: value4,
})
}
return myStruct
}
func writeToCsv(myStruct []MyStruct) {
// prepare writer then write header
for i := range myStruct {
// create the slice of string to be written to csv
}
}
It works correctly. However, I would like to be able to limit the number of rows written in a single csv file to e.g. 500,000 rows without having rows of the same name
(from the names
slice) being separated.
For example, name1
has 200,000 MyStruct rows, name2
has 289,000 MyStruct rows and name3
has 180,000 MyStruct rows. Since the total number of rows from name1
to name3
is already more than 500,000, I would like it to be written in a single csv file. After that, I would like to continue getting the MyStruct data for name4
, name5
and so on and so forth until their total again exceed 500,000.
Can the above be solved properly using closures?
Also, since I’m using Golang, I think it would be better if I also know how to do the following concurrently:
1. Get the MyStruct rows
2. Write them to CSV
Thanks for the help.