Raw and Interpreted Strings

Getting Help
1

Hi.

I’ve a very simple go program - input-tester.go - that takes a string containing new lines as a parameter and splits the string on the new line:

package main

import (
	"os"
	"fmt"
	"strings"
)

func main() {
	if len(os.Args) == 2 {
			strArray := strings.Split(os.Args[1], "\n")
			fmt.Println("len:", len(strArray))
			for i, s := range strArray {
				fmt.Printf("%d:%s\n", i,s)
			}
	}
}

I’ve got another go program - executor.go - which executes input-tester:

package main

import (
	"os/exec"
	"bytes"
	"fmt"
)

func main() {
	parameter := "col1,col2,col3\nval1,val2,val3"
	cmd := exec.Command("/path/to/package/input-tester", parameter)
	var out bytes.Buffer
	cmd.Stdout = &out
	err := cmd.Run()
	if err != nil {
		panic(err)
	}
	fmt.Println(cmd.Stdout)
}

If I run the executor my output is:
len: 2
0:col1,col2,col3
1:val1,val2,val3

However if I run the input-tester.go from the command line (go run input-tester.go col1,col2,col3\nval1,val2,val3) the output is:
len: 1
0:col1,col2,col3\nval1,val2,val3

I understand that the issue is the difference between interpreted string from the executor and raw string from the command line. I also know if I update the strings.Split to use backticks n the result gets swapped around.

My question is - is there a way to make this work consistantly using the command line and the executor? Is there a way to change an interpreted string to a raw string or vice versa? I’ve checked the the utf8, strings and strconv packages with not much luck.

Thanks in advance.

2

Hey @airomega,

If I’m understanding you correctly, you can simply use something like strings.Replace and replace all raw \n characters with actual new line characters and that should solve your problem.

Example:

package main

import (
	"fmt"
	"strings"
)

func main() {
	str1 := "Hello\nWorld!"
	fmt.Println(str1)

	fmt.Println()

	str2 := `Hello\nWorld!`
	fmt.Println(str2)

	fmt.Println()

	str3 := strings.Replace(str2, `\n`, "\n", -1)
	fmt.Println(str3)
}
3

I probably should have posted a workaround that’s verbose but usable in case anyone else trips over this thread. However this is not an answer to my question (making implied strings raw and vice versa) - just a simple solution to the new line problem posed:

package main

import (
	"os"
	"fmt"
	"strings"
)

func main() {
	if len(os.Args) == 2 {
			var newLineString string
			if strings.Contains(os.Args[1], "\n") {
					fmt.Println("implied")
					newLineString = "\n"
			} else if strings.Contains(os.Args[1], `\n`) {
					fmt.Println("raw")
					newLineString = `\n`
			}

			strArray := strings.Split(os.Args[1], newLineString)
			fmt.Println("len:", len(strArray))
			for i, s := range strArray {
				fmt.Printf("%d:%s\n", i,s)
			}
	}
}
}

command line output:
raw
len: 2
0:col1,col2,col3
1:val1,val2,val3

Executor output:
implied
len: 2
0:col1,col2,col3
1:val1,val2,val3

4

Thanks for your response. Very helpful. I happened to post another solution at the same time as you posted this (I prefer yours) :smiley:

Please don’t think me facetious by saying it doesn’t quite answer the question I was interested to know if

there a way to change an interpreted string to a raw string or vice versa?

The problem with the solutions provided (mine included) is that they would require further code changes to deal with any other escape character that may crop up (e.g. \, '). I’d have to introduce a replace call for all affected characters. This would make the complexity of the replace solution:
O(n)*(affected chars)

Rereading my question it wasn’t clear that’s what I was looking for as I focused on a single new line example - apologies. I was hoping for a single operation that would change the string from raw to implied and deal with all escaped characters.

5

This doesn’t really make sense as the difference only exists in the source code. Once compiled, both are just strings. That said, you can quote and unquote strings:

6

I was just about to post a reply and see that Jakob has already replied with something similar to what I was going to suggest.

Basically, if you think about it, if you wanted to convert a literal \n to a newline in a string such as \nhello, how do you know if the \n is a newline or if the string is actually a \ followed by nhello, so in this regard, I don’t personally know any simple way how you could switch back and forth correctly.

With this in mind, you could always use something like the following if you know for certain that there will not be mix ups in your string and know that anytime there is something such as \nhello, that you actually want a newline character and not a \ and then some text.

package main

import (
	"fmt"
	"log"
	"strconv"
	"strings"
)

func main() {
	str := `Hello\nWorld!`
	str = strconv.Quote(str)
	replaced := strings.Replace(str, `\\`, "\\", -1)
	newstr, err := strconv.Unquote(replaced)
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Println(newstr)
}
7

Thanks for your input. I understood that the difference is in the source code - I was hoping there was some encoding solution I hadn’t considered. I had looked Quote and Unquote already but they didn’t accomplish what I was after.

8

Thanks. The point about converting and not being able to determine whether it was /n and hello or / and nhello makes sense. Replace it is :smile: thanks for the input - much appreciated.

1 Like
9

Your original question about getting consistency between running from command line and running from Go is solvable though. Your Go program passes the parameter with a newline in, while the shell does no such expansion of \n by default. If I were you I would let your program be exactly as it is and solve this at the shell level, if you want your program to expect new lines. I.e., in bash,

jb@ams1:~ $ echo foo\nbar
foo\nbar
jb@ams1:~ $ echo $'foo\nbar'
foo
bar
jb@ams1:~ $

Your behavior is then consistent - it works when getting actual newlines. :slight_smile:

(Or consider piping input from stdin, I guess)

10

Thanks for that. That’s a good solution however I don’t think I can use that either :slight_smile:

The problem arose as I’m writing an in-house utility to parse and manipulate large csv inputs. As a Java dev who is a total convert to Go I was making the case to create the utilty in Go when I ran into this issue - it piqued my curiosity.

The util accepts CSV parameters in various formats - comma new line separated CSV string, json and csv files (where line ending formats cannot be guaranteed). It’ll be run by upstream programs and by individuals. In order to execute the program correctly using your previous suggestion, I’d have to bundle the util with a shell script. The replace solution is a more workable, user proof solution :smile:

It’s been an interesting discussion. Thanks for the input.

11

This has nothing to do with Go as such though. Whatever your current solution does, you can do here to.

12

:slight_smile: I understand that.

I started my last comment by describing the larger utility - I thought contextualising what I was doing would clarify why I was asking the question and was I was looking for a raw -> interpreted solution - it was more for context as I was happy the issue was solved thanks to Bejamins answer. I understood your command line execution suggestion was not anything to do with Go however it’s not a suitable solution for my requirements.

I happy that the issue is resolved. Thanks for your input.

13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.