Mismatch in original structure of the file-content to the same file served on rest api end point

TechStack: HyperledgerFabric 1.4, ProjectRepository- FirstNetwork under FabricSamples
Go – v1.15 , Goa- v3.2.5, Postman

Requirement : User will input channelName and ChannelProfile from front-end. I am using Postman for trigerring the rest-api end point. Based on business logic, from go i have to trigger an .sh file which will generate a binary file. This binary file i have to send it back as a response. I have read the file using ioutil.ReadFile. Since the output of this file is a byte Array, type-casted into string. Printed the contents to the console and checked. The file-contents are displayed correctly.
Below is the screenshot of the file contents.

[screen1: file captured on console]

The output in postman:

I have used strings.ToValidUTF8() and also unicode.IsPrint() functions to remove invalid byte sequence and other non-printable chars.

Now I am getting the output as shown in below screen:

What i am looking for i need the same structure of the file as in screen 1.
Note here the key “FileContent” has the value which displays the contents of the file.
I am kinda new to go and goa. Please help me to understand where I am going wrong. Any help is appreciated.

enter code here // design.go file where api is defined

package design

import (
    "fmt"
    . "goa.design/goa/v3/dsl"
)

var _ = API("fabric", func() {
    Title("An api for fabric")
    Description("A simple goa service")
    Server("fabric", func() {
        Host("localhost", func() {
            URI("http://localhost:8000")
        })
    })
})

var _ = Service("fabric", func() {
    Description("The service creates profile in configtx.yaml file ")
    //Method to add a new Channel Profile
    Method("create", func() {
        Payload(func() {
            Field(1, "channelName", String, "Name of the channel")
            Field(2, "channelProfile", String, "Name of the channel profile")
            Required("channelName", "channelProfile")
        })
    
        Result(String)
        
        Error("not_found", ErrorResult, "channelProfile not found")
        HTTP(func() {
            POST("/createChannelProfile")               
            Response(StatusCreated)
        })
    })
    Files("/openapi.json", "./gen/http/openapi.json")
})
enter code here // fabric.go file where business logic is implemented

package fabricapi

import (
    fabric "GoApp2/gen/fabric"
    "bytes"
    "context"
    "encoding/json"
    "io/ioutil"
    "log"
    "os/exec"
    "strings"
    "unicode"
)

// fabric service example implementation.
// The example methods log the requests and return zero values.
type fabricsrvc struct {
    logger *log.Logger
}

// NewFabric returns the fabric service implementation.
func NewFabric(logger *log.Logger) fabric.Service {
    return &fabricsrvc{logger}
}


type FinalOutput struct {
    Status      string
    FileContent string
}

// Create implements create.
func (s *fabricsrvc) Create(ctx context.Context, p *fabric.CreatePayload) (res string, err error) {
    s.logger.Print("fabric.create")
    s.logger.Print("fabric.create")
    cmd := exec.Command("/bin/bash", "../generateChannel.sh", p.ChannelName, p.ChannelProfile)
    stdout, stderr := cmd.Output()
    if stderr != nil {
        s.logger.Print("error %s", err)
    }
    output := string(stdout)
    s.logger.Print(output)

    content, err := ioutil.ReadFile("../channel-artifacts/" + p.ChannelName + ".tx")
    if err != nil {
        s.logger.Print("error %s", err)
    }

    filecontent := string(content)
     s.logger.Print(filecontent)
    removeInvalidByteSequence := strings.ToValidUTF8(filecontent, " ")
        
    clean := strings.Map(func(r rune) rune {
        if unicode.IsPrint(r) {
            return r
        }
        return -1
    }, removeInvalidByteSequence)

    resfile := clean
    s.logger.Print(resfile)

    buf := new(bytes.Buffer)
    // create JSON encoder for `buf`
    bufEncoder := json.NewEncoder(buf)
    if output == "success\n" {
        // encode JSON from `FinalOutput` structs
        bufEncoder.Encode(FinalOutput{"success", resfile})
        res = buf.String()
        s.logger.Print(res)
        return res, nil
    } else {
        bufEncoder.Encode(FinalOutput{"failed", "File not Found : Please input the correct channelProfile"})
        res = buf.String()
        s.logger.Print(res)
        return res, nil
    }
    return
}

You are removing unprintable and invalid encoded sequences and expect the file to appear the same as before? Sorry I do not get the requirements…

Hi Nobbz,
If you can observe screen1, whatever the readable strings are there it is in vertically aligned with spaces and new lines. But where as in screen3 after sending the response back it is horizontally aligned. The spacing and alignment , I am expecting to be the same what is seen in screen1 to screen3.

I have tried the same with nodejs, It is sending the exact file the way it is, I haven’t removed the invalid byte sequences so, but here i have to write steps to remove the invalid byte sequences and non-printable characters. This is what I am expecting.

In general I do not care for screen 1, as you are printing random, binary data to a terminal, how it will get printed/displayed is an implementation detail of your shell and terminal emulator.

If you need to extract a certain structure from the file, properly parse it and extract the necessary data.

yes Nobbz exactly , this is where I am struggling and I need help how to do that. Any small help/hint is appreciated.

First you need to know the format the original file is in, and you need access to a specification or documentation of the format, then you reimplement a parser or search for a library that implements one.

The original file is of binary format. The extension of the file is “.tx”. I have seen your reply here How to read and write binary files?.

TX seems to be a file ending that is used by a lot of different programs and therefore not enough to learn anything about its format.

If you don’t have a spec of the format, you have nothing but guessing, or in more modern words, falling back to heuristics.

The format might even be proprietary and not publicly available.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.