Skipping last line of file

[MantasSilanskas]

Hello, I have big file 6000+ lines. Last line is time stamp which gets fatal errors when reaches my parsing how I can stop my data parsing when i reach this time stamp or just skip last line of the file

[MantasSilanskas]

I skip first line of document by: scanner.Scan() cause first line has no usefull data.

I use bufio.Scanner. My code :

scanner := bufio.NewScanner(GPSData)
scanner.Scan()
for scanner.Scan() {
if reisaiDataRow, err = rp.allocateFile1Neo(scanner.Text()); err != nil {
return nil, fmt.Errorf(“failed allocate: %s”, err.Error())
}
if reisaiDataRow == nil {
continue
}
reisaiMap[reisaiDataRow.BusNumber] = reisaiDataRow
}

[Christophe_Meessen]

We would need more information to help you. What is the error you get ? What does rp.allocateFile1Neo ?

Instead of passing scanner.Text() directly to rp.allocateFile1Neo, you could assign it to a variable and check if the variable content is valid before passing the value to rp.allocateFile1Neo.

scanner := bufio.NewScanner(GPSData)
scanner.Scan()
for scanner.Scan() {
    line := scanner.Text()
    if !isValid(line) {
        continue
    } 
    if reisaiDataRow, err = rp.allocateFile1Neo(line); err != nil {
        return nil, fmt.Errorf(“failed allocate: %s”, err.Error())
    }
    if reisaiDataRow == nil {
        continue
    }
    reisaiMap[reisaiDataRow.BusNumber] = reisaiDataRow
}

[MantasSilanskas]

Hello,

I will rephrase my question.

I have file with over 6000 lines first line is like explanation
example:
carNumber,destination etc.

Following lines are data
example:
46779978, Kaunas etc.

There is total 11 data fields all separated by coma. I read this data from file with following code :

scanner := bufio.NewScanner(GPSData)
scanner.Scan()
for scanner.Scan() {
if reisaiDataRow, err = rp.allocateFile1Neo(scanner.Text()); err != nil {
return nil, fmt.Errorf(“failed allocate: %s”, err.Error())
}
if reisaiDataRow == nil {
continue
}
reisaiMap[reisaiDataRow.BusNumber] = reisaiDataRow
}

allocateFile1Neo is data parsing like :

arr := strings.Split(data, “,”)
r = new(ReisaiDataRow)

r.VehicleType = arr[0]
r.Route = arr[1]
if r.Schedule, err = strconv.Atoi(arr[2]); err != nil {
return nil, err
}

Last line of data is time stamp like : 2019-10-15 18:42:00

I get following errors :

I think it has to do that it doesn’t have any comas at all and it still tries to parse it? How do program my program to recognise that time stamp or just skip last line of the file?

[Christophe_Meessen]

You are right.
The problem is that with the split or the comma, you don’t get a slice with multiple elements since there are no commas in the last line. The instruction r.Route = arr[1] then fails because the index is out of bound.

A simple solution is to test for the length of the arr slice. If it’s not the expected length, then a field is missing or you are at the last line of the file which contains only a date. You could thus use this code instead:

arr := strings.Split(data, “,”)
if len(arr) != 3 {
    return nil, ErrNotEnoughFields
}
r = new(ReisaiDataRow)
r.VehicleType = arr[0]
r.Route = arr[1]
if r.Schedule, err = strconv.Atoi(arr[2]); err != nil {
    return nil, err
}

And as a global variable you would define

var ErrNotEnoughFields = errors.New("not enough fields")

You can then test the error returned by allocateFile1Neo against this error value to detect if you should skip that line or if there is another kind of error which should be handled differently.

[kync (Ali Koyuncu)]

I see two solution for the problem, the first one is handling by error as @Christophe_Meessen answered, but if you have this error in the middle of the file you won’t be able to solve it. The second is read two lines every iteration, and if the second line is EOF skip it.

[MantasSilanskas]

@Christophe_Meessen

var ErrNotEnoughFields = errors.New(“not enough fields”)

i get this error when i apply your suggestion. It still reads that time stamp?

[MantasSilanskas]

I got idea following your suggestion

I only parse data if arr len is equal 11 that means when he comes to data stamp arr len is 1 and it skips it