Removing context package's dependency on fmt

Technical Discussion
1

The ‘context’ package imports the ‘fmt’ package to format three strings. The ‘fmt’ package then pulls in a number of other packages so it’s rather large (~500K in the executable, unstripped). So if you import ‘net’, you get ‘context’, then you get ‘fmt’, but it would be nice to make smaller executables for embedded devices that just use the ‘net’ package.

I think it could be a useful optimization to remove the dependency on ‘fmt’ and just generate those strings manually somehow, although there might be some work to do if the current behavior needs to be emulated as %v is used with some structs, for example.

For starters, would anyone else find this optimization useful?

2

Do you know Toward Go 2 which describes how such proposals will be dealt with?

3

Roughly, yes, I read the blog entry earlier. So I’ve identified a problem and a vague solution, but I would like to know if anyone else feels this is actually a problem.

For me, I might be able to fit my latency measuring utility on a MIPS based WiFi AP with limited flash space running LEDE, but that’s just my specific use case.

4

It’s not a problem for me personally, but there are some significant contortions already in the standard library to avoid internal dependencies. I’m mildly surprised something as small as context would depend on something as large as fmt. I’d be happy to see this fixed.

1 Like
5

Thanks for that. One thing I’m realizing though is that I’m not sure of the actual delta of removing this dependency, because while ‘fmt’ itself is pretty sizable (how to quantify that), any other packages that ‘fmt’ imports that ‘net’ already imports anyway would not be included in the size reduction.

Short of re-building Go with a modified ‘net’ package and comparing executables with and without the dependency on ‘fmt’, is there a way to know the actual size difference?

So far, I look at the dependency trees with a quick and dirty utility I wrote that calls ‘go list’ recursively starting at a specified root package or the current directory, but it doesn’t have the smarts to do a delta between two dependency trees, nor does it (can it?) show package sizes / totals in the final executable. (One can also list dependencies recursively using go list with .Deps, but I wanted to see the hierarchy also.)

6

You should be able to just yank fmt out of context (replace the formatting with string constants or something) and see what happens.

7

Please start by raising an issue, golang.org/issue/new. Please don’t jump to writing code before the discussion on the issue is completed.

8

Hmm, I tried it and I’m surprised by an unimpressive 144 byte size reduction:

1687296 Aug 23 23:13 hello_net_1.8.3
1687152 Aug 23 23:17 hello_net_1.8.3_nofmt

My bare bones test program:

package main

import "net"

func main() {
	_ = net.IPv4len
}

Then my “nofmt” version of Go replaces the three lines in context.go that use ‘fmt’ with a hardcoded string for now. I’ve indeed removed the dependency, as I compared a ‘go list’ diff between both versions:

% go list -f '{{ join .Deps "\n" }}' > go_list_deps_1.8.3.txt
% # switch Go to the nofmt version
% go list -f '{{ join .Deps "\n" }}' > go_list_deps_1.8.3_nofmt.txt
% diff go_list_deps_1.8.3.txt go_list_deps_1.8.3_nofmt.txt
3d2
< fmt

So for now I guess I can chalk this up to a failed idea, although I’m unclear on why the difference is that small.

9

Thanks, I hadn’t noticed your post before my quick test Dave. I could raise an issue anyway, but it might just turn into an (maybe informative) discussion of why this wouldn’t help.

10

I suspect that the big “unnecessary” things in that program are unicode and unicode/utf8 which get pulled in by reflect via context and sort. I don’t think that’s trivial to remove.

This is a neat tool: GitHub - kisielk/godepgraph: A Go dependency graph visualization tool

I think the code writing so far was just to confirm whether this is indeed an issue or not. It looks like not. :slight_smile:

11

To clarify, this is my standard response to PR’s without issues. The intent is to avoid hurt feelings (on all sides) when time is lost due to misunderstandings.

2 Likes
12

I still would have thought that 2201 lines of source (from ‘fmt’) would compile to more than just 144 bytes.

% sloc fmt/**/*.go~*test.go
            Physical :  3176
              Source :  2201
             Comment :  813
 Single-line comment :  490
       Block comment :  323
               Mixed :  47
               Empty :  261
               To Do :  1
Number of files read :  4

% sloc unicode/**/*.go~*test.go
            Physical :  10204
              Source :  9393
             Comment :  1004
 Single-line comment :  1004
       Block comment :  0
               Mixed :  622
               Empty :  429
               To Do :  1
Number of files read :  8

The diff from ‘bloaty’ after removing fmt:

% bloaty hello_net_1.8.3_nofmt -- hello_net_1.8.3     
     VM SIZE                                FILE SIZE
 ++++++++++++++ GROWING                  ++++++++++++++
   +15%    +740 [None]                   +1.01Ki  +1.0%

 -------------- SHRINKING                --------------
  -0.1%    -416 __TEXT,__text               -416  -0.1%
  -0.1%    -259 __TEXT,__gopclntab          -259  -0.1%
  -0.1%    -167 __DWARF,__debug_info        -167  -0.1%
  -0.2%    -152 __DWARF,__debug_line        -152  -0.2%
  -0.6%     -75 __DWARF,__debug_pubnames     -75  -0.6%
  -0.0%     -65 __TEXT,__rodata              -65  -0.0%
  -0.1%     -64 __DATA,__bss                   0  [ = ]
  -0.1%     -48 __DWARF,__debug_frame        -48  -0.1%

  -0.0%    -506 TOTAL                       -144  -0.0%

I guess it’s academic at this point, but it’s surprising how small this difference is, and may just have to do with the way executables are built.

But thanks for the tips…

13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.