Go's flag Package, Creating a CLI Tool

Go's flag package makes the development of a command line interface tool very simple.

In an attempt to learn more about Go and the GNU Coreutils, I created a Go program which emulates the yes utility with some added functionality. I hope this post provides enough information to inspire readers to start writing their own.

How This Post is Structured

There are three sections of the post that discuss the flag package and development of the yes tool. Because this post is bulky from code examples and explanations, click here to start a trail that mentions only flag package content; proceed to each section through links that begin with "Skip".

Refer to the code examples and their explanations for a thorough experience.

Background Info

The flag package provides functionality for parsing the flags that are provided when executing the utility. We will need this to parse the options/flags.

The yes utility will print the space separated arguments followed by a space and newline, repeating until interrupted. If there are no arguments, the character 'y' is considered output.

Notable Things You Will Learn of the flag Package

  • Two main ways to define a flag.
  • How to define your own flag in a third way using the flag.Var function.
  • The flag.Flag data type.
  • Handling arguments after flags are parsed.

My yes Implementation Differences - This Go implementation of the yes util will include a limit flag to specify number of times the output string will be displayed.

Contents

  1. Usage without arguments.

  2. Using the flag package to parse flags and arguments.

  3. Putting everything together; basic usage.

1. Usage Without Arguments

As specified within the docs, executing the yes tool without any arguments will simply display the character y followed by a space and newline. Here's what that looks like:

usage_without_arguments.go

package main

import (  
    "fmt"
    "os"
)

func main() {  
    for true {
        if len(os.Args) == 1 {
            fmt.Printf("y \n")
        } else {
            break;
        }
    }
}

Explanation

The endless loop outputs "y " on a single line if there aren't any command line arguments. How is this determined? Well, len(os.Args) will always be 1 for a running, argumentless program. In any case, recall that os.Args[0] is a string value which represents the running program.

NOTE It is possible to calculate the number of arguments (excluding program, starting at 0) using len(flag.Args()). flag.Args() returns a string slice of the arguments after the flags have been parsed. Remember, flag.Parse(), explained in the next section, must be executed before this package variable may be used. See usage_without_arguments_using_flagArgs.go for an example.

2. Using the flag Package to Parse Flags and Arguments

Let's review the three flags and the type of each one:

  • version - boolean flag, displays current version of utility.

  • help - boolean flag, displays text information of basic usage.

  • limit - integer flag, determines the number of times that the string argument(s) will be displayed.

The Version flag and a thorough overview of the flag package


Flag package basics - Basically, every flag needs to be defined. Functions in the flag package are used to do so, enabling a variable to contain the flag value or point to a variable containing the value. These two different ways will be covered.

After doing so, the flag.Parse() function must be called to do the processing - validating received flag values, placing them, etc.. Variables which contain or point to flag values are useless until this function is called.

The explanation section shows how the version flag was implemented, a look at the flag.Flag struct, and how the to access the flag.Flag struct fields using flag.Lookup.

Skip to 'Alternative Way of Defining a Flag'

version_flag.go

package main

import (  
    "flag"
    "fmt"
    "os"
)

func main() {  
    var version *bool
    version = flag.Bool("version", false, "0.1")

    flag.Parse()

    if *version {
        fmt.Printf("Version %s\n", (flag.Lookup("version")).Usage)
        os.Exit(0)
    }
}

Explanation

We declare a variable version, which is of type pointer to a boolean.

As the documentation describes flag.Bool:

BoolVar defines a bool flag with specified name, default value, and usage string. The argument p points to a bool variable in which to store the value of the flag.

Let's go through the lines of interest.


`version = flag.Bool("version", false, "0.1")`

flag.Bool defines a flag with a name of version which has a default value of false and a usage string "0.1"

Now that the flag is defined, flag.Parse() is executed and works its magic.


if *version {  
...
}

In the conditional expression of the if statement, we dereference the pointer variable version whose code block will be executed based on the dereferenced value.


flag.Lookup("version")).Usage  

The argument to the Printf statement is simply evaluated to "0.1". Understanding how requires more explanation of the package. I'll start by mentioning the flag.Flag type:

type Flag struct {  
        Name     string // name of flag
        Usage    string // help message
        Value    Value  // value as set
        DefValue string // default value as text
}

When flag.Bool was called, a flag was defined in the name of the first argument (important in next paragraph!) and memory was allocated for a flag.Flag in which its fields were populated with the provided arguments.

Calling flag.Lookup and providing it with a string literal of the name of a defined flag ("version") returns the reflecting flag.Flag.

Thus, we simply append .Usage to first dereference the returned address and then retrieve Usage field of the struct, which is indeed the string "0.1". Cool, huh!?!

Sample Output

$ ./version_flag
>

$ ./version_flag -version
> Version 0.1

$ ./version_flag --version
> Version 0.1

$ ./version_flag --version=false
>

$ ./version_flag --version=123
>invalid boolean value "123" for -version: strconv.ParseBool: parsing "123": invalid syntax
Usage of ./version_flag:  
  -version=false: 0.1

Refer to the package notes for some explanations for the output.

Now that this is out of the way, on to implementing the help flag!

The help Flag, More on the flag Package

The last section was the most intense part of this post. Implementation of the help flag is similar but with minor differences. Like the version flag, the help flag will be used to simply display information.

Alternative Way of Defining a Flag - The implementation of the help flag demonstrates a flag package function that enables the variable to be of type bool using flag.BoolVar rather than a pointer using flag.Bool. See the explanation for more detailed info.

Skip to 'Remaining Flags, Defining Your Own'

help_flag.go

package main

import (  
    "flag"
    "fmt"
    "os"
)

func main() {

    var help bool

    var helpText = /*help text omitted for readability, shown in output instead.*/

    flag.BoolVar(&help, "help", false, helpText)

    flag.Parse()

    if help {
        fmt.Printf("%s\n", flag.Lookup("help").Usage)
        os.Exit(0)
    }
}

Explanation

The minor difference in implementing this flag is the use of a different function and variable type.


var help bool  

Unlike the version flag implementation, help is of type bool and not a pointer to a bool. The flag.BoolVar function is different from the flag.Bool function by its additional, first argument - a pointer to a bool. In other words, the return value of flag.Bool is shoved into flag.BoolVar as an argument (See bullet A in "Notes on the Flag Package" for more details).

What does this mean? The difference is simply in the type and usage of the variable defined to store the flag value. For the version flag it was a pointer to a boolean, while this help flag is actually of type boolean. Obviously the help variable does not need to be dereferenced.

Sample Output (modified for readability)

$ ./help_flag

> 
Usage:

--help               displays usage information

--version            displays utility version 

--limit              specifies number of times arguments will be displayed.

Remaining Flags, Defining Your Own

At this point you are more than capable of creating your own CLI tool.

The implementation of the last flag limit demonstrates how a custom flag can be created.

Basically, use the flag.Var function:

Var(value Value, name string, usage string)

The first argument is of interface type Value

type Value interface {  
        String() string
        Set(string) error
}

All that's needed for the custom flag is to satisfy the two methods. String (String() string) returns a string value of an argument, and Set (Set(string) error) is responsible for processing and placing the string argument(s) into the data type which will contain it.

To retrieve the contents of a custom flag, we should satisfy the Getter interface. It is simply a wrapper of Value and includes one additional, aptly named Get function. That's about it!

Skip to 'Parsing Non Flag Arguments'

The following section is an example of how a custom flag could be designed, view limit_flag_with_yes.go for a practical example.

limit_flag.go

package main

import(  
"fmt"  
"flag"  
"strconv"  
)

type CustomStruct struct {  
    FlagValue int
}

func (cF *CustomStruct) String() string {  
    return strconv.Itoa(cF.FlagValue)
}

func (cF *CustomStruct) Set(s string) error {  
    cF.FlagValue, _ = strconv.Atoi(s)

    return nil
}

func (cF *CustomStruct) Get() int {  
    return cF.FlagValue
}

func (cF *CustomStruct) IsSet() bool {  
    if cF.FlagValue == -1 {
        return false
    }

    return true
}

func main() {  
    limitFlag := CustomStruct{-1}

    flag.Var(&limitFlag, "limit", "Limits output")

    flag.Parse()

    if limitFlag.IsSet() {
        fmt.Printf("\nLimit: %d\n\n", limitFlag.Get())
    } else {
        fmt.Printf("\nLimit flag not included.\n\n")
    }
}

Explanation

A struct was defined, CustomStruct, which contains an integer field. Methods Set and String are defined which satisfies the Value interface, and Get is defined which satisfies the Getter interface. Note that an extra method was defined, IsSet.

Let's take a look at the Set, String, and IsSet methods.


The Set Method

func (cF *CustomStruct) Set(s string) error {  
    cF.FlagValue, _ = strconv.Atoi(s)

    return nil
}

This method is called by the flag.Var function, used to parse the received string argument and place the value(s) in a container. In this case, we simply just convert the string to an integer and store it accordingly into the FlagValue field.


The String Method

func (cF *CustomStruct) String() string {  
    return strconv.Itoa(cF.FlagValue)
}

This method simply returns a string value used to describe the contents of the flag value. Nothing else is needed in this case other than a conversion, but think about how this could be used to describe a flag value of a slice of strings or something...


The IsSet Method

func (cF *CustomStruct) IsSet() bool {  
    if cF.FlagValue == -1 {
        return false
    }

    return true
}

Notice that the flag.Var function doesn't have an argument to specify a default value. I implemented this method to determine when the flag has been set. -1 was used since valid flag values can only be 0 or greater.

In order for this method to behave correctly, the flag value must be initialized with the value -1, as seen in the main function.

Now, on to the final part...

3. Putting it All Together, Parsing Remaining Arguments

Parsing Non Flag Arguments -After flag.Parse is called, use the flag.Args function to retrieve a string slice of the arguments. Remember that flag.Parse must be called first! See parsing_arguments.go for an example.

Skip to 'Conclusion'

Yes In Action

The final product, yes.go incorporates all of the material that has been covered.

yes.go sample output

The help option was omitted for readibility but works as it should. the ... denotes infinite output.

$ ./yes
>y 
y  
...

$ ./yes cat    dog mouse
> cat dog mouse
cat dog mouse  
...

$ ./yes -version
>Version 0.1

$ ./yes -limit 2
>y
y

$ ./yes -limit 3 duck duck goose
>duck duck goose
duck duck goose  
duck duck goose  

Conclusion

You have now learned enough about the flag package to:

  • Define and parse flags
  • Define your own custom flags
  • Parse non flag arguments

If you've made it this far, thanks so much. While this is the lengthiest of all the articles, I hope that every bit of it was useful and entertaining.

This post in particular has led me to think about the way future posts should be written in terms of length and design - I'd especially appreciate feedback on these aspects. Until next week!


Notes on the flag package

  • Including Flags As the documentation notes, when providing a flag, - and -- are equivalent.

  • Flag Syntax The documentation mentions the accepted syntax for including flags:
    -flag -flag=x -flag x

    The third option is not permitted for boolean flags.

    Acceptable integer flag values:

    9876 00239 0x4321 -1999

    Acceptable boolean flag values:

    1, 0, t, f, T, F, true, false, TRUE, FALSE, True, False

  • One Variable, Multiple Flags

    It is possible to define more than one flag name for a variable.

    For instance, we can implement a shorthand version of a flag 'crash'.

    Flag Definition flag.IntVar(&randomFlag, "crash", -1, "foobarbaz")

    Shorthand Flag Definition flag.IntVar(&randomFlag, "c", -1, "foobarbaz")

  • Similar Functions The flag package contains multiple functions that are "pairs" (Int, IntVar, Float64, Float64Var).

    I've pondered why both implementations need to exist and counted a total of eight "pairs" - eight functions in the package which differs its counterpart by only one argument. At first this seemed to me like clutter.

    I can only explain the need with a couple of reasons and would appreciate your input.

    Imagine that the values of flags all needed to be evaluated as a whole (not uncommon). Instead of dealing with different data types, it would be more manageable for every flag to be represented through a pointer.

Other Notes

  • Fun Fact - yes recevied publicity in 2006 for being used to detect a random shutdown defect in Macbooks. Source :Wikipedia

  • POSIXLY_CORRECT - As noted in recent coreutils news (Section starting on line 3469), yes parses the --help and --version flags as strings unless the POSIXLY_CORRECT environment variable is set (see for more information on what this means). Initially decided to implement a flag for this but decided not to in the interest of value. Still worth a quick read: http://en.wikipedia.org/wiki/POSIX

Unrelated Notes

  • Looking into coreutils did yield some pretty great resources: Exit Codes, Common Command Line Options

  • I mistakenly thought that I had to emulate exit signals and found this awesome Go package which I will probably make a post on. os/signal

  • In retrospect, I probably should have broken this post up into chunks to make this more digestable. This developed into a chapter or large section of a book rather than a blog post.

  • I'm very proud of how this post turned out, given the amount of time and effort that was put into it...