AI Agents in Go
By lantern light, the Gopher sets a brass lantern on a square stone foot into the square socket of a stone pedestal, while the Ghost, a stone android, kneels holding a clay jug on the same square foot, and a bell and a bowl on the same foot wait on the step.

Go in depth

Interfaces and generics in a Go agent: small interfaces, wrappers, and typed tools

The shapes the rest of the code is built from: one-method interfaces, wrappers, and tools typed by their arguments.

Updated Intermediate

On the agent compass The whole loop, all four stages

An agent’s code is a few small parts that other code plugs into: a model it calls, tools it runs, and a policy that decides what each request holds. Go gives you two ways to make a part pluggable. An interface says what a value can do, and any type that can do it fits, with no declaration that it does. A generic function works for many types at once, and the compiler checks each use. This page shows both in the companion code, from the one-method interface the loop calls to tools whose JSON schema is generated from a Go struct.

One method: the Model interface

All the loop asks of a model is a reply to the conversation so far. That is the whole interface:

ch12/agent/types.go
// Model is anything that can produce the next reply in a conversation:
// a scripted fake, or a real model such as the OpenRouter client.
type Model interface {
	Generate(ctx context.Context, req Request) (Reply, error)
}

A type satisfies Model by having a Generate method with that signature, and nothing else. This is how every interface in Go works, and it is called implicit satisfaction: a type never declares which interfaces it implements, as a class does with implements in Java or C#, and the compiler checks the methods wherever the type is used as the interface. So a type can satisfy an interface written after it, in a package it has never heard of. In chapter 12’s code, four types produce replies that way, and the wrappers later on this page satisfy it too:

  • *agent.Fake, which replays a script of replies, for tests and for runs with no API key;
  • *openrouter.Client, the hand-written client for OpenRouter’s API;
  • *anthropic.Client, the same for Anthropic’s API;
  • *orsdk.Client, an adapter that puts OpenRouter’s official Go SDK behind the same method.

None of them mentions Model. The loop takes a Model and never learns which one it has, so a test can run the real loop with no network:

ch12/agent/fake.go
// Fake is a Model that replays a script of replies. It picks the next
// reply by counting the assistant messages already in the conversation,
// so it stays in step with the loop.
type Fake struct {
	Script []Message
}

// Generate returns the next scripted reply.
func (f *Fake) Generate(_ context.Context, req Request) (Reply, error) {
	n := 0
	for _, m := range req.Messages {
		if m.Role == "assistant" {
			n++
		}
	}
	if n >= len(f.Script) {
		return Reply{}, fmt.Errorf("reply %d: %w", n+1, ErrScriptExhausted)
	}
	return Reply{Message: f.Script[n]}, nil
}

The fake is short because the interface asks for one method. An interface with ten methods would need ten in every fake, and every new client would have ten chances to get something wrong. Keep interfaces small enough that writing a second implementation is easy; the standard library’s io.Reader and io.Writer, with one method each, are the usual examples.

Declare an interface where it is used

An interface belongs to the code that calls it, and it can list exactly what that code needs. The program in chapter 12’s main.go needs a client that can do two things, return a whole reply for a compaction summary and stream a reply for the loop, so it declares that, by embedding the two interfaces the agent package defines:

ch12/main.go
// A model is what the program needs from a client: a whole reply, for
// compaction summaries, and a reply in pieces, for the loop.
type model interface {
	agent.Model
	agent.StreamModel
}

The lowercase name keeps it private to main: nothing else needs it. Both hand-written clients satisfy it, because each has both methods. The SDK adapter has only Generate, so it isn’t a model, and the compiler says so if you pass one where a model is wanted.

The same file declares an interface for errors:

ch12/main.go
// retryable is an API error that says whether a retry might help, as
// both the OpenRouter client's and the Anthropic client's do.
type retryable interface {
	error
	Retryable() bool
}

The OpenRouter client’s *APIError and the Anthropic client’s are different types in different packages. Both have a Retryable method, so one check finds either: errors.AsType[retryable](err) searches the chain for any error with that method, and Errors in an agent shows how the chain is searched. Asking for behavior instead of a concrete type is what lets the check work for a client the code doesn’t import.

Adapters: from a whole reply to a stream and back

The streaming loop calls a second interface, from Streams in Go:

ch12/agent/stream.go
// A StreamModel produces the next reply as a stream of pieces. The
// stream ends after the last Delta, or with an error.
type StreamModel interface {
	Stream(ctx context.Context, req Request) iter.Seq2[Delta, error]
}

A scripted fake returns whole replies, and the SDK adapter can’t stream at all. Streamed turns any Model into a StreamModel whose stream holds the whole reply at once:

ch12/agent/stream.go
// Streamed turns a Model into a StreamModel whose stream holds the
// whole reply at once, so scripted models and saved replies work with
// the streaming loop.
func Streamed(m Model) StreamModel { return streamed{m} }

type streamed struct{ m Model }

func (s streamed) Stream(ctx context.Context,
	req Request) iter.Seq2[Delta, error] {
	return func(yield func(Delta, error) bool) {
		reply, err := s.m.Generate(ctx, req)
		if err != nil {
			yield(Delta{Usage: &reply.Usage}, err)
			return
		}
		msg := reply.Message
		ds := []Delta{{Text: msg.Content,
			ReasoningDetails: msg.ReasoningDetails,
			ReasoningFrom:    msg.ReasoningFrom}}
		for i, c := range msg.ToolCalls {
			ds = append(ds, Delta{Call: &CallDelta{Index: i, ID: c.ID,
				Name: c.Name, Arguments: string(c.Arguments)}})
		}
		ds = append(ds, Delta{FinishReason: reply.FinishReason,
			Usage: &reply.Usage})
		for _, d := range ds {
			if !yield(d, nil) {
				return
			}
		}
	}
}

Generator goes the other way, collecting a stream into one reply for calls that don’t stream:

ch12/agent/middleware.go
// Generator turns a StreamModel into a Model that collects each reply
// whole, so a wrapped model can serve calls that don't stream, such as
// chapter 8's compaction summary, and get the same retries.
func Generator(m StreamModel) Model { return generator{m} }

type generator struct{ m StreamModel }

func (g generator) Generate(ctx context.Context, req Request) (Reply,
	error) {
	return Collect(g.m.Stream(ctx, req))
}

Both follow the same pattern: an unexported struct that holds the value it adapts, a method that satisfies the other interface, and an exported function that returns the interface. The usual Go advice is to accept interfaces and return concrete types, and the code mostly does. An adapter is the exception: its struct is an implementation detail with nothing for a caller to use, so returning the interface keeps it out of the package’s API.

A function that is an interface

Some implementations are a single function. Go lets a function type have methods, so a function can satisfy an interface, as http.HandlerFunc does for http.Handler:

ch12/agent/middleware.go
// StreamFunc is a function that is a StreamModel, as http.HandlerFunc
// is a Handler, so a wrapper can return a closure.
type StreamFunc func(ctx context.Context,
	req Request) iter.Seq2[Delta, error]

// Stream calls f.
func (f StreamFunc) Stream(ctx context.Context,
	req Request) iter.Seq2[Delta, error] {
	return f(ctx, req)
}

StreamFunc(f) is a conversion, not a call: it gives the function f the type StreamFunc, whose Stream method calls f. With it, code that needs a StreamModel can write a closure instead of declaring a struct.

Wrappers: retries, spans, and logs around a model

A wrapper takes a model and returns another that does something before and after each call: retry it, trace it, log it, count its tokens. The agent package names the shape of a wrapper with two function types:

ch12/agent/middleware.go
// A ModelWare wraps a model: it takes the model a call goes to and
// returns one that does something before and after, such as a retry or
// a span. It takes and returns StreamModel, the interface the loop
// calls, so no wrapper can hide the stream (chapter 6's rule).
type ModelWare func(StreamModel) StreamModel

// A ToolWare wraps a tool. It sees the whole Tool, its definition and
// its safety flags as well as its Run, which is what a retry needs to
// know whether the tool is safe to run twice.
type ToolWare func(Tool) Tool

// WrapModel wraps m in ws, the first outermost: a call goes through
// ws[0], then ws[1], and so on, to m.
func WrapModel(m StreamModel, ws ...ModelWare) StreamModel {
	for _, w := range slices.Backward(ws) {
		m = w(m)
	}
	return m
}

// WrapTools wraps every tool in ws, the first outermost.
func WrapTools(tools []Tool, ws ...ToolWare) []Tool {
	out := slices.Clone(tools)
	for i := range out {
		for _, w := range slices.Backward(ws) {
			out[i] = w(out[i])
		}
	}
	return out
}

WrapModel applies them from the last to the first, so the first wrapper in the list is the outermost: a call passes through ws[0] first. Here is one, Traced, which records a span for each model call; it returns a closure through StreamFunc, and the closure calls the model it wraps:

ch12/agent/traced.go
// Traced returns a ModelWare that records each model call as a "chat"
// span under the span in the call's context, with the GenAI attributes.
// It goes outside the retries, so one span covers every attempt.
func Traced(rec *Recorder, provider, model string) ModelWare {
	return func(m StreamModel) StreamModel {
		return StreamFunc(func(ctx context.Context,
			req Request) iter.Seq2[Delta, error] {
			return func(yield func(Delta, error) bool) {
				ctx, span := rec.Start(ctx, opChat+" "+model,
					slog.String(attrOperation, opChat),
					slog.String(attrProvider, provider),
					slog.String(attrRequestModel, model))
				var err error
				defer func() { span.Finish(err) }()
				for d, e := range m.Stream(ctx, req) {
					spanDelta(span, d, e)
					if err = e; !yield(d, e) {
						return
					}
				}
			}
		})
	}
}

Chapter 12’s program stacks five of them, and the order is part of the design: the span for the whole call goes outside the retries, so one span covers every attempt, and the meter goes inside them, so a failed attempt’s tokens are counted too.

ch12/reliable.go
// Model wraps m: the chat span outside the retries, then the retries,
// then, for each attempt, its own span, the log line, and the meter.
func (w *wrapped) Model(m agent.StreamModel) agent.StreamModel {
	return agent.WrapModel(m,
		agent.Traced(w.rec, w.provider, w.model),
		agent.Retrying(w.policy),
		agent.TracedAttempts(w.rec),
		agent.Logged(w.log, w.model),
		w.meter.Wrap)
}

// Tools wraps tools: the span, the log line, retries for the tools safe
// to run twice, and each tool's Timeout. Tool retries have a policy of
// their own: a flaky tool says nothing about the provider.
func (w *wrapped) Tools(tools []agent.Tool) []agent.Tool {
	logs := agent.LogOptions{Redact: triage.Redact}
	return agent.WrapTools(tools,
		agent.TracedTools(w.rec),
		agent.LoggedTools(w.log, logs),
		agent.RetryTools(w.tools),
		agent.Timeouts)
}

The last one, w.meter.Wrap, is a method value. Meter.Wrap takes a StreamModel and returns one, so w.meter.Wrap, the method bound to w.meter, has exactly the type ModelWare and goes in the list with no adapter.

Tools are wrapped the same way, but a Tool is a struct, not an interface:

ch12/agent/types.go
// Tool is the harness's side of a tool: the definition the model sees,
// and the Go function that runs it. The model never sees Run.
type Tool struct {
	Def ToolDef
	Run func(ctx context.Context, args jsontext.Value) (string, error)

	// Safety flags (chapter 7). A tool with neither flag set may change
	// something outside the run.
	ReadOnly   bool // it only looks things up
	Idempotent bool // running it twice has the same effect as once

	// Final marks the tool that ends the run: once a call to it
	// succeeds, its arguments become Result.Final (chapter 7).
	Final bool

	// Timeout, if set, bounds one call to Run; a wrapper applies it
	// (chapter 12). Zero means the tool's own limits only.
	Timeout time.Duration
}

That is deliberate. A wrapper that retries a tool has to know whether running it twice is safe, which is what Idempotent says, and a wrapper that applies a time limit needs Timeout. An interface with only a Run method would hide both. So a ToolWare takes the whole struct and returns it with Run replaced, keeping the definition and the flags:

ch12/agent/tooltime.go
// Timeouts is a ToolWare that applies each tool's Timeout. A timeout
// the tool reports itself, such as an MCP client's own deadline, counts
// the same. A tool that ignores its context can't be stopped: Timeouts
// waits for it, and a result that arrives late still counts.
func Timeouts(t Tool) Tool {
	if t.Timeout <= 0 {
		return t
	}
	run := t.Run
	t.Run = func(ctx context.Context,
		args jsontext.Value) (string, error) {
		tctx, cancel := context.WithTimeoutCause(ctx, t.Timeout,
			ErrToolTimeout)
		defer cancel()
		out, err := run(tctx, args)
		if err == nil || ctx.Err() != nil ||
			!errors.Is(context.Cause(tctx), ErrToolTimeout) &&
				!timedOut(err) {
			return out, err
		}
		return "", timeoutError(t, err)
	}
	return t
}

Several implementations, and a slice that is one

What each request holds is decided by a ContextPolicy, from Context management:

ch12/agent/policy.go
// A ContextPolicy builds the messages one request sends from the full
// conversation. The loop keeps the full conversation in
// Result.Messages and never rewrites it; a policy only decides what the
// model sees. Its state, such as how much it has trimmed, lives in a
// ContextState that the loop holds and passes back each turn, so a
// policy keeps nothing between calls and the same conversation and
// state always build the same request.
type ContextPolicy interface {
	Build(ctx context.Context, msgs []Message, st ContextState) (Window, error)
}

// A Window is what a policy built: the request's messages, the state
// to pass back next turn, and what building it cost, such as a summary
// call.
type Window struct {
	Messages []Message
	State    ContextState
	Usage    Usage
}

// ContextState is a context policy's memory between turns. Each field
// is an index into the full conversation or text derived from it.
type ContextState struct {
	Cut     int    // Trim: messages before this are left out
	Cleared int    // ClearResults: tool results before this are cleared
	Summary string // Compact: the latest summary of earlier turns
	Covered int    // Compact: the summary covers messages before this
}

Trim, ClearResults, Compact, and CacheMarks each implement it, and a run takes whichever it is given. To use several, Chain returns a slice of policies that is itself a policy:

ch12/agent/policy.go
// Chain applies policies in order, each to the messages the one before
// it built. The state's indexes count the full conversation, so at
// most one policy in a chain may shorten it, Trim or Compact, and the
// policies before it must keep its length, such as ClearResults before
// Trim. Compact clears through its Sent field instead, so its summary
// reads whole results. CacheMarks goes last, to mark what is sent.
func Chain(ps ...ContextPolicy) ContextPolicy { return chain(ps) }

type chain []ContextPolicy

func (c chain) Build(ctx context.Context, msgs []Message,
	st ContextState) (Window, error) {
	if n := shortening(c); n > 1 {
		return Window{}, fmt.Errorf("chain: %d policies shorten the "+
			"conversation; use one, Trim or Compact", n)
	}
	w := Window{Messages: msgs, State: st}
	for i, p := range c {
		next, err := p.Build(ctx, w.Messages, w.State)
		if err != nil {
			return Window{Usage: w.Usage.Add(next.Usage)},
				fmt.Errorf("policy %d of %d: %w", i+1, len(c), err)
		}
		next.Usage = w.Usage.Add(next.Usage)
		w = next
	}
	return w, nil
}

chain is a named slice type with a Build method, so it satisfies ContextPolicy, and the loop calls it like any other policy. Its method runs each policy in turn on what the one before built.

Sometimes code needs to know which concrete types an interface holds. Chain refuses to hold two policies that both shorten the conversation, and finds them with a type switch:

ch12/agent/policy.go
// shortening counts the policies that shorten the conversation.
func shortening(ps []ContextPolicy) int {
	n := 0
	for _, p := range ps {
		switch p.(type) {
		case Trim, *Trim, Compact, *Compact:
			n++
		}
	}
	return n
}

Each type appears twice, as a value and as a pointer. Trim’s method has a value receiver, func (t Trim) Build, and a method with a value receiver belongs to both Trim and *Trim, so a caller could pass either and the switch has to catch both. A method with a pointer receiver belongs to the pointer only, which is why *agent.Fake is a Model and agent.Fake is not.

Optional behavior: ask with a type assertion

A function can take a small interface and still use more when a value offers it. Chapter 15’s audit log writes each entry to an io.Writer; when that writer is a file, it also asks the operating system to flush the entry to disk, so a crash can’t lose it:

ch15/safety/audit.go
if s, ok := a.W.(interface{ Sync() error }); ok {
	if err := s.Sync(); err != nil {
		return fmt.Errorf("audit: %w", err)
	}
}

a.W.(interface{ Sync() error }) asks whether the value in a.W also has a Sync method. An *os.File does, so the entry is synced; the bytes.Buffer a test passes doesn’t, and the check is skipped. The interface type is written inline, with no name, because nothing else needs it. The standard library works the same way: io.Copy checks whether its source is an io.WriterTo.

Checking at compile time

Go checks that a type satisfies an interface only where the type is used as one. To check it earlier, assign a value to a blank variable of the interface type:

ch12/main_test.go
var _ agent.ContextPolicy = agent.CacheMarks{}

The line does nothing when it runs. If CacheMarks ever loses its Build method, or the method’s signature changes, the code stops compiling here, with an error that names the missing method, instead of somewhere far from the cause.

Generics: one function for every tool’s arguments

Each tool decodes its own JSON arguments into its own Go type. Without generics, every tool would repeat the decoding and the error the model reads when it fails. With a type parameter, one function does it for every type:

ch12/agent/tools.go
// decodeArgs decodes a tool call's arguments. When they don't decode,
// the error says what was wrong and shows the shape the tool expects,
// so the model can fix its next call.
func decodeArgs[T any](args jsontext.Value, example string) (T, error) {
	var v T
	err := json.Unmarshal(args, &v, json.RejectUnknownMembers(true))
	if err != nil {
		return v, &toolError{ErrToolArgs, fmt.Sprintf(
			"bad arguments: %s; send JSON like %s", argError(err), example)}
	}
	return v, nil
}

T is a type parameter: the caller picks it, as in decodeArgs[readArgs](args, example), and the compiler makes a version of the function where T is that type. any is the constraint, the set of types T may be; any allows every type, which is all this function needs, since it only declares a T and passes its address to json.Unmarshal.

Chapter 7 goes further: the tool’s whole definition comes from its argument type. A tool’s arguments are a struct, with a tag for each field’s description:

ch12/triage/codetools.go
type readArgs struct {
	Path  string `json:"path" desc:"a file path, such as store/store.go"`
	Start int    `json:"start,omitzero" desc:"the first line to show; default 1"`
	End   int    `json:"end,omitzero" desc:"the last line to show; default 200 lines after start"`
}

NewTool takes the tool’s Go function and generates the JSON schema the model sees from the struct, so the schema and the code that reads the arguments can’t drift apart:

ch12/agent/registry.go
// NewTool makes a Tool from a Go function whose arguments are the
// fields of a struct, A. The schema the model sees is generated from A,
// so the two can't drift apart. NewTool panics on a type that can't be
// a tool's arguments, which is a programmer's mistake.
func NewTool[A any](name, description string,
	run func(ctx context.Context, args A) (string, error)) Tool {
	t := reflect.TypeFor[A]()
	if t.Kind() != reflect.Struct {
		panic(fmt.Sprintf("agent: tool %s: arguments must be a struct",
			name))
	}
	s, err := schemaFor(t, nil)
	if err != nil {
		panic(fmt.Sprintf("agent: tool %s: %v", name, err))
	}
	params, err := json.Marshal(s)
	if err != nil {
		panic(fmt.Sprintf("agent: tool %s: %v", name, err))
	}
	return Tool{
		Def: ToolDef{Name: name, Description: description,
			Parameters: params, Strict: strictOK(s)},
		Run: func(ctx context.Context, args jsontext.Value) (string, error) {
			a, err := decodeStrict[A](args, s)
			if err != nil {
				return "", err
			}
			return run(ctx, a)
		},
	}
}

Three things happen in it. reflect.TypeFor[A]() gets the struct’s type, which is how the schema is built at run time from the fields and their tags. The function the caller passes takes an A, so inside it a.Path is a string and a.Start an int, checked by the compiler, with no type assertion. And the Tool it returns isn’t generic: its Run takes raw JSON, and the closure decodes it into an A before calling the typed function.

That last point is what makes it work. The loop holds its tools in a []Tool, and a slice holds one type. If Tool were generic, Tool[readArgs] and Tool[searchArgs] would be different types and couldn’t share the slice. So the type parameter lives only in the constructor, and the closure carries the type inside.

You rarely write the type argument yourself. In agent.NewTool("code_read", "...", func(_ context.Context, a readArgs) (string, error) {...}), Go infers A from the function’s parameter. For the read tool in the package’s tests, with a required path and an optional start, the generated schema is this, checked byte for byte by TestSchema:

JSON
{"type":"object","properties":{"path":{"type":"string","description":"a file path, relative to the project"},"start":{"type":"integer","description":"the first line"}},"required":["path"],"additionalProperties":false}

The omitzero on start is what leaves it out of required.

Constraints that say more than any

A constraint is an interface, and it can list types or methods. The code that summarizes the recorded experiments takes the median of ints in one place and of float64s in another, with one function:

evidence/summary.go
func median[T int | float64](xs []T) T {
	if len(xs) == 0 {
		return 0
	}
	s := slices.Sorted(slices.Values(xs))
	if len(s)%2 == 1 {
		return s[len(s)/2]
	}
	return (s[len(s)/2-1] + s[len(s)/2]) / 2
}

int | float64 is a union: T may be either, and since both support +, /, and ordering, the body may use them. With any, the compiler would reject s[i] + s[j], because not every type can be added.

A constraint can also require a method, written inline:

evidence/experiments.go
// next returns the number of the first new run for model, after the
// runs already saved, so runs can be added later.
func next[T interface{ model() string }](file, model string) (int, error) {
	saved, err := load[T](file)
	if err != nil {
		return 0, err
	}
	n := 1
	for _, r := range saved {
		if r.model() == model {
			n++
		}
	}
	return n, nil
}

Each kind of saved run is its own struct type, and each has a model method that says which model it recorded. next loads any of them and counts the runs for one model, calling r.model() on each.

Reach for a type parameter when the function’s types matter to its caller, as in decodeArgs, which returns a T, or median, which returns the type it was given. When a function only calls methods on a value and returns nothing of its type, an interface parameter says the same thing more simply.

Mistakes to avoid

  • A large interface written before its users. Declare an interface where it is called, with the methods that code needs; the implementer exports a concrete type.
  • An interface for one type that will never have a second. A test fake counts as a second, and Fake is why Model exists; a type no one will ever substitute doesn’t need one.
  • A nil pointer in an interface. An interface holding a nil *T is not nil, so err != nil is true. Errors in an agent shows the case.
  • A pointer receiver, then a value where the interface is wanted. *Fake has the method and Fake doesn’t, so passing agent.Fake{} as a Model fails to compile. Pass &agent.Fake{}.
  • A generic type where the values must share a slice. Tool[A] for each A can’t go in one []Tool; keep the type parameter in a constructor, as NewTool does.
  • Wrappers in the wrong order. In WrapModel, the first wrapper is the outermost. A span placed inside the retries records each attempt; placed outside them, it records the whole call once.

Try it

From the code directory of a clone of the companion repository:

terminal
$ go test ./ch12/agent -run 'TestSchema$|TestDecodeStrict|TestGenerator$|TestChainShortening|TestTimeouts' -v
  1. In TestSchema in ch12/agent/registry_test.go, add a field End int with tags json:"end,omitzero" desc:"the last line" to readArgs, and run the test. It fails and prints the new schema, with end in properties and not in required.
  2. Change the Fake’s Generate in ch12/agent/fake.go to have a value receiver, func (f Fake), and run go vet ./ch12/.... Every &agent.Fake{} still compiles, since a value receiver belongs to the pointer too; the methods of Fake are now in both method sets.
  3. In ch12/main_test.go, add var _ agent.StreamModel = &agent.Fake{} and run go vet ./ch12. The compiler refuses it and names the missing Stream method; agent.Streamed(&agent.Fake{}) is the value that fits.

Undo your edits with git restore . in the code directory.