Here's where we started this book:

Functions that run with go are called goroutines. The Go runtime juggles these goroutines and distributes them among operating system threads running on CPU cores. Compared to OS threads, goroutines are lightweight, so you can create hundreds or thousands of them.

That's generally correct, but it's a little too brief. In this chapter, we'll take a closer look at how goroutines work. We'll still use a simplified model, but it should help you understand how everything fits together.

Concurrency • Goroutine scheduler • GOMAXPROCS • Concurrency primitives • Scheduler metrics • Profiling • Tracing • Keep it up

Concurrency

At the hardware level, CPU cores are responsible for running parallel tasks. If a processor has 4 cores, it can run 4 instructions at the same time — one on each core.

  instr A     instr B     instr C     instr D
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│ Core 1  │ │ Core 2  │ │ Core 3  │ │ Core 4  │ CPU
└─────────┘ └─────────┘ └─────────┘ └─────────┘

At the operating system level, a thread is the basic unit of execution. There are usually many more threads than CPU cores, so the operating system's scheduler decides which threads to run and which ones to pause. The scheduler keeps switching between threads to make sure each one gets a turn to run on a CPU, instead of waiting in line forever. This is how the operating system handles concurrency.

┌──────────┐              ┌──────────┐
│ Thread E │              │ Thread F │              OS
└──────────┘              └──────────┘
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Thread A │ │ Thread B │ │ Thread C │ │ Thread D │
└──────────┘ └──────────┘ └──────────┘ └──────────┘
     │           │           │           │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Core 1   │ │ Core 2   │ │ Core 3   │ │ Core 4   │ CPU
└──────────┘ └──────────┘ └──────────┘ └──────────┘

At the Go runtime level, a goroutine is the basic unit of execution. The runtime scheduler runs a fixed number of OS threads, often one per CPU core. There can be many more goroutines than threads, so the scheduler decides which goroutines to run on the available threads and which ones to pause. The scheduler keeps switching between goroutines to make sure each one gets a turn to run on a thread, instead of waiting in line forever. This is how Go handles concurrency.

┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐
│ G15 ││ G16 ││ G17 ││ G18 ││ G19 ││ G20 │
└─────┘└─────┘└─────┘└─────┘└─────┘└─────┘
┌─────┐      ┌─────┐      ┌─────┐      ┌─────┐
│ G11 │      │ G12 │      │ G13 │      │ G14 │      Go runtime
└─────┘      └─────┘      └─────┘      └─────┘
  │            │            │            │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Thread A │ │ Thread B │ │ Thread C │ │ Thread D │ OS
└──────────┘ └──────────┘ └──────────┘ └──────────┘

The Go runtime scheduler doesn't decide which threads run on the CPU — that's the operating system scheduler's job. The Go runtime makes sure all goroutines run on the threads it manages, but the OS controls how and when those threads actually get CPU time.

Goroutine scheduler

The scheduler's job is to run M goroutines on N operating system threads, where M can be much larger than N. Here's a simple way to do it:

1. Put all goroutines in a queue.
2. Take N goroutines from the queue and run them.
3. If a running goroutine gets blocked (for example, waiting to read from a channel or waiting on a mutex), put it back in the queue and run the next goroutine from the queue.

Take goroutines G11-G14 and run them:

┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐
│ G15 ││ G16 ││ G17 ││ G18 ││ G19 ││ G20 │          queue
└─────┘└─────┘└─────┘└─────┘└─────┘└─────┘
┌─────┐      ┌─────┐      ┌─────┐      ┌─────┐
│ G11 │      │ G12 │      │ G13 │      │ G14 │      running
└─────┘      └─────┘      └─────┘      └─────┘
  │            │            │            │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Thread A │ │ Thread B │ │ Thread C │ │ Thread D │
└──────────┘ └──────────┘ └──────────┘ └──────────┘

Goroutine G12 got blocked while reading from the channel. Put it back in the queue and replace it with G15:

┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐┌─────┐
│ G16 ││ G17 ││ G18 ││ G19 ││ G20 ││ G12 │          queue
└─────┘└─────┘└─────┘└─────┘└─────┘└─────┘
┌─────┐      ┌─────┐      ┌─────┐      ┌─────┐
│ G11 │      │ G15 │      │ G13 │      │ G14 │      running
└─────┘      └─────┘      └─────┘      └─────┘
  │            │            │            │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Thread A │ │ Thread B │ │ Thread C │ │ Thread D │
└──────────┘ └──────────┘ └──────────┘ └──────────┘

But there are a few things to keep in mind.

Starvation

Let's say goroutines G11–G14 are running smoothly without getting blocked by mutexes or channels. Does that mean goroutines G15–G20 won't run at all and will just have to wait (starve) until one of G11–G14 finally finishes? That would be unfortunate.

That's why the scheduler checks each running goroutine roughly every 10 ms to decide if it's time to pause it and put it back in the queue. This approach is called preemptive scheduling: the scheduler can interrupt running goroutines when needed so others have a chance to run too.

System calls

The scheduler can manage a goroutine while it's running Go code. But what happens if a goroutine makes a system call, like reading from disk? In that case, the scheduler can't take the goroutine off the thread, and there's no way to know how long the system call will take. For example, if goroutines G11–G14 in our example spend a long time in system calls, all worker threads will be blocked, and the program will basically "freeze".

To solve this problem, the scheduler starts new threads if the existing ones get blocked in a system call. For example, here's what happens if G11 and G12 make system calls:

┌─────┐┌─────┐┌─────┐┌─────┐
│ G17 ││ G18 ││ G19 ││ G20 │                        queue
└─────┘└─────┘└─────┘└─────┘

┌─────┐      ┌─────┐      ┌─────┐      ┌─────┐
│ G15 │      │ G16 │      │ G13 │      │ G14 │      running
└─────┘      └─────┘      └─────┘      └─────┘
  │            │            │            │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Thread E │ │ Thread F │ │ Thread C │ │ Thread D │
└──────────┘ └──────────┘ └──────────┘ └──────────┘

┌─────┐      ┌─────┐
│ G11 │      │ G12 │                                syscalls
└─────┘      └─────┘
  │            │
┌──────────┐ ┌──────────┐
│ Thread A │ │ Thread B │
└──────────┘ └──────────┘

Here, the scheduler started two new threads, E and F, and assigned goroutines G15 and G16 from the queue to these threads.

When G11 and G12 finish their system calls, the scheduler will stop or terminate the extra threads (E and F) and keep running the goroutines on four threads: A-B-C-D.

This is a simplified model of how the goroutine scheduler works in Go. If you want to learn more, I recommend watching the talk by Dmitry Vyukov, one of the scheduler's developers: Go scheduler: Implementing language with lightweight concurrency (video, slides)

GOMAXPROCS

We said that the scheduler uses N threads to run goroutines. In the Go runtime, the value of N is set by a parameter called GOMAXPROCS.

The GOMAXPROCS runtime setting controls the maximum number of operating system threads the Go scheduler can use to execute goroutines concurrently (not counting the goroutines running syscalls). It defaults to the value of runtime.NumCPU, which is the number of logical CPUs on the machine.

Strictly speaking, runtime.NumCPU is either the total number of logical CPUs or the number allowed by the CPU affinity mask, whichever is lower. This can be adjusted by the CPU quota, as explained below.

For example, on my 8-core laptop, the default value of GOMAXPROCS is also 8:

maxProcs := runtime.GOMAXPROCS(0) // returns the current value
fmt.Println("NumCPU:", runtime.NumCPU())
fmt.Println("GOMAXPROCS:", maxProcs)

NumCPU: 8
GOMAXPROCS: 8

You can change GOMAXPROCS by setting GOMAXPROCS environment variable or calling runtime.GOMAXPROCS():

// Get the default value.
fmt.Println("GOMAXPROCS default:", runtime.GOMAXPROCS(0))

// Change the value.
runtime.GOMAXPROCS(1)
fmt.Println("GOMAXPROCS custom:", runtime.GOMAXPROCS(0))

GOMAXPROCS default: 8
GOMAXPROCS custom: 1

You can also undo the manual changes and go back to the default value set by the runtime. To do this, use the runtime.SetDefaultGOMAXPROCS function (Go 1.25+):

GOMAXPROCS=2 go run nproc.go

// Using the environment variable.
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))

// Using the manual setting.
runtime.GOMAXPROCS(4)
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))

// Back to the default value.
runtime.SetDefaultGOMAXPROCS()
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))

GOMAXPROCS: 2
GOMAXPROCS: 4
GOMAXPROCS: 8

CPU quota

Go programs often run in containers, like those managed by Docker or Kubernetes. These systems let you limit the CPU resources for a container using a Linux feature called cgroups.

A cgroup (control group) in Linux lets you group processes together and control how much CPU, memory, and network I/O they can use by setting limits and priorities.

For example, here's how you can limit a Docker container to use only four CPUs:

docker run --cpus=4 golang:1.24-alpine go run /app/nproc.go

// /app/nproc.go
maxProcs := runtime.GOMAXPROCS(0) // returns the current value
fmt.Println("NumCPU:", runtime.NumCPU())
fmt.Println("GOMAXPROCS:", maxProcs)

Before version 1.25, the Go runtime didn't consider the CPU quota when setting the GOMAXPROCS value. No matter how you limited CPU resources, GOMAXPROCS was always set to the number of logical CPUs on the host machine:

docker run --cpus=4 golang:1.24-alpine go run /app/nproc.go

NumCPU: 8
GOMAXPROCS: 8

Starting with version 1.25, the Go runtime respects the CPU quota:

docker run --cpus=4 golang:1.25-alpine go run /app/nproc.go

NumCPU: 8
GOMAXPROCS: 4

So, the default GOMAXPROCS value is set to either the number of logical CPUs or the CPU limit enforced by cgroup settings for the process, whichever is lower.

Fractional CPU limits are rounded up:

docker run --cpus=2.3 golang:1.25-alpine go run /app/nproc.go

NumCPU: 8
GOMAXPROCS: 3

On a machine with multiple CPUs, the minimum default value for GOMAXPROCS is 2, even if the CPU limit is set lower:

docker run --cpus=1 golang:1.25-alpine go run /app/nproc.go

NumCPU: 8
GOMAXPROCS: 2

The Go runtime automatically updates GOMAXPROCS if the CPU limit changes. It happens up to once per second (less frequently if the application is idle).

Concurrency primitives

Let's take a quick look at the three main concurrency tools for Go: goroutines, channels, and select.

Goroutine

A goroutine is implemented as a pointer to a runtime.g structure. Here's what it looks like:

// runtime/runtime2.go
type g struct {
    atomicstatus atomic.Uint32 // goroutine status
    stack        stack         // goroutine stack
    m            *m            // thread that runs the goroutine
    // ...
}

The g structure has many fields, but most of its memory is taken up by the stack, which holds the goroutine's local variables. By default, each stack gets 2 KB of memory, and it grows if needed.

Because goroutines use very little memory, they're much more efficient than operating system threads, which usually need about 1 MB each. Also, switching between goroutines is very fast because it's handled by Go's scheduler and doesn't involve the operating system's kernel (unlike switching between threads managed by the OS). This lets Go run hundreds of thousands, or even millions, of goroutines on a single machine.

Channel

A channel is implemented as a pointer to a runtime.hchan structure. Here's what it looks like:

// runtime/chan.go
type hchan struct {
    // channel buffer
    qcount   uint           // number of items in the buffer
    dataqsiz uint           // buffer array size
    buf      unsafe.Pointer // pointer to the buffer array

    // closed channel flag
    closed uint32

    // queues of goroutines waiting to receive and send
    recvq waitq // waiting to receive from the channel
    sendq waitq // waiting to send to the channel

    // protects the channel state
    lock mutex

    // ...
}

The buffer array (buf) has a fixed size (dataqsiz, which you can get with the cap() builtin). It's created when you make a buffered channel. The number of items in the channel (qcount, which you can get with the len() builtin) increases when you send to the channel and decreases when you receive from it.

The close() builtin sets the closed field to 1.

Sending an item to an unbuffered channel, or to a buffered channel that's already full, puts the goroutine into the sendq queue. Receiving from an empty channel puts the goroutine into the recvq queue.

Select

The select logic is implemented in the runtime.selectgo function. It's a huge function that takes a list of select cases and (very simply put) works as follows:

• Go through the cases and check if the matching channels are ready to send or receive.
• If several cases are ready, choose one at random (to prevent starvation, where some cases are always chosen and others are never chosen).
• Once a case is selected, perform the send or receive operation on the matching channel.
• If there is a default case and no other cases are ready, pick the default.
• If no cases are ready, block the goroutine and add it to the channel queue for each case.

Scheduler metrics

Metrics show how the Go runtime is performing, like how much heap memory it uses or how long garbage collection pauses take. Each metric has a unique name (for example, /sched/gomaxprocs:threads) and a value, which can be a number or a histogram.

We use the runtime/metrics package to work with metrics.

List all available metrics with descriptions:

func main() {
    descs := metrics.All()
    for _, d := range descs {
        fmt.Printf("Name: %s\n", d.Name)
        fmt.Printf("Description: %s\n", d.Description)
        fmt.Printf("Kind: %s\n", kindToString(d.Kind))
        fmt.Println()
    }
}

func kindToString(k metrics.ValueKind) string {
    switch k {
    case metrics.KindUint64:
        return "KindUint64"
    case metrics.KindFloat64:
        return "KindFloat64"
    case metrics.KindFloat64Histogram:
        return "KindFloat64Histogram"
    case metrics.KindBad:
        return "KindBad"
    default:
        return "Unknown"
    }
}

Name: /cgo/go-to-c-calls:calls
Description: Count of calls made from Go to C by the current process.
Kind: KindUint64

Name: /cpu/classes/gc/mark/assist:cpu-seconds
Description: Estimated total CPU time goroutines spent performing GC
tasks to assist the GC and prevent it from falling behind the application.
This metric is an overestimate, and not directly comparable to system
CPU time measurements. Compare only with other /cpu/classes metrics.
Kind: KindFloat64
...

Get the value of a specific metric:

samples := []metrics.Sample{
    {Name: "/sched/gomaxprocs:threads"},
    {Name: "/sched/goroutines:goroutines"},
}
metrics.Read(samples)

for _, s := range samples {
    // Assumes the value is a uint64. Check the metric description
    // or use s.Value.Kind() if you're not sure.
    fmt.Printf("%s: %v\n", s.Name, s.Value.Uint64())
}

/sched/gomaxprocs:threads: 8
/sched/goroutines:goroutines: 1

Here are some goroutine-related metrics:

/sched/goroutines-created:goroutines

• Count of goroutines created since program start (Go 1.26+).

/sched/goroutines:goroutines

• Count of live goroutines (created but not finished yet).
• An increase in this metric may indicate a goroutine leak.

/sched/goroutines/not-in-go:goroutines

• Approximate count of goroutines running or blocked in a system call or cgo call (Go 1.26+).
• An increase in this metric may indicate problems with such calls.

/sched/goroutines/runnable:goroutines

• Approximate count of goroutines ready to execute, but not executing (Go 1.26+).
• An increase in this metric may mean the system is overloaded and the CPU can't keep up with the growing number of goroutines.

/sched/goroutines/running:goroutines

• Approximate count of goroutines executing (Go 1.26+).
• Always less than or equal to /sched/gomaxprocs:threads.

/sched/goroutines/waiting:goroutines

• Approximate count of goroutines waiting on a resource — I/O or sync primitives (Go 1.26+).
• An increase in this metric may indicate issues with mutex locks, other synchronization blocks, or I/O issues.

/sched/threads/total:threads

• The current count of live threads that are owned by the runtime (Go 1.26+).

/sched/gomaxprocs:threads

• The current runtime.GOMAXPROCS setting — the maximum number of operating system threads the scheduler can use to execute goroutines concurrently.

In real projects, runtime metrics are usually exported automatically with client libraries for Prometheus, OpenTelemetry, or other observability tools. Here's an example for Prometheus:

package main

import (
    "net/http"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)

func main() {
    // Export runtime/metrics in Prometheus format at the /metrics endpoint.
    http.Handle("/metrics", promhttp.Handler())
    http.ListenAndServe("localhost:2112", nil)
}

The exported metrics are then collected by Prometheus, visualized, and used to set up alerts.

Profiling

Profiling helps you understand exactly what the program is doing, what resources it uses, and where in the code this happens. Profiling is often not recommended in production because it's a "heavy" process that can slow things down. But that's not the case with Go.

Go's profiler is designed for production use. It uses sampling, so it doesn't track every single operation. Instead, it takes quick snapshots of the runtime every 10 ms and puts them together to give you a full picture.

Go supports the following profiles:

• CPU. Shows how much CPU time each function uses. Use it to find performance bottlenecks if your program is running slowly because of CPU-heavy tasks.
• Heap. Shows the heap memory currently used by each function. Use it to detect memory leaks or excessive memory usage.
• Allocs. Shows which functions have used heap memory since the profiler started (not just currently). Use it to optimize garbage collection or reduce allocations that impact performance.
• Goroutine. Shows the stack traces of all current goroutines. Use it to get an overview of what the program is doing.
• Block. Shows where goroutines block waiting on synchronization primitives like channels, mutexes and wait groups. Use it to identify synchronization bottlenecks and issues in data exchange between goroutines. Disabled by default.
• Mutex. Shows lock contentions on mutexes and internal runtime locks. Use it to find "problematic" mutexes that goroutines are frequently waiting for. Disabled by default.

The easiest way to add a profiler to your app is by using the net/http/pprof package. When you import it, it automatically registers HTTP handlers for collecting profiles:

package main

import (
    "net/http"
    _ "net/http/pprof"
    "sync"
)

func main() {
    // Enable block and mutexe profiles.
    runtime.SetBlockProfileRate(1)
    runtime.SetMutexProfileFraction(1)
    // Start an HTTP server on localhost.
    // Profiler HTTP handlers are automatically
    // registered when you import "net/http/pprof".
    http.ListenAndServe("localhost:6060", nil)
}

Or you can register profiler handlers manually:

var wg sync.WaitGroup

wg.Go(func() {
    // Application server running on port 8080.
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello, World!"))
    })
    log.Println("Starting hello server on :8080")
    log.Fatal(http.ListenAndServe(":8080", mux))
})

wg.Go(func() {
    // Profiling server running on localhost on port 6060.
    runtime.SetBlockProfileRate(1)
    runtime.SetMutexProfileFraction(1)

    mux := http.NewServeMux()
    mux.HandleFunc("/debug/pprof/", pprof.Index)
    mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
    mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
    log.Println("Starting pprof server on :6060")
    log.Fatal(http.ListenAndServe("localhost:6060", mux))
})

wg.Wait()

After that, you can start profiling with a specific profile by running the go tool pprof command with the matching URL, or just open that URL in your browser:

go tool pprof -proto \
  "http://localhost:6060/debug/pprof/profile?seconds=N" > cpu.pprof

go tool pprof -proto \
  http://localhost:6060/debug/pprof/heap > heap.pprof

go tool pprof -proto \
  http://localhost:6060/debug/pprof/allocs > allocs.pprof

go tool pprof -proto \
  http://localhost:6060/debug/pprof/goroutine > goroutine.pprof

go tool pprof -proto \
  http://localhost:6060/debug/pprof/block > block.pprof

go tool pprof -proto \
  http://localhost:6060/debug/pprof/mutex > mutex.pprof

For the CPU profile, you can choose how long the profiler runs (the default is 30 seconds). Other profiles are taken instantly.

After running the profiler, you'll get a binary file that you can open in the browser using the same go tool pprof utility. For example:

go tool pprof -http=localhost:8080 cpu.pprof

The pprof web interface lets you view the same profile in different ways. My personal favorites are the flame graph, which clearly shows the call hierarchy and resource usage, and the source view, which shows the exact lines of code.

You can also profile manually. To collect a CPU profile, use StartCPUProfile and StopCPUProfile:

func main() {
    // Start profiling and stop it when main exits.
    // Ignore errors for simplicity.
    file, _ := os.Create("cpu.prof")
    defer file.Close()
    pprof.StartCPUProfile(file)
    defer pprof.StopCPUProfile()

    // The rest of the program code.
    // ...
}

To collect other profiles, use Lookup:

// profile collects a profile with the given name.
func profile(name string) {
    // Ignore errors for simplicity.
    file, _ := os.Create(name + ".prof")
    defer file.Close()
    p := pprof.Lookup(name)
    if p != nil {
        p.WriteTo(file, 0)
    }
}

func main() {
    runtime.SetBlockProfileRate(1)
    runtime.SetMutexProfileFraction(1)

    // ...
    profile("heap")
    profile("allocs")
    // ...
}

Profiling is a broad topic, and we've only touched the surface. To learn more, start with these articles:

• Profiling Go Programs
• Diagnostics

Tracing

Tracing records certain types of events while the program is running, mainly those related to concurrency and memory:

• goroutine creation and state changes;
• system calls;
• garbage collection;
• heap size changes;
• and more.

If you enabled the profiling server as described earlier, you can collect a trace using this URL:

http://localhost:6060/debug/pprof/trace?seconds=N

Trace files can be quite large, so it's better to use a small N value.

After tracing is complete, you'll get a binary file that you can open in the browser using the go tool trace utility:

go tool trace -http=localhost:6060 trace.out

In the trace web interface, you'll see each goroutine's "lifecycle" on its own line. You can zoom in and out of the trace with the W and S keys, and you can click on any event to see more details:

You can also collect a trace manually:

func main() {
    // Start tracing and stop it when main exits.
    // Ignore errors for simplicity.
    file, _ := os.Create("trace.out")
    defer file.Close()
    trace.Start(file)
    defer trace.Stop()

    // The rest of the program code.
    // ...
}

Flight recorder

Flight recording is a tracing technique that collects execution data, such as function calls and memory allocations, within a sliding window that's limited by size or duration. It helps to record traces of interesting program behavior, even if you don't know in advance when it will happen.

The trace.FlightRecorder type (Go 1.25+) implements a flight recorder in Go. It tracks a moving window over the execution trace produced by the runtime, always containing the most recent trace data.

Here's an example of how you might use it.

First, configure the sliding window:

// Configure the flight recorder to keep
// at least 5 seconds of trace data,
// with a maximum buffer size of 3MB.
// Both of these are hints, not strict limits.
cfg := trace.FlightRecorderConfig{
    MinAge:   5 * time.Second,
    MaxBytes: 3 << 20, // 3MB
}

Then create the recorder and start it:

// Create and start the flight recorder.
rec := trace.NewFlightRecorder(cfg)
rec.Start()
defer rec.Stop()

Continue with the application code as usual:

// Simulate some workload.
done := make(chan struct{})
go func() {
    defer close(done)
    const n = 1 << 20
    var s []int
    for range n {
        s = append(s, rand.IntN(n))
    }
    fmt.Printf("done filling slice of %d elements\n", len(s))
}()
<-done

Finally, save the trace snapshot to a file when an important event occurs:

// Save the trace snapshot to a file.
file, _ := os.Create("/tmp/trace.out")
defer file.Close()
n, _ := rec.WriteTo(file)
fmt.Printf("wrote %dB to trace file\n", n)

done filling slice of 1048576 elements
wrote 8441B to trace file

Use go tool trace to view the trace in the browser:

go tool trace -http=localhost:6060 /tmp/trace.out

Keep it up

Now you can see how challenging the Go scheduler's job is. Fortunately, most of the time you don't need to worry about how it works behind the scenes — sticking to goroutines, channels, select, and other synchronization primitives is usually enough.

This is the final chapter of my "Gist of Go: Concurrency" book. I invite you to read it — the book is an easy-to-understand, interactive guide to concurrency programming in Go.

Pre-order for $10   or read online

Introducing errors.AsType — a modern, type-safe alternative to errors.As.

Ver. 1.26 • Stdlib • High impact

Summary

The new errors.AsType function is a generic version of errors.As:

// go 1.13+
func As(err error, target any) bool
// go 1.26+
func AsType[E error](err error) (E, bool)

It's type-safe, faster, and easier to use:

// using errors.As
var appErr AppError
if errors.As(err, &appErr) {
    fmt.Println("Got an AppError:", appErr)
}

// using errors.AsType
if appErr, ok := errors.AsType[AppError](err); ok {
    fmt.Println("Got an AppError:", appErr)
}

errors.As is not deprecated (yet), but errors.AsType is recommended for new code.

Motivation

The errors.As function requires you to declare a variable of the target error type and pass a pointer to it:

var appErr AppError
if errors.As(err, &appErr) {
    fmt.Println("Got an AppError:", appErr)
}

It makes the code quite verbose, especially when checking for multiple types of errors:

var connErr *net.OpError
var dnsErr *net.DNSError

if errors.As(err, &connErr) {
    fmt.Println("Network operation failed:", connErr.Op)
} else if errors.As(err, &dnsErr) {
    fmt.Println("DNS resolution failed:", dnsErr.Name)
} else {
    fmt.Println("Unknown error")
}

With a generic errors.AsType, you can specify the error type right in the function call. This makes the code shorter and keeps error variables scoped to their if blocks:

if connErr, ok := errors.AsType[*net.OpError](err); ok {
    fmt.Println("Network operation failed:", connErr.Op)
} else if dnsErr, ok := errors.AsType[*net.DNSError](err); ok {
    fmt.Println("DNS resolution failed:", dnsErr.Name)
} else {
    fmt.Println("Unknown error")
}

Another issue with As is that it uses reflection and can cause runtime panics if used incorrectly (like if you pass a non-pointer or a type that doesn't implement error). While static analysis tools usually catch these issues, using the generic AsType has several benefits:

• No reflection¹.
• No runtime panics.
• Less allocations.
• Compile-time type safety.
• Faster.

Finally, AsType can handle everything that As does, so it's a drop-in improvement for new code.

Description

Add the AsType function to the errors package:

// AsType finds the first error in err's tree that matches the type E,
// and if one is found, returns that error value and true. Otherwise, it
// returns the zero value of E and false.
//
// The tree consists of err itself, followed by the errors obtained by
// repeatedly calling its Unwrap() error or Unwrap() []error method.
// When err wraps multiple errors, AsType examines err followed by a
// depth-first traversal of its children.
//
// An error err matches the type E if the type assertion err.(E) holds,
// or if the error has a method As(any) bool such that err.As(target)
// returns true when target is a non-nil *E. In the latter case, the As
// method is responsible for setting target.
func AsType[E error](err error) (E, bool)

Recommend using AsType instead of As:

// As finds the first error in err's tree that matches target, and if one
// is found, sets target to that error value and returns true. Otherwise,
// it returns false.
// ...
// For most uses, prefer [AsType]. As is equivalent to [AsType] but sets its
// target argument rather than returning the matching error and doesn't require
// its target argument to implement error.
// ...
func As(err error, target any) bool

Example

Open a file and check if the error is related to the file path:

// go 1.25
var pathError *fs.PathError
if _, err := os.Open("non-existing"); err != nil {
    if errors.As(err, &pathError) {
        fmt.Println("Failed at path:", pathError.Path)
    } else {
        fmt.Println(err)
    }
}

Failed at path: non-existing

// go 1.26
if _, err := os.Open("non-existing"); err != nil {
    if pathError, ok := errors.AsType[*fs.PathError](err); ok {
        fmt.Println("Failed at path:", pathError.Path)
    } else {
        fmt.Println(err)
    }
}

Failed at path: non-existing

Further reading

𝗣 51945 • 𝗖𝗟 707235

Export goroutine-related metrics from the Go runtime.

Ver. 1.26 • Stdlib • Medium impact

Summary

New metrics in the runtime/metrics package give better insight into goroutine scheduling:

• Total number of goroutines since the program started.
• Number of goroutines in each state.
• Number of active threads.

Motivation

Go's runtime/metrics package already provides a lot of runtime stats, but it doesn't include metrics for goroutine states or thread counts.

Per-state goroutine metrics can be linked to common production issues. An increasing waiting count can show a lock contention problem. A high not-in-go count means goroutines are stuck in syscalls or cgo. A growing runnable backlog suggests the CPUs can't keep up with demand.

Observability systems can track these counters to spot regressions, find scheduler bottlenecks, and send alerts when goroutine behavior changes from the usual patterns. Developers can use them to catch problems early without needing full traces.

Description

Add the following metrics to the runtime/metrics package:

/sched/goroutines-created:goroutines
	Count of goroutines created since program start.

/sched/goroutines/not-in-go:goroutines
	Approximate count of goroutines running
    or blocked in a system call or cgo call.

/sched/goroutines/runnable:goroutines
	Approximate count of goroutines ready to execute,
	but not executing.

/sched/goroutines/running:goroutines
	Approximate count of goroutines executing.
    Always less than or equal to /sched/gomaxprocs:threads.

/sched/goroutines/waiting:goroutines
	Approximate count of goroutines waiting
    on a resource (I/O or sync primitives).

/sched/threads/total:threads
	The current count of live threads
    that are owned by the Go runtime.

The per-state numbers are not guaranteed to add up to the live goroutine count (/sched/goroutines:goroutines, available since Go 1.16).

All metrics use uint64 counters.

Example

Start some goroutines and print the metrics after 100 ms of activity:

func main() {
	go work() // omitted for brevity
	time.Sleep(100 * time.Millisecond)

	fmt.Println("Goroutine metrics:")
	printMetric("/sched/goroutines-created:goroutines", "Created")
	printMetric("/sched/goroutines:goroutines", "Live")
	printMetric("/sched/goroutines/not-in-go:goroutines", "Syscall/CGO")
	printMetric("/sched/goroutines/runnable:goroutines", "Runnable")
	printMetric("/sched/goroutines/running:goroutines", "Running")
	printMetric("/sched/goroutines/waiting:goroutines", "Waiting")

	fmt.Println("Thread metrics:")
	printMetric("/sched/gomaxprocs:threads", "Max")
	printMetric("/sched/threads/total:threads", "Live")
}

func printMetric(name string, descr string) {
	sample := []metrics.Sample{{Name: name}}
	metrics.Read(sample)
	// Assuming a uint64 value; don't do this in production.
	// Instead, check sample[0].Value.Kind and handle accordingly.
	fmt.Printf("  %s: %v\n", descr, sample[0].Value.Uint64())
}

Goroutine metrics:
  Created: 52
  Live: 12
  Syscall/CGO: 0
  Runnable: 0
  Running: 4
  Waiting: 8
Thread metrics:
  Max: 8
  Live: 4

No surprises here: we read the new metric values the same way as before — using metrics.Read.

Further reading

𝗣 15490 • 𝗖𝗟 690397, 690398, 690399

P.S. If you are into goroutines, check out my interactive book on concurrency

Testing concurrent programs is a lot like testing single-task programs. If the code is well-designed, you can test the state of a concurrent program with standard tools like channels, wait groups, and other abstractions built on top of them.

But if you've made it so far, you know that concurrency is never that easy. In this chapter, we'll go over common testing problems and the solutions that Go offers.

Waiting for goroutines • Checking channels • Checking for leaks • Durable blocking • Instant waiting • Time inside the bubble • Thoughts on time 1 ✎ • Thoughts on time 2 ✎ • Checking for cleanup • Bubble rules • Keep it up

Waiting for goroutines to finish

Let's say we want to test this function:

// Calc calculates something asynchronously.
func Calc() <-chan int {
    out := make(chan int, 1)
    go func() {
        out <- 42
    }()
    return out
}

Calculations run asynchronously in a separate goroutine. However, the function returns a result channel, so this isn't a problem:

func Test(t *testing.T) {
    got := <-Calc() // (X)
    if got != 42 {
        t.Errorf("got: %v; want: 42", got)
    }
}

PASS

At point ⓧ, the test is guaranteed to wait for the inner goroutine to finish. The rest of the test code doesn't need to know anything about how concurrency works inside the Calc function. Overall, the test isn't any more complicated than if Calc were synchronous.

But we're lucky that Calc returns a channel. What if it doesn't?

Naive approach

Let's say the Calc function looks like this:

var state atomic.Int32

// Calc calculates something asynchronously.
func Calc() {
    go func() {
        state.Store(42)
    }()
}

We write a simple test and run it:

func TestNaive(t *testing.T) {
    Calc()
    got := state.Load() // (X)
    if got != 42 {
        t.Errorf("got: %v; want: 42", got)
    }
}

=== RUN   TestNaive
    main_test.go:27: got: 0; want: 42
--- FAIL: TestNaive (0.00s)

The assertion fails because at point ⓧ, we didn't wait for the inner Calc goroutine to finish. In other words, we didn't synchronize the TestNaive and Calc goroutines. That's why state still has its initial value (0) when we do the check.

Waiting with time.Sleep

We can add a short delay with time.Sleep:

func TestSleep(t *testing.T) {
    Calc()

    // Wait for the goroutine to finish (if we're lucky).
    time.Sleep(50 * time.Millisecond)

    got := state.Load()
    if got != 42 {
        t.Errorf("got: %v; want: 42", got)
    }
}

=== RUN   TestSleep
--- PASS: TestSleep (0.05s)

The test is now passing. But using time.Sleep to sync goroutines isn't a great idea, even in tests. We don't want to set a custom delay for every function we're testing. Also, the function's execution time may be different on the local machine compared to a CI server. If we use a longer delay just to be safe, the tests will end up taking too long to run.

Sometimes you can't avoid using time.Sleep in tests, but since Go 1.25, the synctest package has made these cases much less common. Let's see how it works.

Waiting with synctest

The synctest package has a lot going on under the hood, but its public API is very simple:

func Test(t *testing.T, f func(*testing.T))
func Wait()

The synctest.Test function creates an isolated bubble where you can control time to some extent. Any new goroutines started inside this bubble become part of the bubble. So, if we wrap the test code with synctest.Test, everything will run inside the bubble — the test code, the Calc function we're testing, and its goroutine.

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        Calc()

        // (X)

        got := state.Load()
        if got != 42 {
            t.Errorf("got: %v; want: 42", got)
        }
    })
}

At point ⓧ, we want to wait for the Calc goroutine to finish. The synctest.Wait function comes to the rescue! It blocks the calling goroutine until all other goroutines in the bubble are finished. (It's actually a bit more complicated than that, but we'll talk about it later.)

In our case, there's only one other goroutine (the inner Calc goroutine), so Wait will pause until it finishes, and then the test will move on.

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        Calc()

        // Wait for the goroutine to finish.
        synctest.Wait()

        got := state.Load()
        if got != 42 {
            t.Errorf("got: %v; want: 42", got)
        }
    })
}

=== RUN   TestSync
--- PASS: TestSync (0.00s)

Now the test passes instantly. That's better!

Checking the channel state

As we've seen, you can use synctest.Wait to wait for the tested goroutine to finish, and then check the state of the data you are interested in. You can also use it to check the state of channels.

Let's say there's a function that generates N numbers like 11, 22, 33, and so on:

// Generate produces n numbers like 11, 22, 33, ...
func Generate(n int) <-chan int {
    out := make(chan int)
    go func() {
        for i := range n {
            out <- (i+1)*10 + (i + 1)
        }
    }()
    return out
}

And a simple test:

func Test(t *testing.T) {
    out := Generate(2)
    var got int

    got = <-out
    if got != 11 {
        t.Errorf("#1: got %v, want 11", got)
    }
    got = <-out
    if got != 22 {
        t.Errorf("#1: got %v, want 22", got)
    }
}

PASS

Set N=2, get the first number from the generator's output channel, then get the second number. The test passed, so the function works correctly. But does it really?

Let's use Generate in "production":

func main() {
    for v := range Generate(3) {
        fmt.Print(v, " ")
    }
}

11 22 33 fatal error: all goroutines are asleep - deadlock!

Panic! We forgot to close the out channel when exiting the inner Generate goroutine, so the for-range loop waiting on that channel got stuck.

Let's fix the code:

// Generate produces n numbers like 11, 22, 33, ...
func Generate(n int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for i := range n {
            out <- (i+1)*10 + (i + 1)
        }
    }()
    return out
}

And add a test for the out channel state:

func Test(t *testing.T) {
    out := Generate(2)
    <-out // 11
    <-out // 22

    // (X)

    // Check that the channel is closed.
    select {
    case _, ok := <-out:
        if ok {
            t.Errorf("expected channel to be closed")
        }
    default:
        t.Errorf("expected channel to be closed")
    }
}

--- FAIL: Test (0.00s)
    main_test.go:41: expected channel to be closed

The test is still failing, even though we're now closing the channel when the Generate goroutine exits.

This is a familiar problem: at point ⓧ, we didn't wait for the inner Generate goroutine to finish. So when we check the out channel, it hasn't closed yet. That's why the test fails.

We can delay the check using time.After:

func Test(t *testing.T) {
    out := Generate(2)
    <-out
    <-out

    // Check that the channel is closed.
    select {
    case _, ok := <-out:
        if ok {
            t.Errorf("expected channel to be closed")
        }
    case <-time.After(50 * time.Millisecond):
        t.Fatalf("timeout waiting for channel to close")
    }
}

PASS

But it's better to use synctest:

func TestClose(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        out := Generate(2)
        <-out
        <-out

        // Wait for the goroutine to finish.
        synctest.Wait()

        // Check that the channel is closed.
        select {
        case _, ok := <-out:
            if ok {
                t.Errorf("expected channel to be closed")
            }
        default:
            t.Errorf("expected channel to be closed")
        }
    })
}

PASS

At point ⓧ, synctest.Wait blocks the test until the only other goroutine (the inner Generate goroutine) finishes. Once the goroutine has exited, the channel is already closed. So, in the select statement, the <-out case triggers with ok set to false, allowing the test to pass.

As you can see, the synctest package helped us avoid delays in the test, and the test itself didn't get much more complicated.

Checking for goroutine leaks

As we've seen, you can use synctest.Wait to wait for the tested goroutine to finish, and then check the state of the data or channels. You can also use it to detect goroutine leaks.

Let's say there's a function that runs the given functions concurrently and sends their results to an output channel:

// Map runs the given functions concurently.
func Map(funcs ...func() int) <-chan int {
    out := make(chan int)
    for _, f := range funcs {
        go func() {
            out <- f()
        }()
    }
    return out
}

And a simple test:

func Test(t *testing.T) {
    out := Map(
        func() int { return 11 },
        func() int { return 22 },
        func() int { return 33 },
    )

    got := <-out
    if got != 11 && got != 22 && got != 33 {
        t.Errorf("got %v, want 11, 22 or 33", got)
    }
}

PASS

Send three functions to be executed, get the first result from the output channel, and check it. The test passed, so the function works correctly. But does it really?

Let's run Map three times, passing three functions each time:

func main() {
    for range 3 {
        Map(
            func() int { return 11 },
            func() int { return 22 },
            func() int { return 33 },
        )
    }

    time.Sleep(50 * time.Millisecond)
    nGoro := runtime.NumGoroutine() - 1 // minus the main goroutine
    fmt.Println("nGoro =", nGoro)
}

nGoro = 9

After 50 ms — when all the functions should definitely have finished — there are still 9 running goroutines (runtime.NumGoroutine). In other words, all the goroutines are stuck.

The reason is that the out channel is unbuffered. If the client doesn't read from it, or doesn't read all the results, the goroutines inside Map get blocked when they try to send the result of f() to out.

Let's fix this by adding a buffer of the right size to the channel:

// Map runs the given functions concurently.
func Map(funcs ...func() int) <-chan int {
    out := make(chan int, len(funcs))
    for _, f := range funcs {
        go func() {
            out <- f()
        }()
    }
    return out
}

Then add a test to check the number of goroutines:

func Test(t *testing.T) {
    for range 3 {
        Map(
            func() int { return 11 },
            func() int { return 22 },
            func() int { return 33 },
        )
    }

    // (X)

    nGoro := runtime.NumGoroutine() - 2 // minus the main and Test goroutines

    if nGoro != 0 {
        t.Fatalf("expected 0 goroutines, got %d", nGoro)
    }
}

--- FAIL: Test (0.00s)
    main_test.go:44: expected 0 goroutines, got 9

The test is still failing, even though the channel is now buffered, and the goroutines shouldn't block on sending to it.

This is a familiar problem: at point ⓧ, we didn't wait for the running Map goroutines to finish. So nGoro is greater than zero, which makes the test fail.

We can delay the check using time.Sleep (not recommended), or use a third-party package like goleak (a better option):

func Test(t *testing.T) {
    defer goleak.VerifyNone(t)

    for range 3 {
        Map(
            func() int { return 11 },
            func() int { return 22 },
            func() int { return 33 },
        )
    }
}

PASS

The test passes now.

By the way, goleak also uses time.Sleep internally, but it does so much more efficiently. It tries up to 20 times, with the wait time between checks increasing exponentially, starting at 1 microsecond and going up to 100 milliseconds. This way, the test runs almost instantly.

Even better, we can check for leaks without any third-party packages by using synctest:

func Test(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        for range 3 {
            Map(
                func() int { return 11 },
                func() int { return 22 },
                func() int { return 33 },
            )
        }
        synctest.Wait()
    })
}

PASS

Earlier, I said that synctest.Wait blocks the calling goroutine until all other goroutines finish. Actually, it's a bit more complicated. synctest.Wait blocks until all other goroutines either finish or become durably blocked.

We'll talk about "durably" later. For now, let's focus on "become blocked." Let's temporarily remove the buffer from the channel and check the test results:

// Map runs the given functions concurently.
func Map(funcs ...func() int) <-chan int {
    out := make(chan int)
    for _, f := range funcs {
        go func() {
            out <- f()
        }()
    }
    return out
}

func Test(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        for range 3 {
            Map(
                func() int { return 11 },
                func() int { return 22 },
                func() int { return 33 },
            )
        }
        synctest.Wait()
    })
}

--- FAIL: Test (0.00s)
panic: deadlock: main bubble goroutine has exited but blocked goroutines remain [recovered, repanicked]

Here's what happens:

1. Three calls to Map start 9 goroutines.
2. The call to synctest.Wait blocks the root bubble goroutine (synctest.Test).
3. One of the goroutines finishes its work, tries to write to out, and gets blocked (because no one is reading from out).
4. The same thing happens to the other 8 goroutines.
5. synctest.Wait sees that all the child goroutines in the bubble are blocked, so it unblocks the root goroutine.
6. The root goroutine finishes.

Next, synctest.Test comes into play. It not only starts the bubble goroutine, but also tries to wait for all child goroutines to finish before it returns. If Test sees that some goroutines are stuck (in our case, all 9 are blocked trying to send to the channel), it panics:

main bubble goroutine has exited but blocked goroutines remain

So, we found the leak without using time.Sleep or goleak, thanks to the useful features of synctest.Wait and synctest.Test:

• synctest.Wait unblocks as soon as all other goroutines are durably blocked.
• synctest.Test panics when finished if there are still blocked goroutines left in the bubble.

Now let's make the channel buffered and run the test again:

=== RUN   Test
--- PASS: Test (0.00s)

Perfect!

Durable blocking

As we've found, synctest.Wait blocks until all goroutines in the bubble — except the one that called Wait — have either finished or are durably blocked. Let's figure out what "durably blocked" means.

For synctest, a goroutine inside a bubble is considered durably blocked if it is blocked by any of the following operations:

• Sending to or receiving from a channel created within the bubble.
• A select statement where every case is a channel created within the bubble.
• Calling WaitGroup.Wait if all WaitGroup.Add calls were made inside the bubble.
• Calling Cond.Wait.
• Calling time.Sleep.

Other blocking operations are not considered durable, and synctest.Wait ignores them. For example:

• Sending to or receiving from a channel created outside the bubble.
• Calling Mutex.Lock or RWMutex.Lock.
• I/O operations (like reading a file from disk or waiting for a network response).
• System calls and cgo calls.

The distinction between "durable" and other types of blocks is just a implementation detail of the synctest package. It's not a fundamental property of the blocking operations themselves. In real-world applications, this distinction doesn't exist, and "durable" blocks are neither better nor worse than any others.

Let's look at an example.

Asynchronous processor

Let's say there's a Proc type that performs some asynchronous computation:

// Proc calculates something asynchronously.
type Proc struct {
    // ...
}

// NewProc starts the calculation in a separate goroutine.
// The calculation keep running until Stop is called.
func NewProc() *Proc

// Res returns the current calculation result.
// It's only available until Stop is called; after that, it resets to zero.
func (p *Proc) Res() int

// Stop terminates the calculation.
func (p *Proc) Stop()

Our goal is to write a test that checks the result while the calculation is still running. Let's see how the test changes depending on how Proc is implemented (except for the time.Sleep version — we'll cover that one a bit later).

Blocking on a channel

Let's say Proc is implemented using a done channel:

// Proc calculates something asynchronously.
type Proc struct {
    res  int
    done chan struct{}
}

// NewProc starts the calculation.
func NewProc() *Proc {
    p := &Proc{done: make(chan struct{})}
    go func() {
        p.res = 42
        <-p.done // (X)
        p.res = 0
    }()
    return p
}

// Stop terminates the calculation.
func (p *Proc) Stop() {
    close(p.done)
}

Naive test:

func TestNaive(t *testing.T) {
    p := NewProc()
    defer p.Stop()

    if got := p.Res(); got != 42 {
        t.Fatalf("got %v, want 42", got)
    }
}

--- FAIL: TestNaive (0.00s)
    main_test.go:52: got 0, want 42

The check fails because when p.Res() is called, the goroutine in NewProc hasn't set p.res = 42 yet.

Let's use synctest.Wait to wait until the goroutine is blocked at point ⓧ:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        // Wait for the goroutine to block at point X.
        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

PASS

In ⓧ, the goroutine is blocked on reading from the p.done channel. This channel is created inside the bubble, so the block is durable. The synctest.Wait call in the test returns as soon as <-p.done happens, and we get the current value of p.res.

Blocking on a select

Let's say Proc is implemented using select:

// Proc calculates something asynchronously.
type Proc struct {
    res  int
    in   chan int
    done chan struct{}
}

// NewProc starts the calculation.
func NewProc() *Proc {
    p := &Proc{
        res:  0,
        in:   make(chan int),
        done: make(chan struct{}),
    }
    go func() {
        p.res = 42
        select { // (X)
        case n := <-p.in:
            p.res = n
        case <-p.done:
        }
    }()
    return p
}

// Stop terminates the calculation.
func (p *Proc) Stop() {
    close(p.done)
}

Let's use synctest.Wait to wait until the goroutine is blocked at point ⓧ:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        // Wait for the goroutine to block at point X.
        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

PASS

In ⓧ, the goroutine is blocked on a select statement. Both channels used in the select (p.in and p.done) are created inside the bubble, so the block is durable. The synctest.Wait call in the test returns as soon as select happens, and we get the current value of p.res.

Blocking on a wait group

Let's say Proc is implemented using a wait group:

// Proc calculates something asynchronously.
type Proc struct {
    res int
    wg  sync.WaitGroup
}

// NewProc starts the calculation.
func NewProc() *Proc {
    p := &Proc{}
    p.wg.Add(1)
    go func() {
        p.res = 42
        p.wg.Wait() // (X)
        p.res = 0
    }()
    return p
}

// Stop terminates the calculation.
func (p *Proc) Stop() {
    p.wg.Done()
}

Let's use synctest.Wait to wait until the goroutine is blocked at point ⓧ:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        // Wait for the goroutine to block at point X.
        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

PASS

In ⓧ, the goroutine is blocked on the wait group's p.wg.Wait() call. The group's Add method was called inside the bubble, so this is a durable block. The synctest.Wait call in the test returns as soon as p.wg.Wait() happens, and we get the current value of p.res.

Blocking on a condition variable

Let's say Proc is implemented using a condition variable:

// Proc calculates something asynchronously.
type Proc struct {
    res  int
    cond *sync.Cond
}

// NewProc starts the calculation.
func NewProc() *Proc {
    p := &Proc{
        cond: sync.NewCond(&sync.Mutex{}),
    }
    go func() {
        p.cond.L.Lock()
        p.res = 42
        p.cond.Wait() // (X)
        p.res = 0
        p.cond.L.Unlock()
    }()
    return p
}

// Stop terminates the calculation.
func (p *Proc) Stop() {
    p.cond.Signal()
}

Let's use synctest.Wait to wait until the goroutine is blocked at point ⓧ:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        // Wait for the goroutine to block at point X.
        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

PASS

In ⓧ, the goroutine is blocked on the condition variable's p.cond.Wait() call. This is a durable block. The synctest.Wait call returns as soon as p.cond.Wait() happens, and we get the current value of p.res.

Blocking on a mutex

Let's say Proc is implemented using a mutex:

// Proc calculates something asynchronously.
type Proc struct {
    res int
    mu  sync.Mutex
}

// NewProc starts the calculation.
func NewProc() *Proc {
    p := &Proc{}
    p.mu.Lock()
    go func() {
        p.res = 42
        p.mu.Lock() // (X)
        p.res = 0
        p.mu.Unlock()
    }()
    return p
}

// Stop terminates the calculation.
func (p *Proc) Stop() {
    p.mu.Unlock()
}

Let's try using synctest.Wait to wait until the goroutine is blocked at point ⓧ:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        // Hangs because synctest ignores blocking on a mutex.
        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

code execution timeout

In ⓧ, the goroutine is blocked on the mutex's p.mu.Lock() call. synctest doesn't consider blocking on a mutex to be durable. The synctest.Wait call ignores the block and never returns. The test hangs and only fails when the overall go test timeout is reached.

You might be wondering why the synctest authors didn't consider blocking on mutexes to be durable. There are a couple of reasons:

1. Mutexes are usually used to protect shared state, not to coordinate goroutines (the example above is completely unrealistic). In tests, you usually don't need to pause before locking a mutex to check something.
2. Mutex locks are usually held for a very short time, and mutexes themselves need to be as fast as possible. Adding extra logic to support synctest could slow them down in normal (non-test) situations.

⌘ ⌘ ⌘

Let's go back to the original question: how does the test change depending on how Proc is implemented? It doesn't change at all. We used the exact same test code every time:

func TestSync(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        p := NewProc()
        defer p.Stop()

        synctest.Wait()
        if got := p.Res(); got != 42 {
            t.Fatalf("got %v, want 42", got)
        }
    })
}

If your program uses durably blocking operations, synctest.Wait always works the same way:

1. It waits until all other goroutines in the bubble are blocked.
2. Then, it unblocks the goroutine that called it.

Very convenient!

Instant waiting

Inside the synctest.Test bubble, time works differently. Instead of using a regular wall clock, the bubble uses a fake clock that can jump forward to any point in the future. This can be quite handy when testing time-sensitive code.

Let's say we want to test this function:

// Calc processes a value from the input channel.
// Times out if no input is received after 3 seconds.
func Calc(in chan int) (int, error) {
    select {
    case v := <-in:
        return v * 2, nil
    case <-time.After(3 * time.Second):
        return 0, ErrTimeout
    }
}

The positive scenario is straightforward: send a value to the channel, call the function, and check the result:

func TestCalc_result(t *testing.T) {
    ch := make(chan int)
    go func() { ch <- 11 }()
    got, err := Calc(ch)

    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    if got != 22 {
        t.Errorf("got: %v; want: 22", got)
    }
}

PASS

The negative scenario, where the function times out, is also pretty straightforward. But the test takes the full three seconds to complete:

func TestCalc_timeout_naive(t *testing.T) {
    ch := make(chan int)
    got, err := Calc(ch) // runs for 3 seconds

    if err != ErrTimeout {
        t.Errorf("got: %v; want: %v", err, ErrTimeout)
    }
    if got != 0 {
        t.Errorf("got: %v; want: 0", got)
    }
}

=== RUN   TestCalc_timeout_naive
--- PASS: TestCalc_timeout_naive (3.00s)

We're actually lucky the timeout is only three seconds. It could have been as long as sixty!

To make the test run instantly, let's wrap it in synctest.Test:

func TestCalc_timeout_synctest(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        ch := make(chan int)
        got, err := Calc(ch) // runs instantly

        if err != ErrTimeout {
            t.Errorf("got: %v; want: %v", err, ErrTimeout)
        }
        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

=== RUN   TestCalc_timeout_synctest
--- PASS: TestCalc_timeout_synctest (0.00s)

Note that there is no synctest.Wait call here, and the only goroutine in the bubble (the root one) gets durably blocked on a select statement in Calc. Here's what happens next:

1. The bubble checks if the goroutine can be unblocked by waiting. In our case, it can — we just need to wait 3 seconds.
2. The bubble's clock instantly jumps forward 3 seconds.
3. The select in Calc chooses the timeout case, and the function returns ErrTimeout.
4. The test assertions for err and got both pass successfully.

Thanks to the fake clock, the test runs instantly instead of taking three seconds like it would with the "naive" approach.

You might have noticed that quite a few circumstances coincided here:

• There's no synctest.Wait call.
• There's only one goroutine.
• The goroutine is durably blocked.
• It will be unblocked at certain point in the future.

We'll look at the alternatives soon, but first, here's a quick exercise.

Time inside the bubble

The fake clock in synctest.Test can be tricky. It move forward only if: ➊ all goroutines in the bubble are durably blocked; ➋ there's a future moment when at least one goroutine will unblock; and ➌ synctest.Wait isn't running.

Let's look at the alternatives. I'll say right away, this isn't an easy topic. But when has time travel ever been easy? :)

Not all goroutines are blocked

Here's the Calc function we're testing:

// Calc processes a value from the input channel.
// Times out if no input is received after 3 seconds.
func Calc(in chan int) (int, error) {
    select {
    case v := <-in:
        return v * 2, nil
    case <-time.After(3 * time.Second):
        return 0, ErrTimeout
    }
}

Let's run Calc in a separate goroutine, so there will be two goroutines in the bubble:

func Test(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        var got int
        var err error

        go func() {
            ch := make(chan int)
            got, err = Calc(ch)
        }()

        if err != ErrTimeout {
            t.Errorf("got: %v; want: %v", err, ErrTimeout)
        }
        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

--- FAIL: Test (0.00s)
    main_test.go:45: got: <nil>; want: timeout
panic: deadlock: main bubble goroutine has exited but blocked goroutines remain [recovered, repanicked]

synctest.Test panicked because the root bubble goroutine finished while the Calc goroutine was still blocked on a select.

Reason: synctest.Test only advances the clock if all goroutines are blocked — including the root bubble goroutine.

How to fix: Use time.Sleep to make sure the root goroutine is also durably blocked.

func Test_fixed(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        ch := make(chan int)
        var got int

        go func() {
            got, _ = Calc(ch)
        }()

        // Wait for the Calc goroutine to finish.
        time.Sleep(5 * time.Second)

        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

PASS

Now all three conditions are met again (all goroutines are durably blocked; the moment of future unblocking is known; there is no call to synctest.Wait). The fake clock moves forward 3 seconds, which unblocks the Calc goroutine. The goroutine finishes, leaving only the root one, which is still blocked on time.Sleep. The clock moves forward another 2 seconds, unblocking the root goroutine. The assertion passes, and the test completes successfully.

But if we run the test with the race detector enabled (using the -race flag), it reports a data race on the got variable:

race detected during execution of test

Logically, using time.Sleep in the root goroutine doesn't guarantee that the Calc goroutine (which writes to the got variable) will finish before the root goroutine reads from got. That's why the race detector reports a problem. Technically, the test passes because of how synctest is implemented, but the race still exists in the code. The right way to handle this is to call synctest.Wait after time.Sleep:

func Test_fixed(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        ch := make(chan int)
        var got int

        go func() {
            got, _ = Calc(ch)
        }()

        // Wait for the Calc goroutine to finish.
        time.Sleep(3 * time.Second)
        synctest.Wait()

        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

PASS

Calling synctest.Wait ensures that the Calc goroutine finishes before the root goroutine reads got, so there's no data race anymore.

synctest.Wait is running

Here's the Calc function we're testing:

// Calc processes a value from the input channel.
// Times out if no input is received after 3 seconds.
func Calc(in chan int) (int, error) {
    select {
    case v := <-in:
        return v * 2, nil
    case <-time.After(3 * time.Second):
        return 0, ErrTimeout
    }
}

Let's replace time.Sleep() in the root goroutine with synctest.Wait():

func Test(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        var got int
        var err error

        go func() {
            ch := make(chan int)
            got, err = Calc(ch)
        }()

        // Doesn't wait for the Calc goroutine to finish.
        synctest.Wait()

        if err != ErrTimeout {
            t.Errorf("got: %v; want: %v", err, ErrTimeout)
        }
        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

--- FAIL: Test (0.00s)
    main_test.go:48: got: <nil>; want: timeout
panic: deadlock: main bubble goroutine has exited but blocked goroutines remain [recovered, repanicked]

synctest.Test panicked because the root bubble goroutine finished while the Calc goroutine was still blocked on a select.

Reason: synctest.Test only advances the clock if there is no active synctest.Wait running.

If all bubble goroutines are durably blocked but a synctest.Wait is running, synctest.Test won't advance the clock. Instead, it will simply finish the synctest.Wait call and return control to the goroutine that called it (in this case, the root bubble goroutine).

How to fix: don't use synctest.Wait.

The moment of unblocking is unclear

Let's update Calc to use context cancellation instead of a timer:

// Calc processes a value from the input channel.
// Exits if the context is canceled.
func Calc(in chan int, ctx context.Context) (int, error) {
    select {
    case v := <-in:
        return v * 2, nil
    case <-ctx.Done():
        return 0, ctx.Err()
    }
}

We won't cancel the context in the test:

func Test(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        ch := make(chan int)
        ctx, _ := context.WithCancel(context.Background())
        got, err := Calc(ch, ctx)

        if err != nil {
            t.Errorf("got: %v; want: nil", err)
        }
        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

--- FAIL: Test (0.00s)
panic: deadlock: all goroutines in bubble are blocked [recovered, repanicked]

synctest.Test panicked because all goroutines in the bubble are hopelessly blocked.

Reason: synctest.Test only advances the clock if it knows how much to advance it. In this case, there is no future moment that would unblock the select in Calc.

How to fix: Manually unblock the goroutine and call synctest.Wait to wait for it to finish.

func Test_fixed(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        var got int
        var err error
        ctx, cancel := context.WithCancel(context.Background())

        go func() {
            ch := make(chan int)
            got, err = Calc(ch, ctx)
        }()

        // Unblock the Calc goroutine.
        cancel()
        // Wait for it to finish.
        synctest.Wait()

        if err != context.Canceled {
            t.Errorf("got: %v; want: %v", err, context.Canceled)
        }
        if got != 0 {
            t.Errorf("got: %v; want: 0", got)
        }
    })
}

PASS

Now, cancel() cancels the context and unblocks the select in Calc, while synctest.Wait makes sure the Calc goroutine finishes before the test checks got and err.

The goroutine isn't durably blocked

Let's update Calc to lock the mutex before doing any calculations:

// Calc processes a value and returns the result.
func Calc(v int, mu *sync.Mutex) int {
    mu.Lock()
    defer mu.Unlock()
    v = v * 2
    return v
}