Compare IP address prefixes the same way IANA does.

Ver. 1.26 • Stdlib • Low impact

Summary

An IP address prefix represents a IP subnet. These prefixes are usually written in CIDR notation:

10.0.0.0/8
127.0.0.0/8
169.254.0.0/16
203.0.113.0/24

In Go, an IP prefix is represented by the netip.Prefix type.

The new Prefix.Compare method lets you compare two IP prefixes, making it easy to sort them without having to write your own comparison code. The imposed order matches both Python's implementation and the assumed order from IANA.

Motivation

When the Go team initially designed the IP subnet type (net/netip.Prefix), they chose not to add a Compare method because there wasn't a widely accepted way to order these values.

Because of this, if a developer needs to sort IP subnets — for example, to organize routing tables or run tests — they have to write their own comparison logic. This results in repetitive and error-prone code.

The proposal aims to provide a standard way to compare IP prefixes. This should reduce boilerplate code and help programs sort IP subnets consistently.

Description

Add the Compare method to the netip.Prefix type:

// Compare returns an integer comparing two prefixes.
// The result will be 0 if p == p2, -1 if p < p2, and +1 if p > p2.
func (p Prefix) Compare(p2 Prefix) int

Compare orders two prefixes as follows:

• First by validity (invalid before valid).
• Then by address family (IPv4 before IPv6).
  10.0.0.0/8 < ::/8
• Then by masked IP address (network IP).
  10.0.0.0/8 < 10.0.1.0/8
• Then by prefix length.
  10.0.0.0/8 < 10.0.0.0/16
• Then by unmasked address (original IP).
  10.0.0.0/8 < 10.0.0.1/8

This follows the same order as Python's netaddr.IPNetwork and the standard IANA convention.

Example

Sort a list of IP prefixes:

prefixes := []netip.Prefix{
    netip.MustParsePrefix("10.0.1.0/8"),
    netip.MustParsePrefix("203.0.113.0/24"),
    netip.MustParsePrefix("10.0.0.0/8"),
    netip.MustParsePrefix("169.254.0.0/16"),
    netip.MustParsePrefix("203.0.113.0/8"),
}

slices.SortFunc(prefixes, func(a, b netip.Prefix) int {
    return a.Compare(b)
})

for _, p := range prefixes {
    fmt.Println(p.String())
}

10.0.0.0/8
10.0.1.0/8
169.254.0.0/16
203.0.113.0/8
203.0.113.0/24

Further reading

𝗣 61642 • 𝗖𝗟 700355

Concepts

vaqt is a partial port of Go's time package. It works with two types of values: Time and Duration.

Time is a pair (seconds, nanoseconds), where seconds is the 64-bit number of seconds since zero time (0001-01-01 00:00:00 UTC) and nanoseconds is the number of nanoseconds within the current second (0-999999999). Time can represent dates billions of years in the past or future with nanosecond precision.

  since     within
  0-time    second
┌─────────┬─────────────┐
│ seconds │ nanoseconds │
└─────────┴─────────────┘
  64 bit    32 bit

Time is always operated in UTC, but you can convert it from/to a specific timezone.

Duration is a 64-bit number of nanoseconds. It can represent values up to about 290 years.

┌─────────────┐
│ nanoseconds │
└─────────────┘
  64 bit

Features

vaqt provides functions for common date and time operations.

Creating time values:

time_now()
time_date(year, month, day, hour, min, sec, nsec, offset_sec)

Extracting time fields:

time_get_year(t)
time_get_month(t)
time_get_day(t)
time_get_clock(t)
time_get_hour(t)
time_get_minute(t)
time_get_second(t)
time_get_nano(t)
time_get_weekday(t)
time_get_yearday(t)
time_get_isoweek(t)

Unix time:

time_unix(sec, nsec)
time_unix_milli(msec)
time_unix_micro(usec)
time_unix_nano(nsec)
time_to_unix(t)
time_to_unix_milli(t)
time_to_unix_micro(t)
time_to_unix_nano(t)

Calendar time:

time_tm(tm, offset_sec)
time_to_tm(t, offset_sec)

Time comparison:

time_after(t, u)
time_before(t, u)
time_compare(t, u)
time_equal(t, u)
time_is_zero(t)

Time arithmetic:

time_add(t, d)
time_add_date(t, years, months, days)
time_sub(t, u)
time_since(t)
time_until(t)

Rounding:

time_truncate(t, d)
time_round(t, d)

Formatting:

time_fmt_iso(t, offset_sec)
time_fmt_datetime(t, offset_sec)
time_fmt_date(t, offset_sec)
time_fmt_time(t, offset_sec)
time_parse(s)

Marshaling:

time_unmarshal_binary(b);
time_marshal_binary(t)

Check the API reference for more details.

Usage example

Here's a basic example of how to use vaqt to work with time:

// Get current time.
Time t = time_now();

// Create time manually.
Time td = time_date(2011, TIME_NOVEMBER, 18, 15, 56, 35, 666777888, 0);

// Create time from Unix time.
Time tu = time_unix(1321631795, 666777888);

// Parse time from string.
Time tp = time_parse("2011-11-18T15:56:35.666777888Z");

// Get time parts.
int iso_year, iso_week;
time_get_isoweek(t, &iso_year, &iso_week);
int yearday = time_get_yearday(t);
enum Weekday weekday = time_get_weekday(t);

// Add duration to a time.
Time ta = time_add(t, 30 * TIME_SECOND);

// Subtract two times.
Time t1 = time_date(2024, TIME_AUGUST, 6, 21, 22, 45, 0, 0);
Time t2 = time_date(2024, TIME_AUGUST, 6, 21, 22, 15, 0, 0);
Duration d = time_sub(t1, t2);

// Convert time to string.
char buf[64];
size_t n = time_fmt_iso(t, 0, buf, sizeof(buf));
// buf = "2011-11-18T15:56:35.666777888Z"

Final thoughts

If you work with date and time in C, you might find vaqt useful.

See the nalgeon/vaqt repo for all the details.

Some concurrent operations don't require explicit synchronization. We can use these to create lock-free types and functions that are safe to use from multiple goroutines. Let's dive into the topic!

Non-atomic increment • Atomic operations • Composition • Atomic vs. mutex • Keep it up

Non-atomic increment

Suppose multiple goroutines increment a shared counter:

total := 0

var wg sync.WaitGroup
for range 5 {
    wg.Go(func() {
        for range 10000 {
            total++
        }
    })
}
wg.Wait()

fmt.Println("total", total)

total 40478

There are 5 goroutines, and each one increments total 10,000 times, so the final result should be 50,000. But it's usually less. Let's run the code a few more times:

total 26775
total 22978
total 30357

The race detector is reporting a problem:

$ go run -race total.go
==================
WARNING: DATA RACE
...
==================
total 33274
Found 1 data race(s)

This might seem strange — shouldn't the total++ operation be atomic? Actually, it's not. It involves three steps (read-modify-write):

1. Read the current value of total.
2. Add one to it.
3. Write the new value back to total.

If two goroutines both read the value 42, then each increments it and writes it back, the new total will be 43 instead of 44 like it should be. As a result, some increments to the counter will be lost, and the final value will be less than 50,000.

As we talked about in the Race conditions chapter, you can make an operation atomic by using mutexes or other synchronization tools. But for this chapter, let's agree not to use them. Here, when I say "atomic operation", I mean an operation that doesn't require the caller to use explicit locks, but is still safe to use in a concurrent environment.

Atomic operations

An operation without synchronization can only be truly atomic if it translates to a single processor instruction. Such operations don't need locks and won't cause issues when called concurrently (even the write operations).

In a perfect world, every operation would be atomic, and we wouldn't have to deal with mutexes. But in reality, there are only a few atomics, and they're all found in the sync/atomic package. This package provides a set of atomic types:

• Bool — a boolean value;
• Int32/Int64 — a 4- or 8-byte integer;
• Uint32/Uint64 — a 4- or 8-byte unsigned integer;
• Value — a value of any type;
• Pointer — a pointer to a value of type T (generic).

Each atomic type provides the following methods:

Load reads the value of a variable, Store sets a new value:

var n atomic.Int32
n.Store(10)
fmt.Println("Store", n.Load())

Store 10

Swap sets a new value (like Store) and returns the old one:

var n atomic.Int32
n.Store(10)
old := n.Swap(42)
fmt.Println("Swap", old, "->", n.Load())

Swap 10 -> 42

CompareAndSwap sets a new value only if the current value is still what you expect it to be:

var n atomic.Int32
n.Store(10)
swapped := n.CompareAndSwap(10, 42)
fmt.Println("CompareAndSwap 10 -> 42:", swapped)
fmt.Println("n =", n.Load())

CompareAndSwap 10 -> 42: true
n = 42

var n atomic.Int32
n.Store(10)
swapped := n.CompareAndSwap(33, 42)
fmt.Println("CompareAndSwap 33 -> 42:", swapped)
fmt.Println("n =", n.Load())

CompareAndSwap 33 -> 42: false
n = 10

Numeric types also provide an Add method that increments the value by the specified amount:

var n atomic.Int32
n.Store(10)
n.Add(32)
fmt.Println("Add 32:", n.Load())

Add 32: 42

And the And/Or methods for bitwise operations (Go 1.23+):

const (
    modeRead  = 0b100
    modeWrite = 0b010
    modeExec  = 0b001
)

var mode atomic.Int32
mode.Store(modeRead)
old := mode.Or(modeWrite)

fmt.Printf("mode: %b -> %b\n", old, mode.Load())

mode: 100 -> 110

All methods are translated to a single CPU instruction, so they are safe for concurrent calls.

Strictly speaking, this isn't always true. Not all processors support the full set of concurrent operations, so sometimes more than one instruction is needed. But we don't have to worry about that — Go guarantees the atomicity of sync/atomic operations for the caller. It uses low-level mechanisms specific to each processor architecture to do this.

Like other synchronization primitives, each atomic variable has its own internal state. So, you should only pass it as a pointer, not by value, to avoid accidentally copying the state.

When using atomic.Value, all loads and stores should use the same concrete type. The following code will cause a panic:

var v atomic.Value
v.Store(10)
v.Store("hi")

panic: sync/atomic: store of inconsistently typed value into Value

Now, let's go back to the counter program:

total := 0

var wg sync.WaitGroup
for range 5 {
    wg.Go(func() {
        for range 10000 {
            total++
        }
    })
}
wg.Wait()

fmt.Println("total", total)

And rewrite it to use an atomic counter:

var total atomic.Int32

var wg sync.WaitGroup
for range 5 {
    wg.Go(func() {
        for range 10000 {
            total.Add(1)
        }
    })
}
wg.Wait()

fmt.Println("total", total.Load())

total 50000

Much better!

Atomics composition

An atomic operation in a concurrent program is a great thing. Such operation usually transforms into a single processor instruction, and it does not require locks. You can safely call it from different goroutines and receive a predictable result.

But what happens if you combine atomic operations? Let's find out.

Atomicity

Let's look at a function that increments a counter:

var counter int32

// increment increases the counter value by two.
func increment() {
    counter += 1
    sleep(10)
    counter += 1
}

// sleep pauses the current goroutine for up to maxMs ms.
func sleep(maxMs int) {
    dur := time.Duration(rand.IntN(maxMs)) * time.Millisecond
    time.Sleep(dur)
}

As you already know, increment isn't safe to call from multiple goroutines because counter += 1 causes a data race.

Now I will try to fix the problem and propose several options. In each case, answer the question: if you call increment from 100 goroutines, is the final value of the counter guaranteed?

Example 1:

var counter atomic.Int32

func increment() {
    counter.Add(1)
    sleep(10)
    counter.Add(1)
}

counter = 200

Is the counter value guaranteed?

Example 2:

var counter atomic.Int32

func increment() {
    if counter.Load()%2 == 0 {
        sleep(10)
        counter.Add(1)
    } else {
        sleep(10)
        counter.Add(2)
    }
}

counter = 184

Is the counter value guaranteed?

Example 3:

var delta atomic.Int32
var counter atomic.Int32

func increment() {
    delta.Add(1)
    sleep(10)
    counter.Add(delta.Load())
}

counter = 9386

Is the counter value guaranteed?

Composition

People sometimes think that the composition of atomic operations also magically becomes an atomic operation. But it doesn't.

For example, the second of the above examples:

var counter atomic.Int32

func increment() {
    if counter.Load()%2 == 0 {
        sleep(10)
        counter.Add(1)
    } else {
        sleep(10)
        counter.Add(2)
    }
}

Call increment 100 times from different goroutines:

var wg sync.WaitGroup
for range 100 {
    wg.Go(increment)
}
wg.Wait()
fmt.Println("counter =", counter.Load())

Run the program with the -race flag — there are no races:

% go run atomic-2.go
192
% go run atomic-2.go
191
% go run atomic-2.go
189

But can we be sure what the final value of counter will be? Nope. counter.Load and counter.Add calls are interleaved from different goroutines. This causes a race condition (not to be confused with a data race) and leads to an unpredictable counter value.

Check yourself by answering the question: in which example is increment an atomic operation?

Sequence independence

In all examples, increment is not an atomic operation. The composition of atomics is always non-atomic.

The first example, however, guarantees the final value of the counter in a concurrent environment:

var counter atomic.Int32

func increment() {
    counter.Add(1)
    sleep(10)
    counter.Add(1)
}

If we run 100 goroutines, the counter will ultimately equal 200.

The reason is that Add is a sequence-independent operation. The runtime can perform such operations in any order, and the result will not change.

The second and third examples use sequence-dependent operations. When we run 100 goroutines, the order of operations is different each time. Therefore, the result is also different.

A bulletproof way to make a composite operation atomic and prevent race conditions is to use a mutex:

var delta int32
var counter int32
var mu sync.Mutex

func increment() {
    mu.Lock()
    delta += 1
    sleep(10)
    counter += delta
    mu.Unlock()
}

// After 100 concurrent increments, the final value is guaranteed:
// counter = 1+2+...+100 = 5050

counter = 5050

But sometimes an atomic variable with CompareAndSwap is all you need. Let's look at an example.

Atomic instead of mutex

Let's say we have a gate that needs to be closed:

type Gate struct {
    closed bool // gate state
}

func (g *Gate) Close() {
    if g.closed {
        return // ignore repeated calls
    }
    g.closed = true
    // free resources
}

func work() {
	var g Gate
	defer g.Close()
	// do something while the gate is open
}

In a concurrent environment, there are data races on the closed field. We can fix this with a mutex:

type Gate struct {
    closed bool
    mu sync.Mutex // protects the state
}

func (g *Gate) Close() {
    g.mu.Lock()
    defer g.mu.Unlock()
    if g.closed {
        return // ignore repeated calls
    }
    g.closed = true
    // free resources
}

Alternatively, we can use CompareAndSwap on an atomic Bool instead of a mutex:

type Gate struct {
    closed atomic.Bool
}

func (g *Gate) Close() {
    if !g.closed.CompareAndSwap(false, true) {
        return // ignore repeated calls
    }
    // The gate is closed.
    // We can free resources now.
}

The Gate type is now more compact and simple.

This isn't a very common use case — we usually want a goroutine to wait on a locked mutex and continue once it's unlocked. But for "early exit" situations, it's perfect.

Keep it up

Atomics are a specialized but useful tool. You can use them for simple counters and flags, but be very careful when using them for more complex operations. You can also use them instead of mutexes to exit early.

In the next chapter, we'll talk about testing concurrent code (coming soon).

Pre-order for $10   or read online

Provide a consistent approach to hashing and equality checks in custom data structures.

Ver. 1.26 • Stdlib • Medium impact

Summary

The new maphash.Hasher interface is the standard way to hash and compare elements in custom collections, such as custom maps or sets:

// A Hasher implements hashing and equality for type T.
type Hasher[T any] interface {
    Hash(hash *maphash.Hash, value T)
    Equal(T, T) bool
}

The ComparableHasher type is the default hasher implementation for comparable types, like numbers, strings, and structs with comparable fields.

Motivation

The maphash package offers hash functions for byte slices and strings, but it doesn't provide any guidance on how to create custom hash-based data structures.

The proposal aims to improve this by introducing hasher — a standardized interface for hashing and comparing the members of a collection, along with a default implementation.

Description

Add the hasher interface to the maphash package:

// A Hasher is a type that implements hashing and equality for type T.
//
// A Hasher must be stateless and their zero value must be valid.
// Hence, typically, a Hasher will be an empty struct.
type Hasher[T any] interface {
    // Hash updates hash to reflect the contents of value.
    //
    // If two values are [Equal], they must also Hash the same.
    // Specifically, if Equal(a, b) is true, then Hash(h, a) and Hash(h, b)
    // must write identical streams to h.
    Hash(hash *maphash.Hash, value T)
    Equal(T, T) bool
}

Along with the default hasher implementation for comparable types:

// ComparableHasher is an implementation of [Hasher] for comparable types.
// Its Equal(x, y) method is consistent with x == y.
type ComparableHasher[T comparable] struct{}
func (h ComparableHasher[T]) Hash(hash *Hash, value T)
func (h ComparableHasher[T]) Equal(x, y T) bool

Example

Here's a case-insensitive string hasher:

// CaseInsensitive is a Hasher for case-insensitive string comparison.
type CaseInsensitive struct{}

func (CaseInsensitive) Hash(h *maphash.Hash, s string) {
    // Use lowercase instead of case folding to keep things simple.
    h.WriteString(strings.ToLower(s))
}

func (CaseInsensitive) Equal(a, b string) bool {
    return strings.ToLower(a) == strings.ToLower(b)
}

And a generic Set that uses a pluggable hasher for custom equality and hashing:

// Set is a generic set implementation with a pluggable hasher.
// Stores values in a map of buckets identified by the value's hash.
// If multiple values end up in the same bucket,
// uses linear search to find the right one.
type Set[H maphash.Hasher[V], V any] struct {
    seed   maphash.Seed   // random seed for hashing
    hasher H              // performs hashing and equality checks
    data   map[uint64][]V // buckets of values indexed by their hash
}

// NewSet creates a new Set instance with the provided hasher.
func NewSet[H maphash.Hasher[V], V any](hasher H) *Set[H, V] {
    return &Set[H, V]{
        seed:   maphash.MakeSeed(),
        hasher: hasher,
        data:   make(map[uint64][]V),
    }
}

The calcHash helper method uses the hasher to compute the hash of a value:

// calcHash computes the hash of a value.
func (s *Set[H, V]) calcHash(val V) uint64 {
    var h maphash.Hash
    h.SetSeed(s.seed)
    s.hasher.Hash(&h, val)
    return h.Sum64()
}

This hash is used in the Has and Add methods. It acts as a key in the bucket map to find the right bucket for a value.

Has checks if the value exists in the corresponding bucket:

// Has returns true if the given value is present in the set.
func (s *Set[H, V]) Has(val V) bool {
    hash := s.calcHash(val)
    if bucket, ok := s.data[hash]; ok {
        for _, item := range bucket {
            if s.hasher.Equal(val, item) {
                return true
            }
        }
    }
    return false
}

Add adds a value to the corresponding bucket:

// Add adds the value to the set if it is not already present.
func (s *Set[H, V]) Add(val V) {
    if s.Has(val) {
        return
    }
    hash := s.calcHash(val)
    s.data[hash] = append(s.data[hash], val)
}

Now we can create a case-insensitive string set:

func main() {
    set := NewSet(CaseInsensitive{})

    set.Add("hello")
    set.Add("world")

    fmt.Println(set.Has("HELLO")) // true
    fmt.Println(set.Has("world")) // true
}

true
true

Or a regular string set using maphash.ComparableHasher:

func main() {
    set := NewSet(maphash.ComparableHasher[string]{})

    set.Add("hello")
    set.Add("world")

    fmt.Println(set.Has("HELLO")) // false
    fmt.Println(set.Has("world")) // true
}

false
true

Further reading

𝗣 70471 • 𝗖𝗟 657296 (in progress)

Learn to decompose problems into smaller chunks, be specific about what you want, pick the right AI model for the task, and iterate on your prompts.

Don't do this.

I mean, "learn to decompose the problem" — sure. "Iterate on your prompts" — not so much. Write the actual code instead:

• Ask AI for an initial version and then refactor it to match your expectations.
• Write the initial version yourself and ask AI to review and improve it.
• Write the critical parts and ask AI to do the rest.
• Write an outline of the code and ask AI to fill the missing parts.

You probably see the pattern now. Get involved with the code, don't leave it all to AI.

If, given the prompt, AI does the job perfectly on first or second iteration — fine. Otherwise, stop refining the prompt. Go write some code, then get back to the AI. You'll get much better results.

Don't get me wrong: this is not anti-AI advice. Use it, by all means. Use it a lot if you want to. But don't fall into the trap of endless back-and-forth prompt refinement, trying to get the perfect result from AI by "programming in English". It's an imprecise, slow and terribly painful way to get things done.

Get your hands dirty. Write the code. It's what you are good at.

You are a software engineer. Don't become a prompt refiner.

Newer languages ranking 2025
(v1 after 2010)

    TS  ■■■■■■■■■■■■■■■■■■■■
    Go  ■■■■■■■■■■
  Rust  ■■■■■■■■
Kotlin  ■■■■■■
  Dart  ■■■■■
 Swift  ■■■■

Sources: IEEE, Stack Overflow, Languish. I'm not using TIOBE because their method has major flaws.

TypeScript's position is very strong, of course (I guess no one likes JavaScript these days). And it's great to see that more and more developers are choosing Go for the backend. Also, Rust scores very close in all rankings except IEEE, so we'll see what happens in the coming years.

Way to Go!

Allow the new built-in to be called on expressions.

Ver. 1.26 • Language • High impact

Summary

Previously, you could only use the new built-in with types:

p1 := new(int)
fmt.Println(*p1)
// 0

Now you can also use it with expressions:

// Pointer to a int variable with the value 42.
p := new(42)
fmt.Println(*p)
// 42

If the argument expr is an expression of type T, then new(expr) allocates a variable of type T, initializes it to the value of expr, and returns its address, a value of type *T.

Motivation

There's an easy way to create a pointer to a composite literal:

type Person struct { name string }
p := &Person{name: "alice"}

But no easy way to create a pointer to a value of simple type:

n := 42
p := &n

The proposal aims to fix this.

Description

Update the Allocation section of the language specification as follows:

➀ is the new part, ➁ already worked as described.

Examples

Pointer to a simple type:

// go 1.25
n := 42
p1 := &n
fmt.Println(*p1)

s := "go"
p2 := &s
fmt.Println(*p2)

42
go

// go 1.26
p1 := new(42)
fmt.Println(*p1)

s := "go"
p2 := new(s)
fmt.Println(*p2)

42
go

Pointer to a composite value:

// go 1.25
s := []int{11, 12, 13}
p1 := &s
fmt.Println(*p1)

type Person struct{ name string }
p2 := &Person{name: "alice"}
fmt.Println(*p2)

[11 12 13]
{alice}

// go 1.26
p1 := new([]int{11, 12, 13})
fmt.Println(*p1)

type Person struct{ name string }
p2 := new(Person{name: "alice"})
fmt.Println(*p2)

[11 12 13]
{alice}

Pointer to the result of a function call:

// go 1.25
f := func() string { return "go" }
v := f()
p := &v
fmt.Println(*p)

go

// go 1.26
f := func() string { return "go" }
p := new(f())
fmt.Println(*p)

go

Passing nil is still not allowed:

// go 1.25 and go 1.26
p := new(nil)
// compilation error

Further reading

𝗣 45624 • 𝗖𝗟 704935, 704737, 704955, 705157

For each accepted proposal, I'll write a one-page summary that explains the change in simple terms.

This should (hopefully) be the easiest way to keep up with upcoming changes without having to read through 2,364 comments on Go's GitHub repo.

Here's a sneak peak:

The plan is to publish the already accepted proposals from the upcoming 1.26 release, and then publish new ones as they get accepted. I'll probably skip the minor ones, but we'll see. Stay tuned!

To safely manage threads and processes in Go, I'd normally need to modify Go's internals. But since this is just a research project, I chose to (ab)use cgo and syscalls instead. That's how I created multi — a small package that explores unconventional ways to handle concurrency in Go.

Features • Goroutines • Threads • Processes • Benchmarks • Final thoughts

Features

Multi offers three types of "concurrent groups". Each one has an API similar to sync.WaitGroup, but they work very differently under the hood:

• goro.Group runs Go functions in goroutines that are locked to OS threads. Each function executes in its own goroutine. Safe to use in production, although unnecessary, because the regular non-locked goroutines work just fine.

• pthread.Group runs Go functions in separate OS threads using POSIX threads. Each function executes in its own thread. This implementation bypasses Go's runtime thread management. Calling Go code from threads not created by the Go runtime can lead to issues with garbage collection, signal handling, and the scheduler. Not meant for production use.

• proc.Group runs Go functions in separate OS processes. Each function executes in its own process forked from the main one. This implementation uses process forking, which is not supported by the Go runtime and can cause undefined behavior, especially in programs with multiple goroutines or complex state. Not meant for production use.

All groups offer an API similar to sync.WaitGroup.

goro.Group

Runs Go functions in goroutines that are locked to OS threads.

ch := make(chan int, 2)

g := goro.NewGroup()
g.Go(func() error {
    // do something
    ch <- 42
    return nil
})
g.Go(func() error {
    // do something
    ch <- 42
    return nil
})
g.Wait()

goro.Group starts a regular goroutine for each Go call, and assigns it to its own thread. Here's a simplified implementation:

// Thread represents a goroutine locked to an OS thread.
type Thread struct {
    f       func()
    done    chan struct{}
}

// Start launches the goroutine.
func (t *Thread) Start() {
    go func() {
        runtime.LockOSThread()
        defer runtime.UnlockOSThread()
        t.f()
        close(t.done)
    }()
}

// Wait blocks until the goroutine completes.
func (t *Thread) Wait() {
    <-t.done
}

goro/thread.go

You can use channels and other standard concurrency tools inside the functions managed by the group.

pthread.Group

Runs Go functions in separate OS threads using POSIX threads.

ch := make(chan int, 2)

g := pthread.NewGroup()
g.Go(func() error {
    // do something
    ch <- 42
    return nil
})
g.Go(func() error {
    // do something
    ch <- 42
    return nil
})
g.Wait()

pthread.Group creates a native OS thread for each Go call. It uses cgo to start and join threads. Here is a simplified implementation:

/*
#include <pthread.h>

extern void* threadFunc(void*);
*/
import "C"

// Thread represents a Go function executed in a separate OS thread.
type Thread struct {
    tid     C.pthread_t
    f       func()
}

// Start launches the thread.
func (t *Thread) Start() {
    h := cgo.NewHandle(t) // ensure t is kept alive for the thread lifetime
    C.pthread_create(&t.tid, nil, (*[0]byte)(C.threadFunc), unsafe.Pointer(h))
}

// Wait blocks until the thread completes.
func (t *Thread) Wait() {
    C.pthread_join(t.tid, nil)
}

//export threadFunc
func threadFunc(arg unsafe.Pointer) unsafe.Pointer {
    h := cgo.Handle(arg)
    t := h.Value().(*Thread)
    t.f()
    h.Delete()
    return nil
}

pthread/thread.go

You can use channels and other standard concurrency tools inside the functions managed by the group.

proc.Group

Runs Go functions in separate OS processes forked from the main one.

ch := proc.NewChan[int]()
defer ch.Close()

g := proc.NewGroup()
g.Go(func() error {
    // do something
    ch.Send(42)
    return nil
})
g.Go(func() error {
    // do something
    ch.Send(42)
    return nil
})
g.Wait()

proc.Group forks the main process for each Go call. It uses syscalls to fork processes and wait for them to finish. Here is a simplified implementation:

// Process represents a Go function executed in a forked OS process.
type Process struct {
    f       func()
    pid     int
}

// Start launches the process.
func (p *Process) Start() {
    pid, _, _ := syscall.Syscall(syscall.SYS_FORK, 0, 0, 0)

    if pid != 0 {
        // In parent.
        p.pid = int(pid)
        return
    }

    // Fork succeeded, now in child.
    p.f()
    os.Exit(0)
}

// Wait blocks until the process completes.
func (p *Process) Wait() {
    var ws syscall.WaitStatus
    syscall.Wait4(p.pid, &ws, 0, nil)
}

proc/process.go

You can only use proc.Chan to exchange data between processes, since regular Go channels and other concurrency tools don't work across process boundaries.

Benchmarks

Running some CPU-bound workload (with no allocations or I/O) on Apple M1 gives these results:

goos: darwin
goarch: arm64
ncpu: 8
gomaxprocs: 8
workers: 4
sync.WaitGroup: n=100   t=60511 µs/exec
goro.Group:     n=100   t=60751 µs/exec
pthread.Group:  n=100   t=60791 µs/exec
proc.Group:     n=100   t=61640 µs/exec

And here are the results from GitHub actions:

goos: linux
goarch: amd64
ncpu: 4
gomaxprocs: 4
workers: 4
sync.WaitGroup: n=100   t=145256 µs/exec
goro.Group:     n=100   t=145813 µs/exec
pthread.Group:  n=100   t=148968 µs/exec
proc.Group:     n=100   t=147572 µs/exec

One execution here means a group of 4 workers each doing 10 million iterations of generating random numbers and adding them up. See the benchmark code for details.

As you can see, the default concurrency model (sync.WaitGroup in the results, using standard goroutine scheduling without meddling with threads or processes) works just fine and doesn't add any noticeable overhead. You probably already knew that, but it's always good to double-check, right?

Final thoughts

I don't think anyone will find these concurrent groups useful in real-world situations, but it's still interesting to look at possible (even if flawed) implementations and compare them to Go's default (and only) concurrency model.

Check out the nalgeon/multi repo for the implementation.

──

P.S. Want to learn more about concurrency? Check out my interactive book

Here's a toy example:

// Given a channel of documents.
docs := make(chan []string, 10)
docs <- []string{"go", "is", "awesome"}
docs <- []string{"cats", "are", "cute"}
close(docs)

// Extract all words from the documents.
words := make(chan string, 10)
chans.Flatten(ctx, words, docs)
close(words)

// Calculate the total byte count of all words.
step := func(acc int, word string) int { return acc + len(word) }
count := chans.Reduce(ctx, words, 0, step)
fmt.Println("byte count =", count)

byte count = 22

Now let's go over the features.

The all-time classics

Filtering and sampling

Batching and windowing

De-duplication

Routing

Motivation

I think third-party concurrency packages are often too opinionated and try to hide too much complexity. As a result, they end up being inflexible and don't fit a lot of use cases.

For example, here's how you use the Map function from the rill package:

// Concurrency = 3
users := rill.Map(ids, 3, func(id int) (*User, error) {
    return db.GetUser(ctx, id)
})

The code looks simple, but it makes Map pretty opinionated and not very flexible:

• The function is non-blocking and spawns a goroutine. There is no way to change this.
• The function doesn't exit early on error. There is no way to change this.
• The function creates the output channel. There is no way to control its buffering or lifecycle.
• The function can't be canceled.
• The function requires the developer to use a custom Try[T] type for both input and output channels.
• The "N workers" logic is baked in, so you can't use a custom concurrent group implementation.

While this approach works for many developers, I personally don't like it. With chans, my goal was to offer a fairly low-level set of composable channel operations and let developers decide how to use them.

For comparison, here's how you use the chans.Map function:

err := chans.Map(ctx, users, ids, func(id int) (*User, error) {
    return db.GetUser(ctx, id)
})

chans.Map only implements the core mapping logic:

• Reads values from the input channel.
• Calls the mapping function on each value.
• Writes results to the output channel.
• Stops if there's an error or if the context is canceled.
• Does not start any additional goroutines.

You decide the rest:

• Want Map to be non-blocking? Run it in a goroutine.
• Don't want to exit early? Gather the errors instead of returning them.
• Want to buffer the output channel or keep it open? You have full control.
• Need to process input in parallel? Use errgroup.Group, or sync.WaitGroup, or any other implementation.

The same principles apply to other channel operations.

Usage example

Let's say we want to calculate the total balance of VIP user accounts:

func calcVIPsBalance(ctx context.Context, ids chan string) (float64, error) {
	// ...
}

Here's how we can do it using chans.

First, use Map to get the accounts from the database:

errs := make([]error, 2)
accounts := make(chan *Account)
go func() {
    getAcc := func(id string) (*Account, error) {
        return db.GetAccount(ctx, id)
    }
    err := chans.Map(ctx, accounts, ids, getAcc)
    errs[0] = err
    close(accounts)
}()

Next, use Filter to select only the VIP accounts:

vips := make(chan *Account)
go func() {
    onlyVIP := func(acc *Account) (bool, error) {
        return acc.VIP, nil
    }
    err := chans.Filter(ctx, vips, accounts, onlyVIP)
    errs[1] = err
    close(vips)
}()

Next, use Reduce to calculate the total balance:

sumBalance := func(total float64, acc *Account) float64 {
    return total + acc.Balance
}
total := chans.Reduce(ctx, vips, 0, sumBalance)

Finally, check for errors and return the result:

err := errors.Join(errs[0], errs[1], ctx.Err())
if err != nil {
    return 0, err
}
return total, nil

That's it!

Final thoughts

If you're building concurrent pipelines in Go, you might find chans useful.

See the nalgeon/chans repo if you are interested.

──

P.S. Want to learn more about concurrency? Check out my interactive book