I work in a small developer team in a Golang-based private repo. With limited bandwidth, a monorepo seemed optimal to minimizing overhead and to spend more time writing code. Needing a solution for our growing codebase, transitioning to a multimodule monorepo proved effective.
This article will describe intricacies of a multimodule setup, and provide a full walkthrough initializing one. The walkthrough will be a side-by-side module structure, and contain cross-module imports and replace statements. For the full hosted example, see the github link below. Or the git gist at the bottom of this article.
Full Multimodule Walkthrough
This example will demonstrate how to setup your own multimodule monorepo. I've posted the code publicly at…
Working Towards a Multimodule Monorepo
Less overhead came with working in a single repo: one CI/CD pipeline setup, simpler pull requests, etcetera. Eventually, our team will grow past this monorepo. Too many devs in the pot will produce a lot of merge conflicts. At that time, we’ll simply pull modules out to their own repo.
This is the directory structure we ended up with. Your circumstances will dictate your own module structure, but here’s a few key takeaways.
$ tree repo/
│ ├── go.mod
│ ├── go.sum
│ └── main.go
│ ├── go.mod
│ └── go.sum
│ ├── go.mod
│ └── go.sum
You’ll notice we have four modules, not just two for the binaries.
coreholds our generalized code, usable by any module
toolsetare our binaries, and hold binary-specific code
integrationholds our integration tests for all the modules
core module is like a parent, and is only imported from. Both
toolset import from
integration imports both
A perk to having a separate integration module allows us to test separate module parts, without having to import them into the other. An example would testing our client’s GET request with an auth token to our server, and verifying in the database that a user exists. This also kept our module import graph acyclic, which I’ll touch on soon.
Multimodule Monorepo Aspects
Code is bundle in its own module. Like the name suggests, this is very modular and therefore rearranged easily. Utilizing
replace statements, you can easily choose local disk or an origin url.
Take this example of module
client importing module
core in its
module myvcs.org/repo/clientgo 1.14require (
gopkg.in/yaml.v2 v2.2.8 // indirect
)replace myvcs.org/repo/core v0.0.0 => ../core
Notice how the
core points back one directory. This is in relative path to the
client module, and only needs to adjusted slightly should
core be moved.
Customizable Import Paths
You can customize golang imports with a thing known as vanity import paths. If you go the semver-free multimodule route, you can use a vanity import path without needing to setup the http meta tag for redirection.
package my-package2import (
// typical import
// slightly shorter vanity import
With a vanity import and
replace statement combo, you can use any valid vanity import url to point to any local directory module. Just be sure to edit the importing
Personally, we found the shorter import path to be convenient. We also put in a vanity import path that we’ll use in the future.
Requires Acyclic Module Importing
You won’t be able to have your modules import each other in a cycle. Normally your code can bounce around a bit within a module, but splitting it into multiple modules walks a finer import line. If you do end up with a module import cycle, it may look like this:
import cycle not allowed
Circumstantially Semantic Version Maintainable
Bear with me. If you have an established, slowly-changing multimodule repo, even with modules importing each other, using semver tags can be maintainable. Alter one module, commit, tag changes, push and merge. Update other modules accordingly.
However, if you are planning on a fast-paced monorepo, often involving changes across multiple modules that import each other, using semver could be quite a bit of overhead. The small example below demonstrates what a minor instance of that overhead could look like.
module A and
module B are semver tagged
module A imports
module A needs changes that involves changes in
You alter, commit, tag, then push changes to origin
module BNow you alter, commit, tag, and push changes to origin
This could be even more complicated, with multiple modules importing one another and requiring changes. And if you want to test changes locally, you’ll have to manually add/alter module
replace statements, since your
go.modfiles point to origin, not your local disk. Gross.
Excel with Semver-Free Modules
Instead of tagging each module, use
replace statements to point to your other monorepo modules. Then, make cross-module changes and save with your favorite version control system. It’s that easy.
And, when the day comes to break out of one repo, move your modules out. Remove or change your existing
replace statements to point to the new module origin urls. Don’t forget to update the module semver tag, too.
Setting it Up
Here’s a git gist of the README from the walkthrough repo. It will walk you through the entire setup process. For copy-pastable commands and code, check out the README.md in the repo, or this git gist’s raw. Happy coding!