post://self-hosting-the-whole-git-workflow

Self-hosting the whole Git workflow

read: 8 min words: 1,441
Self-hosting the whole Git workflow
toc://sections
outline

    Most developers can self-host almost everything they depend on. Databases, queues, object storage, monitoring, all of it has a credible open-source path. The forge is the exception. The place where every commit, review, and release passes through is usually someone else's server with a per-seat price attached, and the workflow features that make it pleasant are treated as the moat that keeps you there.

    OpenCodeHub started as a question about that moat. If the pleasant parts are mostly workflow features, stacked diffs and merge queues and CI, then they are software, not magic, and software can be written. What I did not expect was how much of the work would go into the boring parts: making one codebase serve a browser, a Git client over HTTP and SSH, a GraphQL consumer, and a pipeline runner without splitting into eleven services.

    Repo: swadhinbiswas/OpencodeHub · Docs: docs.opencodehub.space

    The shape of the thing

    The whole platform is one application with clear internal seams, which is what makes a single container possible and a small team able to run it.

    flowchart TB
      C[Browser + Git CLI + och CLI] --> W[Web UI]
      C --> R[REST API - 175+ routes]
      C --> G[GraphQL endpoint]
      C --> GS[Git server - HTTP RPC]
      C --> SS[SSH server]
      W --> DB[(Postgres / SQLite / Turso)]
      R --> DB
      G --> DB
      GS --> DB
      R --> Q[(Redis queues)]
      Q --> WK[Worker - webhooks and automation]
      WK --> RUN[Runner - Docker executors]
      DB --> S3[(Pluggable S3 storage)]
    

    Three container images ship: the platform (web UI, REST and GraphQL API, Git over HTTP and SSH), a worker for queues and webhooks and automation, and a runner that executes pipelines in Docker. The images are around 251 MB, which is small enough that pulling the stack on a small server is not a project of its own.

    The persistence layer is deliberately swappable. Postgres for a real deployment, SQLite for a laptop, Turso when you want the database to live somewhere managed. That choice costs some care in the data layer, because the three engines do not agree on every detail, but it removes the "you need a database before you can try this" wall that keeps people from evaluating a self-hosted tool at all.

    Stacked pull requests

    The feature I reach for first on anything larger than a bug fix is the stack. A feature that touches the schema, the API, and the UI is three coherent changes pretending to be one, and as a single pull request it becomes a thousand-line review that nobody reads carefully.

    och stack create builds a chain of dependent branches. Each layer is a pull request against the one below it, so a reviewer can approve the schema change while the API on top of it is still moving. och stack submit pushes every branch and opens the linked pull requests. och stack sync rebases the whole stack when main moves, and och stack log draws the topology in the terminal so you can see what depends on what.

    The reason this belongs in the forge rather than a separate CLI is the review state. A stack has relationships between pull requests, and those relationships need to be first-class: which one is approved, which one is blocked, which one merged and left its children pointing at a gone parent. A third-party tool can only infer that from the API.

    A merge queue that does not serialize everything

    The second feature is the one that protects main without making everyone wait.

    If you validate pull requests one at a time against main, a queue of ten becomes a queue of ten builds, and the last author waits for work that has nothing to do with them. The alternative is speculative builds: when several pull requests are ready, each one is tested against the result of the ones ahead of it in the queue, so a build that passes proves the combination, not just the branch.

    <svg viewBox="0 0 900 280" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Three pull requests merging into main through the queue">
      <g fill="none" stroke="currentColor" stroke-opacity="0.4" stroke-width="2">
        <path d="M60,60 H520"/>
        <path d="M60,140 H520"/>
        <path d="M60,220 H520"/>
        <path d="M520,60 C620,60 640,140 700,140"/>
        <path d="M520,140 H700"/>
        <path d="M520,220 C620,220 640,140 700,140"/>
      </g>
      <g fill="currentColor" opacity="0.6" font-family="ui-monospace, monospace" font-size="13">
        <text x="60" y="40">PR 1</text>
        <text x="60" y="120">PR 2</text>
        <text x="60" y="200">PR 3</text>
      </g>
      <circle cx="700" cy="140" r="28" fill="none" stroke="#a6e3a1" stroke-width="3"/>
      <text x="700" y="146" text-anchor="middle" font-family="ui-monospace, monospace" font-size="14" fill="currentColor">main</text>
      <circle r="7" fill="#a6e3a1">
        <animateMotion dur="2.4s" repeatCount="indefinite" path="M60,60 H520 C620,60 640,140 700,140"/>
      </circle>
      <circle r="7" fill="#89b4fa">
        <animateMotion dur="2.4s" begin="0.8s" repeatCount="indefinite" path="M60,140 H700"/>
      </circle>
      <circle r="7" fill="#f9e2af">
        <animateMotion dur="2.4s" begin="1.6s" repeatCount="indefinite" path="M60,220 H520 C620,220 640,140 700,140"/>
      </circle>
    </svg>
    

    Speculative builds make the hard parts elsewhere. A failed build in the middle invalidates everything behind it, so the queue has to rebuild the tail, and the queue has to stay ordered while several lanes run in parallel. Priority lanes solve the other half of the problem: an urgent fix should not sit behind a long automatic queue, and a human needs a way to say "this one first" that the scheduler respects.

    The queue is also where the CLI earns its keep. och queue add 42, och queue status 42, and och queue list are what a developer actually types, and och focus opens a terminal cockpit that shows the lanes and the speculative build states live, so you can watch a queue without switching to a browser.

    CI without asking for a rewrite

    Pipelines run in isolated Docker executors and accept GitHub Actions YAML. That compatibility is a deliberate constraint, not a shortcut. A team evaluating a forge already has a repository full of workflows, and asking them to rewrite automation is the fastest way to lose them. Running what they already have is the difference between a migration that takes an afternoon and one that never happens.

    The runner image exists because pipeline execution is the one part of the system that should not share a process with the API. It runs Docker-in-Docker, which is heavy and occasionally hostile, and keeping it separate means a runaway build cannot take the platform down with it.

    AI review, provider-agnostic

    Automated review runs through GPT-4, Claude 3.5, Gemini 1.5, Groq, and a local Ollama model behind one interface. The value is a first pass before a human spends time: an obvious bug, a missing test, a naming inconsistency, the sort of thing that is easier to catch when something else reads the diff first.

    The provider abstraction is the point. Model quality changes every few weeks, and a review system wired to one vendor turns a model upgrade into a code change. Behind one interface, swapping providers is configuration.

    Storage and federation

    Storage is pluggable across the local filesystem and any S3-compatible bucket: AWS, MinIO, Cloudflare R2, Backblaze B2, or Ceph. A homelab deployment keeps everything local, and the same code points at object storage when the repositories and build artifacts outgrow the disk.

    Federation came from a real annoyance. Two teams running separate instances still share libraries, and without federation the only options are a shared account on one instance or a mirror that drifts. With it, one instance can fork from another, push branches, and open a pull request across the boundary. It is the feature that makes self-hosting a network instead of a set of islands.

    What the work actually was

    The impressive-sounding part, Git over HTTP and SSH and a pipeline runner in one binary, was mostly plumbing that had to be correct. The hard part was keeping one codebase coherent while it served very different clients, and the thing that made it tractable was refusing to split it into services before there was a reason. A modular monolith gets called a stepping stone, but for a team of one it is the difference between a system you can operate and a distributed system you cannot debug.

    The test suite is 679 unit, integration, and contract tests, which is partly about correctness and partly about being able to change the storage engine and the queue without fear. The deployment is Docker Compose, an admin seed script, and a browser at port 4321.

    There is a real trade-off in a project like this. A self-hosted forge will never have the ecosystem of a hosted one, and pretending otherwise would be dishonest. What it can have is ownership, no per-seat fee, and the workflow features that matter, in a form you can read and change. The repository and the API reference are linked above.

    Reach me at swadhinbiswas.cse@gmail.com or on GitHub and LinkedIn.

    react://self-hosting-the-whole-git-workflow
    comments://self-hosting-the-whole-git-workflow

    No comments yet.