Add containerization, web dashboard, push strategies, and code quality fixes#5
Add containerization, web dashboard, push strategies, and code quality fixes#5ev-the-dev merged 8 commits intoev-the-dev:mainfrom
Conversation
- Multi-stage Dockerfile (golang:1.24-alpine build, alpine:3.21 runtime) - Runs as non-root user with foreground daemon entrypoint - Add dynamic make help target with inline ## comments - Add .dockerignore to keep build context lean
for interval/retry config values, scrub credentials from git error output, cap concurrent syncs, add shutdown timeout, and clean up leaked file handles and temp files.
…fixes Adds push_strategy config field (mirror default, branches+tags for hosting platforms that reject refs/pull/*). Adds make targets for vet, lint, govulncheck, docker-build, and docker-run. Adds .golangci.yml and fixes errcheck warnings.
Optional web dashboard (net/http + html/template) showing repo sync status, manual sync trigger, recent errors, and /healthz endpoint. Per-mirror push_strategy (mirror or branches+tags) to avoid refs/pull/* conflicts with hosting platforms. GitHub Actions CI for vet, lint, govulncheck, build, and test. Dockerfile, docker make targets, golangci-lint config. Error handling and validation fixes across config, daemon, mirror, and logging packages. Updated README and example config.
Upgrade to Go 1.26
|
you have absolutely no obligation to review or merge this if you don't want. I just wanted to show you a few thoughts I had and you can take them or leave them. But mostly I just wanted you to know that somebody actually looked at your software and built it and actually used it to solve a task they had pending and will be continuing to use it. Other than adding the containerization and related Makefile stuff and the UI with the in-memory buffer it pulls from I really didn't have to do anything to get this running in my nomad cluster and successfully syncing some of my repos between github and my local forgejo just by reading the docs. If you don't count the time it took setting up the docker build stuff then I basically read your docs, made a config, set it to have vault inject the secret token, and deployed it to nomad and it worked first try and took basically no time because everything worked as advertised. Well done. |
daemon/daemon.go
Outdated
| return fmt.Errorf("repo %q not found", repoName) | ||
| } | ||
|
|
||
| if d.Status == nil { |
There was a problem hiding this comment.
I think this is a good check but I'm wondering if this instead means we should decouple the notion of Status being tied to the web config. The status package seems useful regardless of implementation. What are your thoughts @afreidah
There was a problem hiding this comment.
let me think on this for a minute. You might be correct, I just gotta refresh my memory on this. I'll give my thoughts shortly when I finish up something I'm working on with s3-orchestrator. Also @ev-the-dev , is there an easier way to communicate besides reddit...I rarely check comments there and often miss stuff.
There was a problem hiding this comment.
@ev-the-dev ok, I looked at it closer. agreed...the status store is light weight and useful beyond just the web ui, so it makes sense to always create it in the daemon regardless of the web config. That would also clean up the nil-check guards scattered through the daemon and make potential richer cli output possible too.
Good call out. I'll rework the PR real quick to be that way...cleaner and more useful.
There was a problem hiding this comment.
updated PR, see below comment.
There was a problem hiding this comment.
This makes sense for now, but I think long-term I'm going to publish binaries for different architectures and allow each user to either use the binary directly on a home server or simply download it via their own docker image.
There was a problem hiding this comment.
yeah, this wasn't really something that detailed thought went into, it will need some work, I just needed to slap something there real quick so I could build an image and publish it to my registry and test it in nomad. I 100% agree it should build/publish multi-arch (docker images, binaries, and debian packages or rpm or whatever you think is meaningful...I generally just do debian packages because I use debian for my laptop and all my cluster nodes). I do a slightly better job if it in s3-orchestrator which does multi-arch dockers and debs if you want to look at that. It is probably still a bit customized to my setup but I think I made most things variables you can override in ENV).
but yeah....this is definitely a placeholder more than a final idea...
There was a problem hiding this comment.
also, the other thing I was thinking of for future usage with the container is for integration testing where you also spin up two light-weight git servers (maybe gitea or something) in docker containers, configure repos, then spin up the gitgogit container configured to talk to the git containers and verify end-to-end functionality without the messiness of doing that directly on a local dev machine. docker-compose up; integration tests; docker-compose down. clean. then it can use that in CI too and tests run identically locally and in CI. Just a thought for the future.
…ption Always create the status store in the daemon regardless of web UI config, removing nil-check guards throughout. Add a per-mirror `force` config option that appends --force to git push commands for overwriting diverged refs.
|
Made these changes/additions based on your comment @ev-the-dev : Decoupled status store from web config The status.Store was previously only created when the web dashboard was enabled, with nil passed otherwise. This required nil-check guards scattered throughout daemon.go. Per your suggestion I made the daemon always create its own store internally in New(), since it's light weight and useful regardless of the web UI. Added per-mirror force config option To fix non-fast-forward push rejections when mirroring to target (in my case forgejo where refs had diverged), I added a force: bool field on MirrorTarget. When true, --force is appended to git push commands for both branches+tags and mirror strategies. Defaults to false so existing behavior is unchanged. Had to add this when testing the changes in my nomad cluster and one repo threw errors on sync due to a rebase on src that target didn't have. EDIT: a thought I had about this for the future...if an operator didn't want to enable force-push by default but needed to deal with a scenario like I ran into, they can see the log error, confirm they actually WANT to force-push to fix that and use the cli tool (or a button on the admin web-ui) to do a one-time force-sync on that repo but otherwise maintain non-force-sync behavior. Just thinking of the types of things as an operator/admin/devops guy you would be annoyed by but also allow stricter behavior by default for obvious reasons in a production environment. |
|
feel free to merge if you are happy with the updates, I'm happy to rework/remove anything you want. |
2 participants
Summary