[git-aggregator] commit status inside a project

[Mat-moran]

Good morning,

I was talking with a friend yesterday about the usage of git-aggregator instead of git submodules to manage the addons of a project, because he says that with with git-aggregator the state of the modules are automatically updated to the latest commit if you aggregate.

For example if you want to clone a project the addons are not going to have the same state due to gitaggregator and basically each time you rebuild the image are going to update to the latest version. I am not judging if this is god or bad approach, but I think that can be a functionality of doodba (or of git-aggregator) to be able to point a certain state/commits.

I made 2 script that does this functionality( the first approach, probably is a better or simpler way.)

save_last_commit.py ➡️ this save the commits state in a project when you run the script.
checkout_status.py ➡️ thit checkout the repo to the commit in save_last_commit.py

I would like to know if this is an interesting feature to be added to doodba. this status should be saved in de filestore or somewhere in a permanent volume to be able to backup. And later on be able to recover passing this file via the docker-compose.yaml and a VARIABLE to run checkout_status.py or not.

Maybe some concepts are wrong because I am relatively new to doodba and git-aggregater, maybe is a way to do that and I do not know...

Regards and thanks again for this project!

save_last_commit.py

import os
from git import Repo
import yaml

commit_state = {}

folders = os.listdir(os.chdir("./"))
for folder in folders:
    try:
        repo = Repo(folder)
        commit = repo.rev_parse("HEAD")
        commit_state[folder] = str(commit)
    except Exception:
        pass

print(commit_state)

with open("commits_status.yaml", "w") as file:
    yaml.dump(commit_state, file)

checkout_status.py

import os
from git import Repo
import yaml


with open("commits_status.yaml", "r") as file:
    checkout_status = yaml.load(file, Loader=yaml.FullLoader)


folders = os.listdir(os.chdir("./"))
for folder in folders:
    try:
        repo = Repo(folder)
        print(folder, ":--{>}", checkout_status[folder])
        repo.git.checkout(checkout_status[folder])

    except Exception:
        pass

[pedrobaeza]

This already exists. Instead of the environment variable ODOO_VERSION, put the commit sha.

[Mat-moran]

Yes but this is tedious and you should manage each repo independently in repos.yaml right? So you loose the good of managing all the repos with a simple YAML addons file that for me is beautiful!.

Imagine this scenario:

• my server is broken
• I need to restore a backup ( my project is only manged via addons.yaml, in repos.yaml I have only the defaults )
• I rebuild my image in another server via cloning my doodba-repo.
• Consecuently I rebuild the database with other addons version that I do not know how is going to behave, and the most important **I have loosed the commit where I was because this information was not set in any permanent volume, only in the image that is broken. **

With this approach we are able to keep where the project was in the broken serve via restauring the backup where you can find a file with the name of the repo and the commit they was pointing. I you want you can using it, if you do not want just set the variable to FALSE, and it will continue doing the actual way.

[agent-z28]

I totally agree with your requirement :)

[yajo]

I think that there are 2 proper solutions for your problem:

1. Use an image registry. Thus, build your project in CI, test it and, if tests pass, push the image to a registry. Then you can tag your image with whatever tagging system you want (semver, calver, custom, or just rely in the automated sha-encoded tag...). If you need the same image again, you can just pull it from your registry, and then all your environment is the same (not just the git code, but every byte of your deployment).
2. Enhance git-aggregator itself to support lockfiles. In a similar fashion of how Poetry uses both the pyproject.toml file to specify dependency constraints + poetry.lock to provide reproducible builds, git-aggregator could support using the repos.yaml file to specify merges + some git-aggregator.lock to provide reproducible builds.

We use option 1, which I think is better, and besides it already exists.

If you want option 2 (that's what you're asking here, actually), then the place to implement it and to ask for feedback is git-aggregator itself, because it isn't specific to doodba.

[agent-z28]

@yajo Option 1) will loose the possibility to patch a running version with a cherry-pick, since you cannot rebuild the same environment