#150 Internal Git Package Management

Merged
vimuser merged 1 commits from shmalebx9/gitframework into osboot/master 1 year ago

This commit introduces a sort of git package manager for osbmk. Hopefully, this makes it somewhat easier for users to jump between version of osbmk. Git projects managed by the internal git package manager are called 'gitmodules' in the included scripts. Every entry inside the text file resources/git/revisions is a 'gitmodule.'

Each gitmodule has a name, destination (location), revision hash, a url, and, optionally, a bkup url. Gitmodules are downloaded by the script at resources/scripts/download/gitmodule. The old download scripts link back to this script for compatability.

The script at .gitcheck has also been updated to read the revisions file and check if updates are needed for the gitmodule.

For example, running ./download gitmodule memtest86 will do the following:

  • Clone the git repo at url to a randomly generated tmp directory
  • Check out memtest86plus in the tmp dir to the specified revision
  • Check for patch files at resources/memtest86plus/patches/ and, if they exist, apply them in the tmp dir.
  • If all the previous steps are completed successfully, then the tmp dir is moved to location (memtest86plus/ in this case).

When .gitcheck is called, it will do the following:

  • Get a list of gitmodules from the revisions file automatically
  • For each gitmodule, check if the revision is the same as revision and exit successfully if it is
  • If revision differs from revision (always the case where a project is patched), then grep for revision in the last 15 commits. If it's not there, then warn the user that memtest86plus might be out of date and specify the command to update it.

Benefits:

  • Lots of redundant code is removed from each of the separate download scripts
  • The user does not end up with a borked clone of name if it fails to download or patches fail to apply
  • User can be certain that the gitmodule downloaded by osbmk is working after it's downloaded (since nothing will be there if not)
  • All git revisions are stored in one place, making it dead simple to test different versions. Testing a new version will only require modifying revision and redownloading with ./download gitmodule [name]

Drawbacks:

  • .gitcheck can only see whether the current version is ahead of the latest revision by <15 commits.
    • This means user generally gets a warning if the gitmodule is too old, but not if it's too new
  • Updates won't be automatically downloaded, the user only gets a warning
    • A possible solution is to have .gitcheck count the number of patches and then compare the revision in the revisions file to the git revision of the module n commits ago. Such a solution would add considerable overhead for little gain.
  • All patches must be placed according to the naming convention (resources/name/patches/)
This commit introduces a sort of git package manager for osbmk. Hopefully, this makes it somewhat easier for users to jump between version of osbmk. Git projects managed by the internal git package manager are called 'gitmodules' in the included scripts. Every entry inside the text file resources/git/revisions is a 'gitmodule.' Each gitmodule has a name, destination (location), revision hash, a url, and, optionally, a bkup url. Gitmodules are downloaded by the script at `resources/scripts/download/gitmodule.` The old download scripts link back to this script for compatability. The script at `.gitcheck` has also been updated to read the revisions file and check if updates are needed for the gitmodule. For example, running `./download gitmodule memtest86` will do the following: + Clone the git repo at `url` to a randomly generated tmp directory + Check out memtest86plus in the tmp dir to the specified `revision` + Check for patch files at `resources/memtest86plus/patches/` and, if they exist, apply them in the tmp dir. + If all the previous steps are completed successfully, then the tmp dir is moved to `location` (memtest86plus/ in this case). When `.gitcheck` is called, it will do the following: + Get a list of gitmodules from the revisions file automatically + For each gitmodule, check if the revision is the same as `revision` and exit successfully if it is + If revision differs from `revision` (always the case where a project is patched), then grep for `revision` in the last 15 commits. If it's not there, then warn the user that memtest86plus might be out of date and specify the command to update it. **Benefits:** + Lots of redundant code is removed from each of the separate download scripts + The user does not end up with a borked clone of `name` if it fails to download or patches fail to apply + User can be certain that the gitmodule downloaded by osbmk is working after it's downloaded (since nothing will be there if not) + All git revisions are stored in one place, making it dead simple to test different versions. Testing a new version will only require modifying `revision` and redownloading with `./download gitmodule [name]` **Drawbacks:** + `.gitcheck` can *only* see whether the current version is ahead of the latest revision by <15 commits. + This means user generally gets a warning if the gitmodule is too old, but not if it's too new + Updates won't be automatically downloaded, the user only gets a warning + A possible solution is to have `.gitcheck` count the number of patches and then compare the revision in the revisions file to the git revision of the module `n` commits ago. Such a solution would add considerable overhead for little gain. + All patches must be placed according to the naming convention (resources/`name`/patches/)
Leah Rowe commented 1 year ago
Owner

back at the helm again. sorry for not merging earlier. been a hectic year to say the least

back at the helm again. sorry for not merging earlier. been a hectic year to say the least
Leah Rowe commented 1 year ago
Owner

idea for follow up patch

what if gitcheck could sha1sum the patches

sort them in a list alphabetically by file name, cat them together (in /tmp) and get a sha1sum

sha1sum is quite fast and by doing this, you can then know if patches changed or not, and then just re-apply that way; can make it skip re-downloading if the revision hasn't changed (only if the .git folder is there, on releases it won't be)

piece of piss code-wise

idea for follow up patch what if gitcheck could sha1sum the patches sort them in a list alphabetically by file name, cat them together (in /tmp) and get a sha1sum sha1sum is quite fast and by doing this, you can then know if patches changed or not, and then just re-apply that way; can make it skip re-downloading if the revision hasn't changed (only if the .git folder is there, on releases it won't be) piece of piss code-wise
This pull request has been merged successfully!
Sign in to join this conversation.
No Label
No Milestone
No assignee
2 Participants
Loading...
Cancel
Save
There is no content yet.