https://okatsn.github.io/OkPkgTemplates.jl/
OkPkgTemplates
initiate julia package with CI presets that automate test, release tag, documentation and registry management based on okatsn/OkRegistry with my personal preferences. This package is developed referring PkgTemplates.jl especially the part Saving Templates.
You can fork this repository and replace okatsn/OkRegistry
and its path by your own.
@genpkg "NewPackage"
and @upactions
generate/update files base on the information of your current julia environment at one click. By default @genpkg
generate your NewPackage
in JULIA_PKG_DEVDIR
(environment variable for your path for Pkg.dev
).
For more information, see
help?> @genpkg
to see how to create a new package with this template.help?> @upactions
to see how to update files in.github/workflows
at one click once your package is already created.
Please also read README.md
of the generated package for further instructions to complete the workflow setting.
- On push of master/main branch, test and documentation actions in
CI.yml
runs.- it runs test, where compatibility is checked through
CompatHelperLocal
. - it runs Documenter.
- it runs test, where compatibility is checked through
- On the push of
Project.toml
, there is an attempt to register to OkRegistry. If the register process success, TagBot and CompatHelper will be triggered. If there is a section in thechangelog.md
with head level 2 containing the version number inProject.toml
(e.g.,## v0.2.10
inchangelog.md
), the section will be part of the release message. - It is suggested to run
pkg> test
in local and fix[compat]
inProject.toml
to avoid excessive noise fromCompatHelper
pull requests.
Noted that both test locally and github actions do the followings:
- test and doctest
- check compatibility
It's no harm but you may remove either by deleting files in .github/workflows
or relative lines in test/runtests.jl
.
OkPkgTemplates
is compatible to PkgTemplates
of this commit: https://github.com/JuliaCI/PkgTemplates.jl/commit/0de5d855e050d93169f8661a13b3a53a8cb2b283 or v0.7.29
By default, either CompatHelperLocal
in test and CompatHelper
in .github/workflows is suggestive.
You may replace the @test true
line in @testset "CompatHelperLocal"
in runtests.jl by
@test CompatHelperLocal.@check()
to make the compatibility check mandatory.
!!! warning On @test CompatHelperLocal.@check()
in runtests.jl
In this way, test in .github/workflows/CI.yml will fail if you did not fix [compat]
properly before push into branch main
.
CompatHelper
or CompatHelperLocal
may give misleading/confusing suggestion for the [compat]
field of your project in Project.toml
WHEN there are packages having exact the same name but in different registry.
For example, both General and OkRegistry has DataFrameTools
. In this case, CompatHelper
may suggest the version number of that in the General Registry even if you only apply the DataFrameTools
of OkRegistry in a project.
So far, I simply add both the suggested version number ("1.0.8" for DataFrameTools
of General Registry) and the right version number ("0.5.2" for that in OkRegistry) to suppress the suggestion from CompatHelper
.
[compat]
DataFameTools= ["0.5.2", "1.0.8"]
Follow the the tips in Interpreting and resolving version conflicts and Fixing conflicts where:
- Make packages in your default julia environment as few as possible
- For a project of scientific report, use DrWatson or delete
[compat]
,CompatHelper
in CI andCompatHelperLocal
in test, sinceManuscript.toml
is sufficient for reproducibility of your project.
TagBot looks into your registry to get the registered versions and their corresponding git-tree-sha1
to know on which commit a version is registered.
If a tag is already on the commit, the corresponding version is skipped and TagBot goes to the next commit of registered version.
That is, if you tag a commit yourself, TagBot won't help you create the release of that commit. On the other hand, you can fix some TagBot error by manually adding the correct tag yourself. See below.
Seemingly, TagBot is designed to be interrupted in the process of scanning sequentially the existent tags until Error: TagBot experienced an unexpected internal failure
. In some cases, it experience this error at a much earlier stage that no further version can be tagged anymore.
For example, in the case I modify the TagBot.yml
for the attempt to include commit message as the release note by adding ${{ github.event.head_commit.message }}
in the changelog
template as the input for the TagBot action, the internal failure will occurred if the tagbot does not triggered and run successfully exactly at that commit. The error is as followings:
raise self.__createException(status, responseHeaders, output)
github.GithubException.GithubException: 403 {"message": "Resource not accessible by integration", "documentation_url": "https://docs.github.com/rest/reference/git#create-a-reference"}
And it is found that the problem always occur in the changelog generating stage for this example, because github.event.head_commit.message
is not accessible.
Generating changelog for version v0.2.3 (1281cdccbc33566fce239fbcd098797144020d98)
Warning: No registry pull request was found for this version
Error: TagBot experienced an unexpected internal failure
The Action that includes the commit message will always fail if it is triggered by other action.
This is because beyond that commit (starting a new machine), github.event.head_commit.message
does not exist anymore.
Once it experienced this kind of error in the history, further tagging will never be reached EVEN IF you fixed your TagBot.yml
. To deal with this problem, just manually tag on all commits that should be tagged correctly, and TagBot can go further from the last tag.
If the failure originates from templating changelog, run TagBotNoChangelog
in Github Actions might solve the problem.
Manually add tags correctly may solve many problems:
- Go to Github Actions of the repo, click TagBot to see at what version it stuck.
- Go to okatsn/OkRegistry to get the git-tree-sha1 of the version where it always failed.
- Convert
git-tree-sha1
to commit hash in bash:git log --pretty=raw | grep -B 1 <git-tree-sha1>
(no<
and>
). - In Git Graph, add tag at that commit hash.
- https://discourse.julialang.org/t/tagbot-unexpected-internal-error/74680/3
- https://discourse.julialang.org/t/tagbot-github-action-runs-successfully-but-a-new-release-does-not-show-up-on-github-releases/80770
- JuliaRegistries/TagBot#242
You cannot define the variable custom
for the changelog
template in your repository.
As referred in julia-tagbot#changelogs, for the changelog template
...
...
with:
token: ${{ secrets.GITHUB_TOKEN }}
changelog: |
This is release {{ version }} of {{ package }}.
{% if custom %}
Here are my release notes!
{{ custom }}
{% endif %}
where "The data available to you looks like this:"
{
"compare_url": "https://github.com/Owner/Repo/compare/previous_version...current_version (or null for first release)",
"custom": "your custom release notes",
"package": "YourPackage",
"previous_release": "v1.1.2 (or null for first release)",
// Go to julia-tagbot#changelogs to see all available variables
}
is not what you can explicitly set.
However, the variable custom
can be defined when you use PkgDev.tag("YourPackage", v"0.2.6"; release_notes="Trying release note")
, for example, as mentioned in TagBot's Readme.
As you dig into the source code of PkgDev
, you will find the variable release_notes
eventually goes into GitHub.create_pull_request(...)
as the :body
entry (see tag.jl).
That is, you can only have your final changelog templated with these existing variable, through for example {{ package }}of version {{ version }} after {{ previous_release }}
; noted that this is a Jinja templating (no evaluation $
sign of Github Actions - Expression).
That is, it is useless to solely set environment variable (e.g., env: custom: "My message..."
) in your YAML; you have to also add evaluation sign but in this way you can easily break the changelog generating process since the variable itself might contain something that violates the templating rules.
In conclusion, TagBot is Pull-Request based that you can only manipulate the changelog with information captured from pull-request or issues.
In docs/make.jl
, add pages for example:
pages=[
"Home" => "index.md",
"Examples" => "examples/examples.md",
"Exported Functions" => "functions.md",
"Models" =>
["Model 1" => "models/model1.md",
"Model 2" => "models/model2.md"],
"Reference" => "reference.md",
]
In which,
- the
index.md
is the home page, where@index
block makes a list of links to docstrings once generated by@autodoc
in any of your.md
files for pages.@autodocs
block automatically generates docstrings given- the
Modules
(e.g.,Modules = [Foo, Bar, Bar.Baz]
) - the
Pages
(e.g.,Pages = ["a.jl", "b.jl"]
)
- the
@contents
block generates table of contents.- Example
Pages = ["examples/examples.md"] Depth = 5
- Example
!!! note
- If error occurred in generating the page, try the followings in your local machine:
julia makedocs(root=joinpath(dirname(pathof(YourPackage)), "..", "docs"), sitename="TEMP")
- Use dirname(pathof(YourPackage))
to locate YourPackage/src
!
!!! warning for @autodocs
:
Noted that you cannot generate duplicate doc strings with @autodocs
, with later ones all ignored with warning messages. For example:
In index.md
, I have @autodocs
block:
@autodocs Modules = [YourPackage] Order = [:function, :type]
, which generates docstrings for ALL instances under YourPackage
.
After that, in "models/model2.md"
, I have @autodocs
block:
@autodocs Modules = [YourPackage] Order = [:function, :type] using FileTools, YourPackage Pages = filelist(r".+\.jl", joinpath(dirname(pathof(YourPackage)),"mymodels"); join=false) # which might be `Pages = [model2.jl, model2a.jl]` in the directory "mymodels", for example
, which is intended to generate docstrings for instances (functions, macros, structs...) defined in all .jl
files in the directory "mymodels".
However, since the docstrings for an instance cannot be @autodocs
twice, the corresponding @autodocs
block in the page "Model 2" => "models/model2.md"
will be empty.
For more information, see this.
- include this README in https://okatsn.github.io/OkPkgTemplates.jl/