Linters:

See these issues for more info:

• atc0005/go-ci#287
• atc0005/go-ci#288

TODO:

• Create dependabot/docker/go/Dockerfile file
• Specify the version of Go desired/required for new binaries
• Add GitHub Actions Workflow to validate this file
  • I can borrow from the atc0005/go-ci project for this

As with #3, I can borrow from earlier projects. The idea is to setup the same sort of build/test/lint workflow for local and remote/CI use.

Lines:

I'll need to keep it as one long line.

refs atc0005/send2teams#4

https://stackoverflow.com/questions/30005878/avoid-debugging-information-on-golang

I recommend usage of -ldflags="-s -w" which removes symbol table and debugging information.

As a bonus with Go 1.13 -trimpath can be used to reduce length of file paths stored in the file

The first I've been doing for a bit, but haven't tried the latter option yet.

The atc0005/bounce application does this, but it's intended to show/display as much useful information as possible. It's goal isn't in any way tied to security, whereas this application is probably better off defaulting to only displaying via web interface what is required for its operation. All other information should go to log files or be sent via one or more notifications (e.g., email, Teams).

That said, some sites may want an auto-generated index of available endpoints with a summary of each. This could be useful for initial implementation/testing and then later disabled before going into production use.

If this is eventually implemented, it should default to "off" and require that the user explicitly enable it.

Linting error from updating atc0005/go-teams-notify/v2 from v2.4.2 to v2.5.0 as part of GH-125:

config/config.go:542:17: SA1019: goteamsnotify.IsValidWebhookURL is deprecated: use API.ValidateWebhook() method instead. (staticcheck)
		if ok, err := goteamsnotify.IsValidWebhookURL(c.WebhookURL); !ok {
		              ^

Output from using Ctrl+c to stop a running instance:

  WARN[2702] bounce is shutting down, please wait ...
 ERROR[2702] gracefulShutdown: could not gracefully shutdown the server: context canceled
  INFO[2702] bounce successfully shutdown

The WARN and INFO messages are fine, but the ERROR is not an error, it is expected. The error code check currently applied by the gracefulShutdown goroutine (or its handling) should should that particular error.

While working on #21 a lot of new code has been written to handle notifications, thankfully with much of it consolidated together in one place. With some additional work this functionality could be abstracted further to create a new subpackage.

Potential name: notify

With (hopefully) a little more work, this new package can be worked into shape for use with another project I have in the queue (the "real" focus I'm building towards).

Instead of trying to tackle this as part of #21 I plan to work on it separately, likely as part of building the new project.

Idea: Email requests to a specified email address. This could be useful for recording submissions for a period of time.

This would (likely) be primarily used when this app runs as a daemon/service and tailing the output isn't easily done.

• https://godoc.org/github.com/atc0005/bounce

The package itself is documented well enough, but the top-level synopsis is not there.

As noted on atc0005/todo#3:

1. go mod tidy
   • correct as necessary
2. go mod vendor
3. git add vendor
4. git commit -m "go mod tidy, go mod vendor"

See also:

• atc0005/todo#6
• #34
• #35

While implementing #21, I've setup a NotifyTeams() and NotifyEmail() pair of methods to determine whether those notification types are enabled. Unfortunately they're tied directly to a Config type, so only the value created from that type can be used to call the methods. This either means I pass the Config value to each part of the code that needs it, or I find a way to track and expose the state of all notifiers (enabled/disabled) from relevant parts of the code that need it.

For now, the stats output related to Teams (#21) and Email notifications (#19) will be generated regardless of whether they're actually used.

See atc0005/todo#20 for details.

I need to add explicit config settings to disable default exclusions. The default exclusions are masking some issues that need to be examined closer.

refs atc0005/todo#29

See also #34 and atc0005/todo#4.

I've just finished incorporating all previously "queued" changes intended for a future upstream inclusion directly into the now standalone fork of the parent project.

Unless I missed something (I've checked many times now), all pending changes for upstream included in v1.3.1-0.20200419155834-55cca556e726 (vendored in this project) have now been included in the new v2.2.0 release.

I plan to do all further development as local changes to that fork until such time as upstream responds to my inquiries. All further development on my fork is planned as future tagged releases.

EDIT: v2.3.0 has now been released, updating to that version instead.

Same problem as noted on atc0005/check-cert#89.

see atc0005/todo#9

Linting issues exposed by the changes in atc0005/go-ci#287 and atc0005/go-ci#288:

config/config.go:251:13: fieldalignment: struct with 112 pointer bytes could be 72 (govet)
type Config struct {
            ^
routes/routes.go:18:12: fieldalignment: struct with 80 pointer bytes could be 64 (govet)
type Route struct {
           ^
cmd/bounce/decode-json-body.go:24:23: fieldalignment: struct with 16 pointer bytes could be 8 (govet)
type malformedRequest struct {
                      ^
cmd/bounce/notify.go:16:19: fieldalignment: struct with 32 pointer bytes could be 24 (govet)
type NotifyResult struct {
                  ^
cmd/bounce/notify.go:34:18: fieldalignment: struct with 32 pointer bytes could be 24 (govet)
type NotifyQueue struct {
                 ^

Summary

The Go 1.16 series is going EOL in ~ 2 months, so will need to update the Go version used for this project to reflect this.

TODO

• Update README "requirements" to generically indicate "recent Go release" since this project doesn't (at present) have a fixed version requirement.
• Update go.mod file to reflect Go 1.17
• Update Dependabot (dependabot.yml) configuration to ignore Go versions greater than or less than 1.17.x
• Update "canary" Dockerfile to reflect Go 1.17.6, the latest in the 1.17.x series
• Run go mod tidy && go mod vendor

A minor refactoring to mirror what I've done with the atc0005/dnsc project:

• small function to handle applying logging settings within the config subpackage instead of in main
• small function to handle flags instead of including directly within the NewConfig() factory function
• move hard-coded flag help text to constants block

I've already worked the first two into a prototype branch I'm working with, but I do not plan to merge that branch as-is to master; I'll need to port relevant portions of changes as separate Pull Requests, these changes included.

Options I'm considering (in some capacity):

• https://github.com/sirupsen/logrus
  • using in atc0005/elbow project
• https://github.com/rs/zerolog
  • already using in https://github.com/atc0005/check-mail
• https://github.com/apex/log
  • strong arguments for being a successor to logrus
  • https://github.com/apex/log/tree/master/_examples
  • https://medium.com/@tjholowaychuk/apex-log-e8d9627f4a9a

EDIT

Some additional TODO items for the v0.3.0 release.

• Update README
  • New flags
  • Notes/details regarding less obvious details (e.g., what is different about a cli handler vs a text handle in regards to output format?)
  • Update example output to reflect this work vs the hard-coded "DEBUG" prefixes used in the prior releases
• Update CHANGELOG
• Prune dead code left in from development of this feature branch (assuming I left anything behind)
• Update GoDoc coverage to reflect new flags / output

When I implemented use of the local top-level vendor directory, the implementation was incomplete. Later, while performing the same work in atc0005/elbow#232 I spotted the CI jobs downloading dependencies instead of using the local vendor directory.

Further digging found that the cause was mostly related to not supplying -mod=vendor to go test and the real culprit (at least for that work): go list.

This issue is to track the work to add the missing references to the build process for this project.

refs:

• atc0005/elbow#232
• #33

Swap out the GitHub Actions used for CI work with custom Docker containers from the atc0005/go-ci project.

refs:

• atc0005/todo#22
• atc0005/dnsc#76
• atc0005/elbow#275, atc0005/elbow#276

Currently the installation section only lists building from source. Split into "using release binaries" (or something similar) and "from source".

I can follow what I did with the READMEs from these projects:

• https://github.com/atc0005/check-vmware#installation
• https://github.com/atc0005/check-mail#installation

I partially implemented the intended http.Server settings in 798e9b1 by setting a default value for ReadHeaderTimeout, but missed the other two:

• ReadTimeout
• WriteTimeout

Line 267 should move below the block that starts at line 269. Spotted this while reviewing the code for use elsewhere.

Replace hard-coded version with LTS alias for "latest version", supported by the actions/setup-node update to v2.2.0.

While working on atc0005/brick#4, I was again reminded of how I ended up implementing notification delays:

// Delay between message submission attempts; this will *always*
// delay, regardless of whether the attempt is the first one or not

This comment above this portion of the select block:

prompted me to quickly stub in this FIXME entry that I later removed for inclusion into this GH issue:

// FIXME: Need to update wording of constant, docs, etc to emphasize that
// the delay is enforced regardless of whether the notification attempt is
// the first, or a retry

Since I opted to apply the delay regardless of the delivery attempt, I should probably "unhook" the "retries" bit from the associated variables/constants associated with the delay. For example, it is in the function parameter list:

and is also used in the exported SendMessage() function from the atc0005/send2teams project:

As of this writing I've not reviewed that project yet to see if it has similar artifacts from the early design/implementation work, but it's likely.

• README.md
  • contains output examples from earlier builds; need to update to include most recent output
• LICENSE file
• CHANGELOG.md
• doc.go

see atc0005/todo#6

Use already existing files from earlier projects and adapt for this one.

Both of these functions are nearly identical with the exception of the parameter list (just one?) and the anonymous function called within to perform the actual notification.

One route I tinkered with was creating a common settings "bundle" within the notifications manager and passing that bundle into two instances of a common notifier function. Another approach is to have only a single instance, but apply checks within to determine if notification type should be attempted.

Regardless of the final outcome, the initial implementation on the bench for #21 duplicates nearly 1:1 the content of teamsNotifier and emailNotifier functions and this is a strong code smell.

https://pkg.go.dev/badge/

see atc0005/todo#8

Having the JSON output colorized live as requests are coming in would prove helpful for visually parsing the data. I wouldn't want to trade reliability/compatibility in order to have the colorized output however.

If we add this support, perhaps only activate it upon explicitly enabling the setting (e.g., via a flag)? This way if there are issues parsing the JSON for colorized output, the user could remove the setting and re-run the application.

Same here as with atc0005/check-mail#94.

For our purposes a hard-limit of say one message every 4-5 seconds may do the trick.

References

• #21

• https://docs.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using

see atc0005/todo#10

Related to #19 and #21.

Currently the design has main() start a collection of persistent goroutines that:

• receive incoming client request details and generates notifications
  • this goroutine in turn starts a persistent goroutine for each notification type which is in charge of schedule each notification
• monitor queue levels
• monitor and report notification stats
  • total
  • pending
  • success
  • failure
• monitor for SIGINT and trigger context cancellation and graceful shutdown of http server

Currently both (intended) nested bullets display as a code-block instead of sub-bullets. The fix will be to incorporate their details into the parent item or split into two medium-length items instead of one parent and poorly displayed sub-items.

While useful, the current status badges link out to the SVG used to provide the current status, not the workflow results themselves.

Update all badges to use a syntax similar to 'GoDoc' or 'Latest Release' so that the badge displays, but also functions as a "GoTo" button as well.

TODO:

• add new features (mirror GoDocs coverage, CHANGELOG)
• remove feature TODO for leveled logging (done)

Likely some other items that I've missed.

What

Instead of specifying the options like this:

golangci-lint run \
    -E goimports \
    -E gosec \
    -E stylecheck \
    -E goconst \
    -E depguard \
    -E prealloc \
    -E misspell \
    -E maligned \
    -E dupl \
    -E unconvert \
    -E golint \
    -E gocritic

Specify them something like this (copy/paste from another repo, not customized for our linters):

linters:
  enable:
    - dogsled
    - dupl
    - gocognit
    - goconst
    - gocritic
    - gocyclo
    - gofmt
    - golint
    - gosec
    - maligned
    - nakedret
    - prealloc
    - scopelint
    - unconvert
    - unparam
    - whitespace

linters-settings:
  funlen:
    lines: 60
    statements: 40

  gocognit:
    # minimal code complexity to report, 30 by default (but we recommend 10-20)
    min-complexity: 10

  gocyclo:
    # minimal code complexity to report, 30 by default (but we recommend 10-20)
    min-complexity: 15

  golint:
    # minimal confidence for issues, default is 0.8
    min-confidence: 0.3

  maligned:
    # print struct with more effective memory layout or not, false by default
    suggest-new: true

  nakedret:
    # make an issue if func has more lines of code than this setting and it has naked returns; default is 30
    max-func-lines: 2

  unparam:
    # Inspect exported functions, default is false. Set to true if no external program/library imports your code.
    # XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
    # if it's called for subdir of a project it can't find external interfaces. All text editor integrations
    # with golangci-lint call it on a directory with the changed file.
    check-exported: true

  unused:
    # treat code as a program (not a library) and report unused exported identifiers; default is false.
    # XXX: if you enable this setting, unused will report a lot of false-positives in text editors:
    # if it's called for subdir of a project it can't find funcs usages. All text editor integrations
    # with golangci-lint call it on a directory with the changed file.
    check-exported: false

  whitespace:
    # Enforces newlines (or comments) after every multi-line if statement
    multi-if: true
    # Enforces newlines (or comments) after every multi-line function signature
    multi-func: true

issues:
  # Not using default exclusions because we want to require comments on public
  # functions and types.
  exclude-use-default: false

# options for analysis running
run:
  # include test files or not, default is true
  tests: false

  # by default isn't set. If set we pass it to "go list -mod={option}". From "go help modules":
  # If invoked with -mod=readonly, the go command is disallowed from the implicit
  # automatic updating of go.mod described above. Instead, it fails when any changes
  # to go.mod are needed. This setting is most useful to check that go.mod does
  # not need updates, such as in a continuous integration and testing system.
  # If invoked with -mod=vendor, the go command assumes that the vendor
  # directory holds the correct copies of dependencies and ignores
  # the dependency descriptions in go.mod.
  modules-download-mode: vendor

service:
  # use the fixed version to not introduce new linters unexpectedly
  golangci-lint-version: 1.20.x

Why

By having an external config file (that is picked up automatically), this can be used in the Makefile, in manual golangci-lint runs and (I believe) via Visual Studio Code when golangci-lint is set as the default linter. It would be good to be able to set the linting options in one place and have all applicable use cases covered.

References

• https://github.com/dasrick/go-teams-notify/blob/master/.golangci.yaml
• https://github.com/golangci/golangci-lint#config-file

The initial implementation could send each endpoint request over to a specified Microsoft Teams channel, but a later refinement might improve on this by batching the requests for bulk submission, either immediately when hitting a specified queue threshold or at a schedule frequency (if any are in the queue).

The existing atc0005/send2teams project code could serve as a starting point for adding this feature.

As of v0.5.0 of the atc0005/go-ci project, images are available from both ghcr.io and index.docker.io.

Switch to ghcr.io to:

• speed up CI runs
• reduce bandwidth demands on Docker Hub
• increase availability of images

Right now the current functionality is pretty limited. After reading over the "How to Parse a JSON Request Body in Go" article by Alex Edwards (again), I am reminded that there are a number of problem cases that the current JSON error handling code doesn't guard against.

One thing we're doing differently is displaying the raw request body, whereas the article's use case is specifically an endpoint receiving a JSON payload. We're doing both, so we may need to adapt both our code and the provided helper function to fit.

refs https://www.alexedwards.net/blog/how-to-properly-parse-a-json-request-body