taskmedia / nuscrape Goto Github PK

It seems that not all logs are converted in JSON as defined here:

Currently this only are logs from gin-gonic

🔽 low priority

sometimes the spielplan gets approved which could be added.
also sometimes a additional note will be at the last column.

spielplan approved should be one after the meeting report with id 9 (meeting report does not use ids).
the note will follow after spielplan approved and has id 10.

<td nowrap="nowrap">
          <img alt="Spielbericht genehmigt" border="0" title="Spielbericht genehmigt" src="/WebObjects/nuLiga.woa/Frameworks/nuLigaFramework.framework/WebServerResources/img/icons/check.gif" width="13" height="11">
        </td>

<td nowrap="nowrap">
          <img border="0" alt="HT Halberstadt hat telefonisch abgesagt. SL MB" title="HT Halberstadt hat telefonisch abgesagt. SL MB" src="/WebObjects/nuLiga.woa/Frameworks/nuLigaFramework.framework/WebServerResources/img/icons/note_14x14.gif" width="14" height="14">&nbsp; 
          <span></span>
        </td>

example at: courtInfo?federation=HVSA&location=19274

In the func parseGermanTime a static timezone is used. This causes wrong display time in calendar.

increase tests to get better code coverage

Add a readme which describes the project

Currently only the Championship is a string element - all others are a own type:

This would be great for e.g. GetDescription:

Here only the Championship will be returned as abbreviation.

The URL https://bhv-handball.liga.nu/cgi-bin/WebObjects/nuLigaHBDE.woa/wa/groupPage?championship=AV+2021%2F22&displayDetail=meetings&displayTyp=gesamt&group=281103 is mentioned for many tests - this should be centralized to be changed better.

Maybe it is possible to add a file with static configurations.

• AgeCategory: Männer
• class: Bezirksklasse
• relay: Nord-West

next to http a endpoint for msg queue should be added.

useful links:

• https://pkg.go.dev/github.com/gin-gonic/gin#section-readme
• https://github.com/marcel-dempers/docker-development-youtube-series/tree/master/messaging/rabbitmq

remove empty values from JSON object.

e.g.

{
  goal: {
    home: -1,
    guest: -1
  },
  referee: null
},

add result annotations as const or struct as e.g. relay or class.

This will be parsed here:

The values are as described here:

Legende
v ... Spiel wurde in/auf genannte/n Halle/Termin verlegt
u ... Spiel wird in/auf noch unbekannte/n Halle/Termin verlegt
h ... Halle wurde geändert
t ... Heimrecht wurde getauscht
NH ... Heimmannschaft nicht angegreten
NG ... Gastmannschaft nicht angetreten
N2 ... Heim- und Gastmannschaft nicht angetreten
WH ... Wertung gegen Heimmannschaft
WG ... Wertung gegen Gastmannschaft
W2 ... Wertung gegen Heim- und Gastmannschaft
U ... Begegnung wurde umgewertet
ZH ... Heimmannschaft zurückgezogen
ZG ... Gastmannschaft zurückgezogen

example in gsp:

{"season": "2021_22", "championship": "AV", "group": "273516"}

example from sourcecode:

<td nowrap="nowrap" alt="Wertung gegen Gastmannschaft" class="center" title="Wertung gegen Gastmannschaft">
          
           
	          
          	WG&nbsp;
          
        </td>

see: golangci/golangci-lint-action

Add a pipeline to build the app and if possible add it as asset to releases

If possible add swagger file or equal to describe the API

The JSON response example does not have the latest implementation displayed.

The match id will always be zero in the JSON response instead of e.g. 27031301.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update docker/build-push-action action to v3.1.1
• Update module github.com/sirupsen/logrus to v1.9.0
• Update module go to 1.19
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Add a full parsing test (old season) which checks if the generated Gesamtspielplan struct will be correctly parsed and created.

the added labeler configuration from #38 did not work as expected in #39.

• if a README.md will be added / modified it should be added (also subdirectories)
• k8s/ changes were not tracked

Add a Dockerfile to containerize the application (automatic upload at build)

depends on #10

The following table was present at 2021_22/SW/273332:

red: no match id was present
orange: no location id was present

this occured in following log messages:

{
  "error":"strconv.Atoi: parsing \"\": invalid syntax",
  "level":"warning",
  "locationId":"",
  "msg":"can not parse location ID",
  "time":"2021-11-15T21:23:30+01:00"
}

The error is no problem because no location was given.

But to avoid invalid matches to be added to the DB it is required to check if the ID of the match is 0.
In this case the match entry should be dropped.

run go scripts as go routine (async) - this may help performance on multi requests?