Add additional metadata about azure-devops-marketplace HOT 17 OPEN

normally it would be enough to have it on package level. but if it can change on different versions, it needs to be on the release.

i would prefer to have it on package level to reduce data size.

from azure-devops-marketplace.

There were a few illegal characters in the identifiers, I solved that with:

$identifier.ToLowerInvariant() -replace "[^a-z0-9_.-]", "_"

for now.

from azure-devops-marketplace.

I think I can normalize most tasks to have the metadata at the package level.

Some tasks however, have been forked or transferred to a different publisher, in which case the metadata would have changed at some point. The flutter task is an example of that. It started out from:

• https://marketplace.visualstudio.com/items?itemName=aloisdeniel.flutter

Then got forked and taken over by:

• https://marketplace.visualstudio.com/items?itemName=Hey24sheep.flutter

This means that while the taskname sometimes stays the same between these extensions, the taskid and longtaskname will be different between them. So the taskid and longtaskname would be straightforward and should be able to use the latest version metadata, the shortname can be problematic.

@viceice Can the SourceUrl or the Homepage be set at the package level and the only where it need to differ at the release level? Or, since this would be for older versions in almost all cases, simply follow the latest version?

eg:

    "flutter": {
      "sourceUrl": "https://github.com/Hey24Sheep/Flutter",
      "homepage": "https://marketplace.visualstudio.com/items?itemName=Hey24Sheep.Flutter"
      releases: [
         {
             "version":  "0.3.10"
         },
         ...
         ...
         {
             "version:"  "0.2.39"
             "sourceUrl": "https://github.com/aloisdeniel/Flutter"
         },
    }

from azure-devops-marketplace.

Here's a generated sample json file containing:

• releaseTimeStamp
• soucreUrl
• homepage
• isStable
• isDeprecated
• downloadUrl

This amounts to +- 50MB of json.

Without the downloadUrl it's about 20MB.

https://github.com/renovatebot/azure-devops-marketplace/tree/features/preview-and-deprecation

from azure-devops-marketplace.

i think we don't need the download url

from azure-devops-marketplace.

what about creating a JSON file per taskid and a meta file with mapping only. then the renovate datasources first fetches the metadata file and the taskid files on demand via GitHub CDN.

that way we don't need to update renovate to get newer data.

from azure-devops-marketplace.

Sure a possibility. We now have the data and can slice and dice it in any way you want :).

from azure-devops-marketplace.

@JamieMagee @rarkins WDYT?

from azure-devops-marketplace.

An even better option would be to switch over top an API based approach, the TFS/Azure DevOps API has all the information needed for installed tasks:

• renovatebot/renovate#19725

On top of the data the server already knows, the marketplace data can help people find newer versions that weren't installed into their server.

For the built-in tasks it usually doesn't make much sense to report availability of versions the server doesn't have, since they only get updated through server upgrades or Microsoft's provisioning process and there isn't much an admin or user can do unless they go the "I'll install them myself route". I have a another project to give admins of Azure DevOps Server the ability to overwrite tasks:

• https://github.com/jessehouwing/azure-pipelines-tasks-zips

from azure-devops-marketplace.

@viceice I like that idea. We use GitHub pages as a JSON API.

@jessehouwing we can still implement the API-based datasource, and prefer it whenever Renovate is running in an Azure DevOps context. For the case where code is hosted in GitHub, but builds are run on Azure DevOps (this is the case for almost all the Microsoft org on GitHub for compliance reasons) we can use the cached data from this repo.

from azure-devops-marketplace.

I was thinking we could add additional config to make the API work even in a GitHub context. The Docs for configuring renovateme on Azure DevOps also show how to add GitHub configuration to generate release notes, this would be similar but the other way round.

Eg

--host-rules="[{\"domainName\":\"dev.rzure.com/myorg\",\"token\":\"$(AZURE_DEVOPS_TOKEN)\"}]"

from azure-devops-marketplace.

Yeah, good point. That's another good way to provide an API token.

from azure-devops-marketplace.

So the idea is to generate a json file per identifier and stick the ReleaseRef object in that file, right? That should be relatively simple to do from where we are now.

from azure-devops-marketplace.

yes please. than update the renovate datasource to fetch from github cdn instead of buildin big fat json.

from azure-devops-marketplace.

I've created a feature issue on renovate side for required datasource changes.

• renovatebot/renovate#20019

from azure-devops-marketplace.

@viceice check out https://github.com/renovatebot/azure-devops-marketplace/tree/features/preview-and-deprecation/_data

from azure-devops-marketplace.

@JamieMagee any news on the front-end implementation of this?

from azure-devops-marketplace.