twisterrob / twisterrob-env Goto Github PK

My environment setup (to be added) and other random scripts.

Home Page: https://github.com/TWiStErRob/TWiStErRob-env#contents

License: The Unlicense

Dockerfile 0.08% sed 0.61% Shell 0.73% Batchfile 2.12% Python 0.57% Perl 1.45% Kotlin 74.22% HTML 5.17% jq 1.38% JavaScript 0.63% PowerShell 13.04%

git github-api intellj-idea maildir notion-api scripts svn2git

twisterrob-env's Introduction

My Environment setup & scripts

I have a complex environment setup on Windows, this repository shall contain things that make the environment easily reproducible.

caches files and folders that are emphemeral, and if deleted, they'll regenerate.
config configuration files relocated into a central place.
projects workspaces for different projects I'm working on.
repos master copies of files, e.g. SVN server, Maven repository, Artifactory, etc.
scripts see below.
secrets private keys.
tools programs, SDK, etc. that do things.

Scripts

Special
- svn2git-migration A script to migrate from an SVN monorepo into git repository(s).
- local-intellij How to start Android Studio (IntelliJ IDEA) without a trace in a specific directory.
- pst-maildir-imap A process to convert a .pst file to Maildir format and then sync with an IMAP server.
- bitrise-mermaid Convert Bitrise configuration to Mermaid diagram.
Git
- github-away How to set up GitHub and git on a foreign machine.
- merged-branches Find squashed branches from git log.
GitHub
- repository-convention Self-controlled defaults for GitHub user/orgs.
- manage-draft-issues Create/update draft issues in a GitHub project.
- user-contribs Interactive GitHub User Contribution Stats for multiple repositories.
Notion
- import-data Import data from JSON and CSV into Notion Databases.
- move-to-property Move a page section to a property of the page.
- database-filter-page-content Create a "has content" column based on page content.
Google
- gmail-play-reviews Load reviews from emails.

License

Unlicense, see .github/LICENSE.

twisterrob-env's People

Contributors

Stargazers

Watchers

twisterrob-env's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

dockerfile

scripts/special/pst-maildir-imap/Dockerfile

ubuntu 24.04

github-actions

.github/workflows/ci.yml

TWiStErRob/github-workflows v3@1e06403c5d561d70e40109c2701ea3092ca6ded7

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

actions/checkout v4

addnab/docker-run-action v3

actions/upload-artifact v4

actions/checkout v4

addnab/docker-run-action v3

actions/upload-artifact v4

actions/checkout v4

gradle/actions v3

actions/checkout v4

actions/upload-artifact v4

github/codeql-action v3

actions/checkout v4

actions/upload-artifact v4

github/codeql-action v3

actions/checkout v4

microsoft/psscriptanalyzer-action v1.1

actions/upload-artifact v4

github/codeql-action v3

gradle

.gradle

config/gradle/gradle.properties

config/gradle/init.gradle.kts

config/gradle/init.d/includes/agp_disable-profile-protos.init.gradle.kts

config/gradle/init.d/includes/gradle_auto-warning.init.gradle.kts

config/gradle/init.d/includes/gradle_buildCache-local-expiry.init.gradle.kts

config/gradle/init.d/includes/gradle_caches-expiry.80.init.gradle.kts

config/gradle/init.d/includes/gradle_caches-expiry.88.init.gradle.kts

config/gradle/init.d/includes/gradle_lifecycle-logging.init.gradle.kts

config/gradle/init.d/test/gradle.properties

config/gradle/init.d/test/settings.gradle.kts

repos/maven/configure.gradle

org.apache.maven.wagon:wagon-ftp 3.5.3

gradle-wrapper

config/gradle/init.d/test/gradle/wrapper/gradle-wrapper.properties

gradle 8.9-rc-1

kotlin-script

scripts/git/merged-branches/find-merged-branches.main.kts

org.jetbrains.kotlinx:kotlinx-coroutines-core 1.8.1

scripts/github/project-manage-draft-issues/manage-draft-issues.main.kts

io.ktor:ktor-client-java-jvm 2.3.12

io.ktor:ktor-client-content-negotiation-jvm 2.3.12

io.ktor:ktor-client-logging-jvm 2.3.12

io.ktor:ktor-client-auth-jvm 2.3.12

io.ktor:ktor-serialization-jackson-jvm 2.3.12

com.jayway.jsonpath:json-path 2.9.0

scripts/github/repository-convention/validate.main.kts

javax.json:javax.json-api 1.1.4

org.glassfish:javax.json 1.1.4

org.jetbrains.kotlinx:kotlinx-html-jvm 0.11.0

io.ktor:ktor-client-java-jvm 2.3.12

io.ktor:ktor-client-content-negotiation-jvm 2.3.12

io.ktor:ktor-client-logging-jvm 2.3.12

io.ktor:ktor-client-auth-jvm 2.3.12

io.ktor:ktor-serialization-jackson-jvm 2.3.12

tech.tablesaw:tablesaw-core 0.43.1

scripts/github/user-contribs/get.main.kts

org.jetbrains.kotlinx:kotlinx-html-jvm 0.11.0

io.ktor:ktor-client-java-jvm 2.3.12

io.ktor:ktor-client-content-negotiation-jvm 2.3.12

io.ktor:ktor-serialization-jackson-jvm 2.3.12

scripts/github/user-contribs/summarize.main.kts

com.fasterxml.jackson.core:jackson-databind 2.17.1

com.fasterxml.jackson.module:jackson-module-kotlin 2.17.1

com.fasterxml.jackson.dataformat:jackson-dataformat-yaml 2.17.1

org.jetbrains.kotlinx:kotlinx-html-jvm 0.11.0

scripts/google/gmail-play-reviews/reviews.main.kts

com.google.api-client:google-api-client 2.6.0

com.google.oauth-client:google-oauth-client-jetty 1.36.0

com.google.apis:google-api-services-gmail v1-rev20240520-2.0.0

scripts/notion/database-filter-page-content/filter-content.main.kts

com.github.seratch:notion-sdk-jvm-core 1.11.1

com.github.seratch:notion-sdk-jvm-okhttp4 1.11.1

scripts/notion/import-data/notion-import-csv.main.kts

com.opencsv:opencsv 5.9

com.github.seratch:notion-sdk-jvm-core 1.11.1

com.github.seratch:notion-sdk-jvm-okhttp4 1.11.1

scripts/notion/import-data/notion-import-dev-summit.main.kts

com.fasterxml.jackson.module:jackson-module-kotlin 2.17.1

com.fasterxml.jackson.datatype:jackson-datatype-jsr310 2.17.1

com.github.seratch:notion-sdk-jvm-core 1.11.1

com.github.seratch:notion-sdk-jvm-okhttp4 1.11.1

scripts/notion/import-data/notion-import-droidcon.main.kts

com.fasterxml.jackson.module:jackson-module-kotlin 2.17.1

com.fasterxml.jackson.datatype:jackson-datatype-jsr310 2.17.1

com.github.seratch:notion-sdk-jvm-core 1.11.1

com.github.seratch:notion-sdk-jvm-okhttp4 1.11.1

scripts/notion/import-data/siliconmilkroundabout/companies-attending.main.kts

com.fasterxml.jackson.module:jackson-module-kotlin 2.17.1

scripts/notion/page-section-move-to-property/notion-move-to-property.main.kts

com.github.seratch:notion-sdk-jvm-core 1.11.1

com.github.seratch:notion-sdk-jvm-okhttp4 1.11.1

scripts/special/bitrise-mermaid/bitrise-mermaid.main.kts

org.snakeyaml:snakeyaml-engine 2.7

scripts/special/lastpass2google-migration/pass.main.kts

com.fasterxml.jackson.dataformat:jackson-dataformat-csv 2.17.1

org.skyscreamer:jsonassert 1.5.2

regex

.github/workflows/ci.yml

detekt/detekt 1.23.6

.github/workflows/ci.yml

koalaman/shellcheck 0.10.0

.github/workflows/ci.yml

psastras/sarif-rs 0.4.2

.github/workflows/ci.yml

psastras/sarif-rs 0.4.2

Check this box to trigger a request for Renovate to run again on this repository

Split SVN monorepo

For how, see https://github.com/TWiStErRob/TWiStErRob-env/tree/master/svn2git-migration

Steps:

Create issue in target repo
Link it here
Clean up local copy
Use svn2git
Replace svn:externals with a submodule (example)
Renormalize line endings with .gitattributes (example)
Review .gitignore files
Optional: Build CI
Delete from SVN
Replace p:\projects\workspace\x (.svn) with p:\project\workspace\repo.name.x (.git)
Create tag svn for the last SVN commit in git, and create a pre-release on GitHub: "SVN to GIT migration" and description: "This is the point of the last commit in SVN. After this commits went to GitHub git."
recreate/tag existing releases on GH
link new releases from website (see guide)
tick the box in split epic

Tasks

Reusable Android instrumentation GitHub Action

e.g. TWiStErRob/net.twisterrob.libraries#12

Rebuild SVN repository

Remove JARs and other binaries
Filter removed files/folders like they didn't exist

The target is to reduce 133MB to at least 50MB

Script to clean caches

Split Gradle init to init.d files

Try Snyk Security

https://github.com/snyk/actions/blob/master/gradle/README.md

Track Windows hidden system readonly attributes

Sadly this is not possible out of the box: https://stackoverflow.com/q/8715043/253468

But a post-checkout hook could be written to set the attributes based on some rules:

git tracked descript.ion -> hidden
git tracked desktop.ini and folder.ico -> hidden system
git tracked desktop.ini's parent folder -> readonly
.gradle in root folder -> hidden system readonly

Review Antivirus settings

Automatic backup system for code

Scripts for clearing:

clear caches (#23)
clear temporary files from config
clean all projects

Scripts for rebuilding:

checkout svn from repos
checkout a list of remote repositories from various VCSs
create env vars from folder structure
download and extract some dists (gradle, maven, idea, eclipse, ...)

Scripts for backup

backup folder structure, including desktop.ini, descript.ion, folder.ico, and scripts
schedule frequent backups

All backups should be:

idempotent (only backup changed data)
self-contained

Try Gradle dependency submission

https://github.com/mikepenz/gradle-dependency-submission

https://github.blog/changelog/2023-02-08-dependency-submission-suggestions-on-gradle-maven-scala-and-mill-repositories/

Version control IntelliJ config

Collect all the places first into one.
color
code style
settings

Unit tests ps1 scripts

https://devblogs.microsoft.com/scripting/unit-testing-powershell-code-with-pester/

Fix IntelliJ theme

https://youtrack.jetbrains.com/issue/IDEA-300990/Light-theme-and-dark-color-scheme-doesnt-work-together-it-used-to

Clean up renovatebot configurations

Follow up on renovatebot/renovate#14913

Use centralized config from https://github.com/TWiStErRob/renovate-config

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
  	"github>TWiStErRob/renovate-config"
  ],

Research GitHub runner problem matchers

https://fusectore.dev/2021/11/19/github-action-problem-matchers.html

Undocumented: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions
But exists here: https://github.com/actions/toolkit/blob/main/docs/commands.md#problem-matchers
Example: https://github.com/search?q=repo%3Aandroid-actions%2Fsetup-android+matchers.json&type=code
Blog: https://michaelheap.com/getting-started-problem-matchers/
Actual code:
https://github.com/actions/runner/blob/v2.304.0/src/Runner.Worker/Handlers/OutputManager.cs#L93
https://github.com/actions/runner/blob/v2.304.0/src/Runner.Worker/IssueMatcher.cs#L62

Document "usual" repo setup

What do repos have in common, document conventions.

Looks through all repository settings
Query what's possible in GraphQL
Ensure #15 is applied to all repos (see .yml files and look for these lines)
#16
Manually apply diverging settings from #11

Grok

http://algopadawan.blogspot.hu/2012/07/installing-opengrok-on-windows.html