macgregor / alexandria Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 0.0 2.39 MB

Integrate markdown docs with antiquated html document hosting platforms using standard Java build tools.

License: GNU General Public License v3.0

Java 98.87% Shell 1.13%

alexandria ci ci-cd cli jive jive-sdk markdown markdown-to-html maven-plugin

alexandria's People

Contributors

Stargazers

Watchers

alexandria's Issues

Add default extraProps ala defaultTags

Add defaultExtraProps to Config that populates those fields on Metadata by default. For example jiveParentUri: "https://jive.com/groups/group-for-my-docs" which currently needs to be manually added to new documents that are indexed.

fix the relative/absolute link madness

Deploy javadocs with Travis

https://benlimmer.com/2013/12/26/automatically-publish-javadoc-to-gh-pages-with-travis-ci/

add default extra properties to remote config

tidy up javadoc build

Javadocs are built too many times during the process which slows down the build
Javadoc maven plugin is not properly linking to external classes, including submodules which produces warnings (might have something to do with the classpath being used during

Nice to have: aggregate javadocs into a single site.

try to enforce maven version for plugin

requires at least 3.5.2 or you get a nast error

trying to create/update an empty file returns a 400 from jive

They dont like empty content apparently. Could just add a placeholder string if the file is empty.

Jive throwing 401 sometimes doing search before create

BatchProcessException{message=Alexandria batch error., exceptions=[HttpException{
   message="HttpException: GET https://redacted/api/core/v3/contents?fields=id%2CcontentID%2Ctags%2Cupdated%2Cpublished%2CparentPlace%2Csubject%2Cresources%2Ccontent%2Cvia%2Cparent&filter=tag%2842f4477d-fa30-4729-a721-809fb42b3d1c%29&count=25&startIndex=0 - 401 (excepted one of [20X])",
   rootCauseMessage="HttpException: GET https://redacted/api/core/v3/contents?fields=id%2CcontentID%2Ctags%2Cupdated%2Cpublished%2CparentPlace%2Csubject%2Cresources%2Ccontent%2Cvia%2Cparent&filter=tag%2842f4477d-fa30-4729-a721-809fb42b3d1c%29&count=25&startIndex=0 - 401 (excepted one of [20X])",
   metadata=
{
  "sourcePath" : "redacted/docs/result_message.md",
  "title" : "result_message.md",
  "convertedChecksum" : 3321649118,
  "extraProps" : {
    "jiveTrackingTag" : "42f4477d-fa30-4729-a721-809fb42b3d1c"
  }
}
]}

[ERROR] HttpException: GET https://mojo.redhat.com/api/core/v3/contents?fields=id%2CcontentID%2Ctags%2Cupdated%2Cpublished%2CparentPlace%2Csubject%2Cresources%2Ccontent%2Cvia%2Cparent&filter=tag%2842f4477d-fa30-4729-a721-809fb42b3d1c%29&count=25&startIndex=0 - 401 (excepted one of [20X])
com.github.macgregor.alexandria.exceptions.HttpException: GET https://redacted/api/core/v3/contents?fields=id%2CcontentID%2Ctags%2Cupdated%2Cpublished%2CparentPlace%2Csubject%2Cresources%2Ccontent%2Cvia%2Cparent&filter=tag%2842f4477d-fa30-4729-a721-809fb42b3d1c%29&count=25&startIndex=0 - 401 (excepted one of [20X])
    at com.github.macgregor.alexandria.exceptions.HttpException$Builder.build (HttpException.java:109)
    at com.github.macgregor.alexandria.remotes.RemoteDocument.doRequest (RemoteDocument.java:328)
    at com.github.macgregor.alexandria.remotes.RemoteDocument$RemoteDocumentIterator.nextPage (RemoteDocument.java:478)
    at com.github.macgregor.alexandria.remotes.RemoteDocument$RemoteDocumentIterator.hasNext (RemoteDocument.java:438)
    at com.github.macgregor.alexandria.remotes.RemoteDocument$RemoteDocumentPage.first (RemoteDocument.java:368)
    at com.github.macgregor.alexandria.remotes.jive.JiveRemote.findDocument (JiveRemote.java:300)
    at com.github.macgregor.alexandria.remotes.jive.JiveRemote.create (JiveRemote.java:167)
    at com.github.macgregor.alexandria.AlexandriaSync.lambda$syncWithRemote$1 (AlexandriaSync.java:90)
    at com.github.macgregor.alexandria.BatchProcess.execute (BatchProcess.java:97)
    at com.github.macgregor.alexandria.AlexandriaSync.syncWithRemote (AlexandriaSync.java:77)
    at com.github.macgregor.alexandria.Alexandria.syncWithRemote (Alexandria.java:93)
    at com.github.macgregor.alexandria.SyncMojo.execute (SyncMojo.java:19)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Jive rest api suuuucks. Not sure why this happened, as it cleared up eventually on its own. Possibly a transient error server side, in which case this error is expected, perhaps retries might help if it is a very short lived problem. It could also be a quirk in Jive when a newly created document is in a weird state and the search throws a 401 for some reason when it should be a better error code. I've seen similar errors that I built safeguards for, maybe this is another one.

parallelize rest calls

create alexandria-gradle-plugin that does the same thing the mave plugin does

might be worth abstracting an AlexandriaPlugin class in alexandria-core if they seem to be identical but for the maven/gradle I/O, though the Alexandria/Context classes are probably as is. Maybe a lombok builder...

jive code block rendering is still off

missing line numbers:

<pre class="language-none line-numbers"><code>foo<br />bar&zwj;&zwj;&zwj;&zwj;&zwj;&zwj;<span class="line-numbers-rows"><span>&zwj;</span><span>&zwj;</span></span></code></pre>

language markup is also off. The syntax highlighter in the gui is adding more stlying like:

<code>remote<span class="token punctuation">:</span>  <br />  <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>  <br />metadata<span class="token punctuation">:</span>  <br /><span class="token operator">-</span> sourcePath<span class="token punctuation">:</span> <span class="token string">"docs/images.md"</span>

Might not be worth trying to get it working perfectly.

implement delete document

maven plugin doesnt convert document when source changes

Come up with a life cycle to help define remote interface

Off the top of my head: Update metadata -> prepare documents -> create/update/delete -> save metadata

add parameter to force sync operation on remote

Add optional warning to remote doc

Something like "managed by Alexandria changes made here will be overwritten"

Maybe include link to the markdown doc in the source repository.

Implement default document tags

Support other Jive Content types than Document

jive documents have different types (e.g. document, blog, discussion, etc) and need to be specified. Alexandria currently only supports document types. It could be expanded on without too much effort, but will likely not be supported unless there is interest.

jive - new local file based on existing document throws null pointer exception

When adding a local file to the index that already exists in jive, we get a NPE when sync runs. Its probably not properly finding the content id for some reason

[ERROR] BatchProcessException{message=Alexandria batch error., exceptions=[HttpException{
   message="HttpException: Unexpected error executing PUT",
   rootCauseMessage="NullPointerException: pathSegment == null",
   metadata=
{
  "sourcePath" : "REDACTED/docs/index.md",
  "title" : "Unified Message Bus (UMB) Documentation Index",
  "remoteUri" : "https://REDACTED/docs/DOC-1072237",
  "convertedChecksum" : 3681148503,
  "extraProps" : {
    "jiveParentUri" : "https://REDACTED/groups/REDACTED",
    "jiveParentApiUri" : "https://REDACTED/api/core/v3/places/1508053",
    "jiveParentPlaceId" : "1508053",
    "jiveTrackingTag" : "b3e2f348-6a72-47f5-a7dd-20b4df432157"
  }
}
   request=null
   response=null
}
]}
[ERROR] HttpException: Unexpected error executing PUT
com.github.macgregor.alexandria.exceptions.HttpException: Unexpected error executing PUT
    at com.github.macgregor.alexandria.exceptions.HttpException$Builder.build (HttpException.java:107)
    at com.github.macgregor.alexandria.remotes.RemoteDocument.put (RemoteDocument.java:165)
    at com.github.macgregor.alexandria.remotes.jive.JiveRemote.update (JiveRemote.java:220)
    at com.github.macgregor.alexandria.AlexandriaSync.lambda$syncWithRemote$1 (AlexandriaSync.java:97)
    at com.github.macgregor.alexandria.BatchProcess.execute (BatchProcess.java:97)
    at com.github.macgregor.alexandria.AlexandriaSync.syncWithRemote (AlexandriaSync.java:77)
    at com.github.macgregor.alexandria.Alexandria.syncWithRemote (Alexandria.java:93)
    at com.github.macgregor.alexandria.SyncMojo.execute (SyncMojo.java:19)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: java.lang.NullPointerException: pathSegment == null
    at okhttp3.HttpUrl$Builder.addPathSegment (HttpUrl.java:1043)
    at com.github.macgregor.alexandria.remotes.Requests.addSegments (Requests.java:72)
    at com.github.macgregor.alexandria.remotes.Requests.routeBuilder (Requests.java:41)
    at com.github.macgregor.alexandria.remotes.RemoteDocument.route (RemoteDocument.java:276)
    at com.github.macgregor.alexandria.remotes.RemoteDocument.put (RemoteDocument.java:152)
    at com.github.macgregor.alexandria.remotes.jive.JiveRemote.update (JiveRemote.java:220)
    at com.github.macgregor.alexandria.AlexandriaSync.lambda$syncWithRemote$1 (AlexandriaSync.java:97)
    at com.github.macgregor.alexandria.BatchProcess.execute (BatchProcess.java:97)
    at com.github.macgregor.alexandria.AlexandriaSync.syncWithRemote (AlexandriaSync.java:77)
    at com.github.macgregor.alexandria.Alexandria.syncWithRemote (Alexandria.java:93)
    at com.github.macgregor.alexandria.SyncMojo.execute (SyncMojo.java:19)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[WARNING]
com.github.macgregor.alexandria.exceptions.BatchProcessException: Alexandria batch error.
    at com.github.macgregor.alexandria.exceptions.BatchProcessException$Builder.build (BatchProcessException.java:63)
    at com.github.macgregor.alexandria.BatchProcess.execute (BatchProcess.java:135)
    at com.github.macgregor.alexandria.AlexandriaSync.syncWithRemote (AlexandriaSync.java:77)
    at com.github.macgregor.alexandria.Alexandria.syncWithRemote (Alexandria.java:93)
    at com.github.macgregor.alexandria.SyncMojo.execute (SyncMojo.java:19)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Source files with the same names will likely cause collisions when converted

Eg multiple README.md's at different levels, or multiple indexes, etc. The last one converted will be used for all of them. Havent run into it or tested but pretty sure thats what will happen.

add retries for rest calls

mojo parameter to not fail build when there are alexandria exceptions

Jive parent search api is terrible, make Alexandria more intelligent to compensate

maybe fallback to searching though groups the user is following/member/owner of. These queries are expensive though.

Fix Travis build/deploy

maven plugin metadata not quite right

intellij doesnt like configuration section, complains about syntax error. You also have to explicitly set index/convert/sync executions for it to work.

add progress indicator

Resolve relative links to remote urls

improve batched error reporting

it can be hard to figure out exactly where an error is happening currently

add classes to html renderer for jive

code blocks: class="language-%s line-numbers" in the pre tag, not the code tag which is default
tables:

<table class="j-table jiveBorder" style="border: 1px solid #c6c6c6;">
header:

<thead>
  <tr style="background-color: #efefef;">
  <th>Tables</th>
  </tr>
</thead>

alignment uses style="text-align: center;" instead of "align"

Get maven plugin unit tests working

allow loading of remote config from env variables

line breaks arent rendering nicely on jive

It is especially bad for code blocks, but space between

tags are disappearing too.

include/exclude glob patterns unintuitive

file system:

projectRoot/
    -> docs/
        -> messages.md
        -> client_provision.md

mojo plugin:

<plugin>
  <groupId>com.github.macgregor</groupId>
  <artifactId>alexandria-maven-plugin</artifactId>
  <version>0.1.4</version>
  <configuration>
    <failBuild>false</failBuild>
    <timeout>60</timeout>
    <includes>
      <include>docs/client_provision.md</include>
      <include>docs/messages.md</include>
    </includes>
  </configuration>
  <executions>
    <execution>
      <id>alexandria</id>
      <goals>
        <goal>index</goal>
        <goal>convert</goal>
        <goal>sync</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Doesnt match though it seems like it should. also tried "**/docs/messages.md", "/**/docs/messages.md", "./docs/messages.md" none of which worked. "**messages.md" worked as did just "messages.md".

Pretty sure the PathFinder is recursively walking the directory tree and only doing matching against file name, which apart from being unintuitive could lead to problems matching multiple files with the same name but in different directories (like README.md's in submodules). It could also have something to do with matching against absolute paths, since all paths are made absolute during processing, but then "**/docs/messages.md" should have worked.

support multiple remotes or pipeline deployments

two use cases here:

two completely different remotes, like Jive and Google docs (or whatever, not suggesting supporting google docs api).
multiple sequential environments of the same type of remote, so pre-prod then prod instances.

This will likely never be implemented unless there is outside interest.

Add another example remote

Generate starter config and metadata

javadocs for maven plugin and cli

get alexandria-maven-plugin tied to the default deploy execution goal

Provide support for retrieving and updating metadata from the remote

For example, a workaround to the parent place shitty search problem could be for the user to manually make that change and then sync the metadata to pull in the jiveParentUri and all that.

Note: this will not support sync changes to the document itself, just metadata

Proide more control over metadata states to user (e.g. never delete from remote)

always update
never delete
force recreate?
delete without having to remove the file from the file system

Jive search api is really inaccurate and breaks parent place search

The api is limiting. Best solution may be to just page through and do the exact name match myself. Though, there are filters for only looking for palces you are a part of so that might help.

maven plugin throwing null pointer exception for rootDir()

this it has to do with having a parent pom that isnt relative (the parent doesnt have a baseDir because its remote on maven central or wherever).

workaround: manually specify config path and input:

<configuration>
                        <configPath>${basedir}/.alexandria</configPath>
                        <inputs>
                            <input>${basedir}</input>
                        </inputs>
</configuration>

Some field level javadocs arent being copied to the relevant getter/setter. See https://macgregor.github.io/alexandria/com/github/macgregor/alexandria/Config.DocumentMetadata.html vs https://macgregor.github.io/alexandria/com/github/macgregor/alexandria/Config.RemoteConfig.html

possible soultion: projectlombok/lombok#132

macgregor / alexandria Goto Github PK

alexandria's People

Contributors

Stargazers

Watchers

alexandria's Issues

Recommend Projects

Recommend Topics

Recommend Org