vipickering / mastr-cntrl Goto Github PK

Both Swarm posts failed last night.

Investigate

When the content is converted to base64 and encoded to Github, it sometimes garbles characters like

/n
'
"

Expand webmentions so that:

1. Any posts modified or updated are added to the webmention queue, along with new posts. (vincentp site)
2. Loop through all the items in the JSON array and POST each one to Telegraph.
3. Update date in Github again.

Getting fancy here:

https://www.w3.org/TR/micropub/#h-source-content

I should probably own my own data and import all my old checkins

Now the media endpoint exists, I can support multiple images per post easily.

Extend the formatter to allow this.

1. I need to accept an incoming feed.
2. Classify it in to a format
3. Save/Create in memory the file
4. Submit it to Github somehow.

I need to write the Readme and instructions

Mastr-cntrl should be submitting not Zapier.

I need to accept ownyourswarm content and format it correctly.

Support uploading images by using Multer and creating a media endpoint.

https://www.w3.org/TR/micropub/#media-endpoint

• Create Media Endpoint
• Return Media endpoint in Auth Header
• Endpoint validates images sent, returns correct URL and uploads to correct location

Use the webmention.io webhook and callback secret instead of Zapier.

Replace the webmention-update (GET) route with a webmention(POST) route.

The new route will need to check the token validity and if OK, format the webmention and push it in to the JSON file.

Now I have the media endpoint and it supports multiple photos on notes, it also should support multiples on instagram posts.

When a webmention is successful and the code is sent, the publish date is updated, but the file is wrongly encoded, leaving slashes.

I can either pull this data myself from last.fm. Or use some 3rd party service to get the data summaries.

If posts/notes etc are deleted at any point I need some mechanism to send deleted posts.. Perhaps a special URL with validation?

https://www.w3.org/TR/webmention/#h-sending-webmentions-for-deleted-posts

It looks like we are iterating over the tags array incorrectly.

Instead of accessibility, it outputs a c c e s s i b i l i t y for example.

The app needs to respond to a GET request with param q=syndicate-to

See here for example:
https://quill.p3k.io/docs/syndication

Posting on swarm and backfeeding is suddenly resulting in occasional double posting of webmentions or dropped mentions

I have also spotted a failed attempt to update the webmention.json file.

Add some extra logging around all webmention functionality and track behaviour to analyse why this is happening.

Once complete I need to modify the link to enable webmentions

3.8 Error Response

If there was an error with the request, the endpoint must return an appropriate HTTP status code, typically 400, 401, or 403, and may include a description of the error. If an error body is returned, the response body must be encoded as a [JSON] object and include at least a single property named error. The following error codes are defined:

HTTP 403: "error":"forbidden" - The authenticated user does not have permission to perform this request.
HTTP 401: "error":"unauthorized" - No access token was provided in the request. Note that this is different from the HTTP 403 response, as the 403 response should only be used when an access token is provided and the user does not have permission to perform the request.
HTTP 401: "error":"insufficient_scope" - The scope of this token does not meet the requirements for this request. The client may wish to re-authorize the user to obtain the necessary scope. The response may include the "scope" attribute with the scope necessary to successfully perform this request.
HTTP 400: "error":"invalid_request" - The request is missing a required parameter, or there was a problem with a value of one of the parameters
Clients should treat unexpected error strings as a generic error. The response body may also contain an error_description property with a human-readable description of the error message, used to assist the client developer in understanding the error. This is not meant to be shown to the end user.

I need to return more granular errors if possible.

Adding update https://www.w3.org/TR/micropub/#h-add
Removing (deleting/modifying) update https://www.w3.org/TR/micropub/#h-remove

Make views return either a nice formatted page, or a json array.

I need to support syndicating my bookmarks to pinboard.

I haven't done a good job implementing the MF code for post entries.

This needs reviewing and checking against a parser.

The media endpoint supports multiple imagery and will return stings for location in an array.

The frontmatter supports this.

The layout does not. It will only output the first image.

I need to support liking/ favouriting other people's content.

Hi Vincent,

Hope you don't mind, but been playing around with this micro service to see if I can use it for my own purposes. However, I have run across a few issues, so thought best to report them here.

The first is that it seems that even though JSON is being sent with the content value populated, mastr-cntrl is reporting there being no content. In addition to this, on attempting to send a 400 error response, res is not defined:

11:45 PM
23:45:43.134Z  INFO micropubAPI: json body {"h":"entry","access_token":"TOKEN_WOZ_HERE","content":"This is my content","category":["test"],"mp-slug":"test"}
11:45 PM
23:45:43.521Z  INFO micropubAPI: {"me":"https://micropub.paulrobertlloyd.com/","issued_by":"https://tokens.indieauth.com/token","client_id":"https://quill.p3k.io/","issued_at":1545080991,"scope":"create update media","nonce":287327756}
11:45 PM
23:45:43.523Z  INFO micropubAPI: Service Is: https://quill.p3k.io/
11:45 PM
23:45:43.523Z  INFO micropubAPI: Payload JSON: {"h":"entry","access_token":"TOKEN_WOZ_HERE","content":"This is my content","category":["test"],"mp-slug":"test"}
11:45 PM
23:45:43.523Z  INFO micropubAPI: Service Quill. Creating Note
11:45 PM
23:45:43.523Z  INFO micropubAPI: Note JSON: {"h":"entry","access_token":"TOKEN_WOZ_HERE","content":"This is my content","category":["test"],"mp-slug":"test"}
11:45 PM
23:45:43.523Z  INFO micropubAPI: No Content. Ending
11:45 PM
23:45:43.526Z ERROR micropubAPI: res is not defined
11:45 PM
    ReferenceError: res is not defined
11:45 PM
        at Object.note (/app/app/functions/formatters/note.js:29:9)
11:45 PM
        at authAction (/app/app/routes/post/micropub.js:109:34)
11:45 PM
        at process._tickCallback (internal/process/next_tick.js:68:7)

I also noticed that in app/routes/post/micropub.js you use response as a parameter for the sendtoGithub function , but then res inside it. Changing that caused further issues (logger.info(JSON.stringify(json)) on line 63 returned a circular JSON error message) at which point I thought I'd stop fiddling and file this issue.

(N.B. Currently running this app on Glitch, if that helps debug this issue.)

This is a big one:

https://www.w3.org/TR/micropub/#h-delete

Currently this is fixed at Swarm. I just need to work out the logic to determine the originator.

Really, I don't need separate format-type.js files.

Instead I need to isolate the differences and append them to a standard format.

I already have the logic to identify separate sources so I can reuse this to apply the differences and normalise the rest of the code, removing the duplication.

I can use Zapier to push my Twitter content over to the API.

This can then be formatted, and added for webmentions.

I'll need slightly different designs for tweets, replies, retweets and likes.

Instead of adding the data to a JSON file (though we can still do that to keep a record).

The data needs to be written directly in to the post some way.

I need to think around how I add data to the templates to do this.

Authentication via IndieAuth should be passed along the route and not inside individual functions

• Build data file instead of using webmention.io on vincentp.me
• Make vincentp.me use data file
• Write to file directly from API (this works on a test file)

Vouch is anti-spam for Webmentions and involves doing some GET lookups to get authenticity.

Details here https://indieweb.org/Vouch

Notes shouldn't be allowed to be created if they are blank.

• Successfully get content for test post and send correct response
• Format content being injected in to Github with correct filename

support micro.blog

Add Prettier to eslint workflow
https://prettier.io/

Remove/override or disable conflicting rules.

Currently when Emoji are used in comments they get garbled in the conversion.

This is because I am not handling escaping or conversion of anything. Just processing as plain text.

When Micropubbing.

Once we select a syndication target, the system needs to handle syndicating the content.

Content is returned from Quill (best example) in this format.

{
"mp-syndicate-to":["https://twitter.com/vincentlistens/"]
}

On receiving the package we need to look for the existence of this item, and if it exists handle sending the json content to each provider on the list. We should get the destination of each item as it is posted, so they can be added to the array of data posted to the Github API and output as "syndicated to: xxx, xxx,xxx" etc.

If I syndicate to another site, then unless it is Medium it is not output.

I need to make this iterate over an array or list of items. Articles in the future could be syndicated to multiple sources.

I'm pretty confident everything is working as planned so I can step up the frequency.

I need a way to identify if a post is published, then fire a webhook to initiate Micropub to GET content from Github and POST to source.

• On git push vincentp.me fire web hook from Github.
• Web hook pings Micropub API, to get latest post.
• Micropub API does a GET on Github Repo /posts folder where X matches X for latest date.
• GET latest post(s) content as above and publish to external source.

You get 1000 requests a day for free. I should be OK with that!

https://darksky.net/dev

I can now accept Webmention reply strings from Quill or another 3rd party.

Outstanding is

• to send a webmention on publish
• to format the reply text. Get the domain.com instead of dumping the full string.

It would be nice if each week at a certain time the API published a "from the archives" tweet linking to an article.

• Create webmention endpoint URL
• Validate incoming
• Take Webmention and formate to JSON.
• Perform PATCH operation to Github and update data file
• Send outcome status (200/404 etc)
• Support Create New
• Support Update

I've been thinking that perhaps a single JSON file is not the best way to store this content.

If it's in one file a static build site needs to loop through it all multiple times to get all the right content in the right places as it builds.

Would it be quicker if the webmention was stored in front-matter for the blog post it is related to?

• This could introduce a number of challenges, including referencing the right file.
• An alternative would be for each post to have an attached webmention file to read/enter. Although this could simply re-introduce the longer build time in a different manner.

Recommend trying a spike to see how inserting in the front matter affects performance.