hi @voxpelli! when you get a chance, would you mind counting the total number of webmentions webmention.herokuapp.com has successfully received that aren't from bridgy? ie source URL isn't [www.]brid.gy or brid-gy.appspot.com.

i think we're near 1M total webmentions, as a community, and I'm hoping to pinpoint exactly when we cross the line. background: https://snarfed.org/2018-01-02_bridgy-stats-update-3

thanks in advance!

(First, thank you so much for working on this project, I've been looking forward to something like this for ikiwiki!!)

It seems that the script does not support anchors in the URL. So webmentions to http://example.org/page.html do not show up when opening http://example.org/page.html#section-about-blah-blah

(Tested at https://hroy.eu/posts/whats-in-C-131_12/ but I suppose it's the same everywhere)

As a possible follow up of #48 – maybe also look into enable dat-sites to embed webmentions through purely the dat-protocol?

go to http://mention-tech.appspot.com
and put 'http://mention-tech.appspot.com/' in source and 'http://kevinmarks.com/' in target

It verifies the webmention, then calls
'https://webmention.herokuapp.com/api/webmention?source=http%3A%2F%2Fmention-tech.appspot.com%2F&target=http%3A%2F%2Fkevinmarks.com%2F'

and gets '400' {"error":true,"message":"You need to specify both a "source" and a "target" URL"}

First pointed out by @kevinmarks on: http://www.kevinmarks.com/decentralweb2016-06-08.html

Then also discovered on @miklb's blog: https://miklb.com/jekyll-and-microblogging Update: Not using my endpoint, but instead webmention.io, but has a duplication problem as well which points to Bridgy being part of the problem maybe.

This happened before the deploy of 0.11.x so it means that the cause has to be outside of this service, but it could very well be due to me interpreting something wrong. Will have to dig into it.

See: https://indiewebcamp.com/person-tag

That doesn't allow signup but which rather works like https://github.com/voxpelli/webpage-micropub-to-github – has all of its config made through environment variables and is easily deployable to Heroku through a single click

like in #50 :)

With #18 fixed and the comments now being curlable the next step would be to utilize that curlability by becoming part of the salmentions ecosystem and both receive salmentions from others and forward them.

This moves this endpoint closer to be able to take part in a SWAT0 dance.

• Fetch the WebMention pinged target site if it isn't already in the database and store it in entries and especially ensure to save its interaction targets
• Fetch all the interaction targets found that isn't already in the database and extract their WebMention endpoints. Save these as entries just like all other entries?
• Parse the comments included in the WebMention pinged source
• Fetch each found comment that isn't already in the database and save them as entries and mentions. If they are not targeted at a registered site, then maybe mark them as "external" or something
• Present the full comment tree – mentions of mentions of mentions – at least in the embed type created in #18 and especially on the standalone page
• Ensure that a too deep comment tree doesn't break presentation – look at how eg. Disqus does it – after X many levels don't indent the additional levels anymore. This is a pure styling issue.
• Whenever all comments has been fetched, if any new were found – send a WebMention to each interaction target of the original WebMention pinged target
• Ensure that if an earlier post updates to mention a post later in the chain that no infinite salmentioning will happen – this is probably more of a problem in the presentation part than in the pinging part. If the relation gets circular then any attempt at trying to fetch a tree representation of that relation will be infinitely deep. A graph database would really make sense here – fetching an entire tree in PostgreSQL isn't really that optimal – one would really need one JOIN per level
• Make it opt-in? So only those using the embed from #18 actually get Salmentions sent as the rest won't support it anyway?
• Ping person-tags – important – critical part of SWAT0
• Ensure pings are only sent on updates and try to avoid getting into a circular pinging exercise where you ping others and therefore they ping you, so therefore you ping them again and so on
• Ensure pings are only sent when the data presented on the u-responses page has changed – we should not ping when the receiver has no way of detecting any actual change
• Do some caching – don't refetch everything always
• On an update: Diff the content and only ping and refetch what has actually changed – true especially for comments
• Add support for if-modified-since and etag to check for fresh content
• Update the js-embed to also show the full comment tree
• Add a way to actually opt-in in the UI – and require IndieAuthing (related to #30) with the domain to verify ownership or something before allowing it
• Support sending Salmentions for non-interactions, for the mentions, without pinging hundreds of URL:s – a tricky one

So this more or less means that rather than just simply fetching a single page, the source, per WebMention the script would, in the worst case scenario, have to fetch four different types of pages: The source, the target, all comments included in the source and all targets of the target.

And when all that is done it has to send WebMentions to all the WebMentions endpoints of the targets targets.

So a source with a single comment on a target with a single target would mean fetching 4 pages and doing 1 WebMention rather than just fetching 1 page as is done today. And 4 times the metadata will be stored. This metadata will be possible to use to better present reply-context and such on eg. the standalone embed pages and such though and could perhaps be used in other ways as well.

There should check for whether the content in the p-name is already present in e-content or p-summary.

And then all existing mentions should have that check run against them.

Page where duplicated content occurs: http://www.kevinmarks.com/xoxofest2014a.html

Eventually someone will get very many mentions and it will be unfeasible to show them all at once. Paging should be introduced both in the embeds and in the curlable views introduced in #18.

In the embeds the count should be configurable. On the standalone page it should probably have a fixed, high default – like eg. 100 – which would probably be the highest maximum to allow anywhere.

Just like there's now a HTML page for GET requests to the endpoint itself it would be nice to have a HTML page for the response of a POST so that HTML forms posting to the page can end up somewhere nice.

As we know the target we can also provide a nice link pointing back to he target so that the page doesn't become a dead end for people.

Tasks needed for this:

• basic layout system to make the layout of the page reusable (perhaps through the use of node-tema?)
• success page
• error page

Nobody is perfect and neither is this app – it will fail, but it should save the necessary data to be able to recover as well as possible once the failing link in our chain of events has been identified.

By logging (to a reasonably degree) all unexpectedly failed errors into a table of its own (there's already one called failurelog) it will get easier to both identify errors (the log services at Heroku currently just keeps 7 days worth of logs) and to rerun them if we find that a bug in the code was the failing factor.

Until eg. snarfed/granary#31 is completed an RSS/Atom-feed of mentions which one can feed into eg. IFTTT to get notifications from would be great. And it needs to be PuSH-enabled to get push notifications.

This is a requirement for SWAT0.

• Create Atom feed
• Add PuSH to h-feed and Atom feed

See http://indiewebcamp.com/bookmark and the https://aaronparecki.com/bookmarks/2015/01/16/1/indieweb / http://www.kevinmarks.com/we-like-indieweb-software.html interactions

Expose in the facepile using a .webmention-interaction-bookmark class and use a star or bookmark symbol as the default icon on the endpoint page itself.

As the endpoint equivalent of pfefferle/wordpress-webmention#30 we should be as tolerant of clients as the clients are of the servers – that issue is about making a client retry its pings when a server fail and likewise we should also retry a ping confirmation when a client fails to deliver a source page and also be very relaxed in how many pings we accept within a limited timeframe.

This would require that pings are saved in the database before they are confirmed so that they can survive a restart or crash. One reason to not accept more pings right now than we do is that they will all get lost if eg. the Heroku instance goes to sleep so first move would be to solve that. Saving them to the database will also enabled retries over a long timespan as well – as discussed in the other issue.

And lastly: Document the final solution on http://indiewebcamp.com/webmention#retrying

As a follow up to #35 and following the thoughts written down in the wiki at https://indieweb.org/Webmention-brainstorming#de-duplication it probably makes sense to do some more sophisticated deduping than the current one and have the u-url be a basis for deduping as well and when a u-url is used for deduping have the source "closest" to that u-url be the winner (exact match, host match etc).

This will avoid the situation that appeared in #35 from happening again:

By using eg the Bloglovin metadataparser.

This will make presentation of non-microformatted mentions better.

Only GitHub on an IndieWeb service is – um – weird.

Solves mentions of shortened URL:s and other non-canonical URL:s

Should be possible after #21 as it will start fetching the target.

I was just trying to set this up on my server when I realized that it wanted iojs 3.x. I was just wondering if you had any plans to update it to use node 6. Thanks!

As there is no way to export the webmentions, an easy way to implement that is to query per domain and retrieve all mentions for it.

Basically is doing WHERE LIKE instead of WHERE =

See IRC-discussion: http://indiewebcamp.com/irc/2015-07-01/line/1435744239727

Posts on my site https://jeremycherfas.net point to the home page for author. There is an h-card on the home page, but it does not get picked up at the moment.

Meanwhile, I'll try to add the h-card to individual posts.

A follow up to #43.

That issue was about not accepting any new such mentions, but of course old such mentions should also be cleaned away, using a migration script to do it.

Hi,

thanks again fro your app, that's amazing :)
I just checked with checkmention, and I noticed the malicious one were passing by.. Here is an example.
when I'll have time, I'll have a look, I open the issue more to get it logged.

Thanks!

See eg:

• https://github.com/beakerbrowser/beaker/wiki/Authenticated-Dat-URLs-and-HTTPS-to-Dat-Discovery
• https://www.npmjs.com/package/dat-dns
• https://github.com/dat-land/awesome-dat#dat-link-utilties
• https://indieweb.org/Dat

etc

What do you think about a request on a "base" url or a list of urls to get all mentions of these urls.

Why I ask: I have a "contact" section on the http://pfefferle.github.io/openwebicons/ and I want to show every mention there, independent from the specific mentioned URL.

Do you think there is a way to remove (or better refresh) broken avatars?

Example: http://pfefferle.github.io/openwebicons/contact/

A site using webmention.herokuapp.com (https://www.rosemaryorchard.com/microblog/2018-07-09-1816) got a webmention that used an implied p-name and it's all jumbled.

According to microformats/microformats2-parsing#6 p-name is no longer supposed to have such strong implied attributes when it doesn't exist on a page.

Mention URLs are being appended with a post's title as seen in the mention at this URL: https://prettygoodhat.com/note/letsencrypt-pair/; the correct mention URL is https://prettygoodhat.com/post/further-indieweb-adventures/; the URL linked in the mention is https://prettygoodhat.com/post/further-indieweb-adventures/Further%20adventures%20in%20IndieWeb -- the extra string is the title of the source post.

I apologize if this is a usage question and I've made a mistake in my formatting; but I'm honestly not sure how I could be mis-specifying the URL in this way. I don't see that url representation in my page source, anyway. Maybe this is a bug somewhere?

A portion of the JSON for that mention is below:

{"source":"https://prettygoodhat.com/post/further-indieweb-adventures/","target":"https://prettygoodhat.com/note/letsencrypt-pair/","data":{"url":"https://prettygoodhat.com/post/further-indieweb-adventures/Further%20adventures%20in%20IndieWeb","name":"Further adventures in IndieWeb","published":1520780017032,"summary":"[...]","author":{"name":"Alan S.","photo":"https://prettygoodhat.com/pics/u-photo.png","url":"https://prettygoodhat.com/"}}},

If someone finds an unwanted webmention then they should be able to remove it. Kind of a critical feature and without it a version 1.0.0 can't really be claimed to be achieved.

When manual moderation is possible, automatic one can also be added on top – such as spam-protection – but many such mechanisms needs to be possible to audit by users and therefore the manual interface has to come first.

For those who don't want to rely on javascript for displaying their received WebMentions and neither wants to pull them themselves from any of the API:s provided already a standard support for pushing them to a Micropub endpoint would be great.

It's something that @aaronpk is already more or less doing in his endpoint: https://github.com/aaronpk/webmention.io/blob/master/helpers/formats.rb#L80 Almost Micropub, but not really.

Would be great if our two endpoints supported the same API as eg. @dshanske was intrigued by that for use with WordPress. See chat logs: https://chat.indieweb.org/dev/2016-08-15#t1471278307859000

It's great that people can use my endpoint, but in the spirit of the indieweb they should of course be able to host it on their own as well.

So it should be possible to activate a single user mode where people can't sign up but where rather everything is configured like it is in https://github.com/voxpelli/webpage-micropub-to-github

It should also be possible to easily deploy the project, using an Heroku button or any of the other buttons that have been suggested for that project: voxpelli/webpage-micropub-to-github#26

Do however note below such a button that unlike the Micropub endpoint a Webmention endpoint will need to pretty much run 24/7 as long as people interacts with oneself 24/7 and with scripts embedded on one sites that's very likely to happen. With a future #37 the need would be less, but still – every mention or Salmention would activate ones server and that would quickly eat up the free server time one has on Heroku

I changed GitHub accounts recently and my domain is still associated with the old account, which has been deleted. Is there any way I can get the domain released so I can add it to my current account?

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update dependency express to v4.19.2 [SECURITY]
• Update dependency grunt to v1.5.3 [SECURITY]
• Update dependency moment to v2.29.4 [SECURITY]

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency passport to ^0.6.0 [SECURITY]
• Update dependency knex to v2 [SECURITY]
• Replace dependency faker with @faker-js/faker ^2.1.5
• Replace dependency npm-run-all with npm-run-all2 ^5.0.0
• Update dependency @types/knex to ^0.16.0
• Update dependency pg-pubsub to ^0.8.0
• Update Node.js to v20
• Update dependency dotenv to v16
• Update dependency faker to v6
• Update dependency grunt-contrib-concat to v2
• Update dependency grunt-contrib-uglify to v5
• Update dependency pg to v8
• Update dependency typescript to v5
• Update linting dependencies (major) (eslint, eslint-config-standard, eslint-plugin-es, eslint-plugin-node, eslint-plugin-promise, eslint-plugin-security, eslint-plugin-standard, eslint-plugin-unicorn, installed-check)
• Update test dependencies (major) (@types/mocha, @types/sinon, @types/supertest, chai, husky, mocha, nock, nyc, sinon, supertest)
• Lock file maintenance
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

As that's apparently possible currently: https://voxpelli.com/2016/07/better-handle-npm-modules/

When the real non-normalized source and target URL:s are equal that is. Two normalized URL can look the same while still being meaningful when one mentions the other.

Mistakes has happened in the past with self-mentions: http://indiewebcamp.com/irc/2015-01-16#t1421444837504

The main blocker for a 1.0.0 release I think is supporting full CRUD operations. To be a good citizen with in the IndieWeb community one should respect peoples changes to and removal of posts and the current code of this app only accepts the very first initial creation ping and considers all subsequent pings as weird duplicates and that has to change.

Related IndieWeb wiki pages: http://indiewebcamp.com/CRUD

• update of entry and mention on repeated ping
• removal of entry and mention on new ping from same source
• removal of entry (and probably mentions) when source is tombstoned
• removal of entry and mention when source is completely gone (eg. 410)
• sync implementation with the flow described in the wiki
• add any meaningful documentation of this implementation to the wiki

It would be nice if I could decide whether to display the webmention block on my site depending on whether there is any content to display. If it were possible to test that the script will return anything, that would be great.

Since the service collects received webmentions, there needs to be a way (authenticated API or UI) to persistently delete a received webmention. That is:

• notify the service via UI or API) that a particular received webmention should be deleted from webmention.io's collection (that is, no longer returned in any API calls etc.)
• if the service receives another webmention with that same 'source', it should reject it (hence the "persistent delete"j)

This is essential for removing specific abusive webmentions (whether comments, or likes/reposts by abusive accounts).

As discussed in snarfed/bridgy#473 it would be useful to be able to import a Twitter block list, or any block list, to automatically block some mentions (so one doesn't have to manually block them though #28).

This endpoint should be able to consume and subscribe to such block lists (and ideally discover them from a user profile when someone eg. signs in using IndieAuth, #30)

Related wiki pages:

• https://indiewebcamp.com/private_posts#Public_Page_Upgrading
• http://microformats.org/wiki/block-list
• https://indiewebcamp.com/block

See more at: https://indiewebcamp.com/rsvp

Per discussion on IRC this morning, base the embeds on rel=response links that links to an h-entry-list of mentions so that mentions are discoverable without having to execute any javascript.

This makes the mentions degrade gracefully + makes the mentions curlable and thus possible for clients to find them.

Bonus fix: This also enables multiple embeds on the same page.

IRC-links: http://indiewebcamp.com/irc/2015-05-05/line/1430814384495

Things to consider as follow ups:

• Pubsubhubbub for the h-entry-list? Will probably be hard to get it to work on anything else than the simplest lists though unless the endpoint itself keeps tracks of who is subscribing so that it knows when to ping. If using a hub only the simplest lists can be feasibly supported.
• Render the original post as a context when possible: http://indiewebcamp.com/irc/2015-05-05/line/1430814641180 Start out with a link to the original post (when possible): http://indiewebcamp.com/irc/2015-05-05/line/1430814879737

Similar to what Aaron has done: http://aaronparecki.com/articles/2013/10/13/1/realtime-indieweb-comments

Should probably use EventSource for it.

See eg chat around here: https://chat.indieweb.org/2016-08-23#t1471976085095000

So that it says "Someone liked this" in summary/name rather than just "liked this" – the latter doesn't sound very friendly and neither is it very grammatically correct 😜

somehow, the script when embedded on christopheducamp.com, was trying to use https://webmention.herokuapp.com/api/embed?url=http%3A%2F%2Fchristopheducamp.com%2F2014%2F01%2F11%2FWebmentions-Jekyll%2F%2F instead of https://webmention.herokuapp.com/api/embed?url=http%3A%2F%2Fchristopheducamp.com%2F2014%2F01%2F11%2FWebmentions-Jekyll%2F

I edited the script tag to put a work-around in place:

    sn.src = sn.src.substring(0, sn.src.length - 3);

and now it works on http://christopheducamp.com/2014/01/11/Webmentions-Jekyll/ but it does seem like this is some sort of bug in how it determines the url.

Hello,

I was reading this post, and was trying to set up receiving of webmentions on my site scripter.co too..

But looks like this service is maxed out at 150 users?

Is there a plan to allow more users?

Thanks.

Examine the output of https://webmention.herokuapp.com/api/mentions?site=voxpelli.com :

[
  {"name":"New service: WebMentions for static pages",
   "published":1388833302000,
   "summary":"Want to implement webmentions but you’re using static pages a-la Jekyll? No problem. Pelle’s got you covered.",
   "author":{"name":null,"photo":null,"url":null},
   "url":"http://adactio.com/links/6625"
  }, ...]

Note that this describes a mention and where the mention came from, but not the page at which the mention was targeted! This means that it's not possible to fetch all the mentions for one's static site in one go; instead, one has to pass url=fullurl for each individual post to get the details, which takes one site=whatever request and makes it into (for my site) 1,700 url=whatever/full/url requests. Which is not good for anyone's bandwidth. :) Adding a target_url member to returned data would solve this problem.