Add support for new modmail API endpoints about snoowrap HOT 11 CLOSED

While working on this feature for PRAW, I collected some information on the API which might be helpful in implementing this:

https://gist.github.com/leviroth/dafcf1331737e2b55dd6fb86257dcb8d

from snoowrap.

Thanks! I'd been procrastinating on this because I still needed to investigate the endpoints and see how they worked, so this is a big help.

from snoowrap.

I've hacked around some parts of it over at https://github.com/VolunteerLiveTeam/Modmail2Slack/blob/master/vlt-modmail.js#L77 using oauthRequest - I haven't implemented the full API but it's something to go on

from snoowrap.

@not-an-aardvark any progress on this? Just asking, because I'd really want this to be a thing, and I'll probably do it myself if needed :P

from snoowrap.

Help with this would certainly be appreciated -- I haven't had much time to work on it myself lately.

I've done a small amount of work on it, but it would still need tests/docs, and there are probably a few missing endpoints too.

from snoowrap.

I was looking at implementing this. Some progress has been made, but I need some guidance as to how to parse everything. I got it to work doing the following, though we're then missing out on 90% of the response.

getNewModmailConversations (options = {}) {
  return this._getListing({uri: 'api/mod/conversations', qs: options, _name: 'ModmailConversation', _transform: response => {
    response.after = null;
    response.before = null;
    response.children = response.conversationIds.map(id => {
      return this._newObject('ModmailConversation', {id});
    });
    return this._newObject('Listing', response);
  }});

The get all Modmail conversations endpoint gives you the following structure. @not-an-aardvark this doesn't look like anything on the other endpoints, what's your thoughts?

{
    "conversations" : {
        "75hxt" : { ... },
        "75hxl" : {
            "isAuto" : true,
            "objIds" : [
                {
                    "id" : bdyfx,
                    "key" : messages
                }
            ],
            "isRepliable" : true,
            "lastUserUpdate" : ,
            "isInternal" : false,
            "lastModUpdate" : 2019-03-07T14:10:44.618955+00:00,
            "lastUpdated" : 2019-03-07T14:10:44.618955+00:00,
            "authors" : [
                {
                    "isMod" : true,
                    "isAdmin" : false,
                    "name" : SpyTecSnoowrap,
                    "isOp" : true,
                    "isParticipant" : false,
                    "isHidden" : true,
                    "id" : 262998090119,
                    "isDeleted" : false
                }
            ],
            "owner" : {
                "displayName" : SpyTecSnoowrapTesting,
                "type" : subreddit,
                "id" : t5_xogi3
            },
            "id" : 75hxl,
            "isHighlighted" : false,
            "subject" : you are an approved submitter,
            "participant" : {
                "isMod" : false,
                "isAdmin" : false,
                "name" : EliteDangerousBot,
                "isOp" : false,
                "isParticipant" : true,
                "isHidden" : true,
                "id" : 47524494,
                "isDeleted" : false
            },
            "state" : 0,
            "lastUnread" : ,
            "numMessages" : 1
        }
    },
    "messages" : {
        "bdyfx" : {
            "body" : <!-- SC_OFF --><div class=\"md\"><p>you have been added as an approved submitter to <a href=\"/r/SpyTecSnoowrapTesting\">/r/SpyTecSnoowrapTesting: SpyTecSnoowrapTesting</a>.</p>\n</div><!-- SC_ON -->,
            "author" : {
                "name" : {
                    "isMod" : true,
                    "isAdmin" : false,
                    "name" : SpyTecSnoowrap,
                    "isOp" : true,
                    "isParticipant" : false,
                    "isHidden" : true,
                    "id" : 262998090119,
                    "isDeleted" : false
                }
            },
            "isInternal" : false,
            "date" : 2019-03-07T14:10:44.618955+00:00,
            "bodyMarkdown" : you have been added as an approved submitter to [/r/SpyTecSnoowrapTesting: SpyTecSnoowrapTesting](/r/SpyTecSnoowrapTesting).,
            "id" : bdyfx
        },
        "bdyge" : {
            "body" : <!-- SC_OFF --><div class=\"md\"><p>test message</p>\n</div><!-- SC_ON -->,
            "author" : {
                "name" : {
                    "isMod" : false,
                    "isAdmin" : false,
                    "name" : EliteDangerousBot,
                    "isOp" : true,
                    "isParticipant" : true,
                    "isHidden" : false,
                    "id" : 47524494,
                    "isDeleted" : false
                }
            },
            "isInternal" : false,
            "date" : 2019-03-07T14:11:06.909685+00:00,
            "bodyMarkdown" : test message,
            "id" : bdyge
        }
    },
    "viewerId" : t2_3cti9t3b,
    "conversationIds" : [
        75hxt,
        75hxl
    ]

}

from snoowrap.

So if I'm understanding correctly: The conversations that get returned are identified by the IDs in conversationIds, and more information about each conversation ID can be found in conversations (presumably, each id in conversationIds is guaranteed to have a corresponding entry in conversations?). So when creating a new ModmailConversation, we would want to also include all the information corresponding to the given id from the conversations object.

Similarly, it seems like each conversation has a list of message IDs, and more information about each message ID appears in the messages object. So we would want to reassemble that too.

I'm not sure how pagination works here. Does the endpoint support the same type of limit queries as other listing endpoints?

from snoowrap.

(presumably, each id in conversationIds is guaranteed to have a corresponding entry in conversations?).

I'm quite certain that is the case. PRAW iterates through the conversations from the conversationsIds array.

Similarly, it seems like each conversation has a list of message IDs, and more information about each message ID appears in the messages object. So we would want to reassemble that too.

That is true, though I have not tested yet or looked at how other wrappers handle this. If how modmail functions in the GUI is anything to go by it will just show the latest message per ModmailConversations

I'm not sure how pagination works here. Does the endpoint support the same type of limit queries as other listing endpoints?

Yes, pagination works here as well, the API has pagination by after which is the base64 ID and limit

So how would we parse all of this in a good way? Is the way I started writing it the right approach to this? Will it detect the types going downwards such as for the conversations.75hxl.owner object? And would we need custom parsing for the conversations.75hxl.authors, conversations.75hxl.participant, and messages.bdyfx.author objects since it doesn't have type or id of the RedditUsers?

from snoowrap.

Right so when we get a Listing of ModmailConversation I made it as follows for now:

https://github.com/SpyTec/snoowrap/blob/feature/new-modmail/src/snoowrap.js#L863-L896

getNewModmailConversations (options = {}) {
  return this._getListing({uri: 'api/mod/conversations', qs: options, _name: 'ModmailConversation', _transform: response => {
    response.after = null;
    response.before = null;
    response.children = [];
    /** @member {Array} response.conversationIds */
    for (const conversation of response.conversationIds) {
      const messages = [];
      const otherObjects = [];
      response.conversations[conversation].participant = this._newObject('ModmailConversationAuthor', {
        ...response.conversations[conversation].participant
      });
      for (const objId of response.conversations[conversation].objIds) {
        if (objId.key === 'messages') {
          messages.push(response[objId.key][objId.id]);
        } else {
          if (!otherObjects[objId.key]) {
            otherObjects[objId.key] = [];
          }
          otherObjects[objId.key].push(response[objId.key][objId.id]);
        }
      }
      const data = {
        messages,
        ...otherObjects,
        ...response.conversations[conversation]
      };
      response.children.push(this._newObject('ModmailConversation', data));
    }
    return this._newObject('Listing', response);
  }});
}

With specific listing doing the following:

https://github.com/SpyTec/snoowrap/blob/feature/new-modmail/src/objects/ModmailConversation.js#L8-L45

_transformApiResponse (response) {
  response.conversation.owner = this._r._newObject('Subreddit', {
    id: response.conversation.owner.id,
    display_name: response.conversation.owner.displayName
  });
  response.conversation.participant = this._r._newObject('ModmailConversationAuthor', {
    modmailConversation: this,
    ...response.user
  });
  for (let author of response.conversation.authors) {
    author = this._r._newObject('ModmailConversationAuthor', {
      ...author
    });
  }
  const messages = [];
  const modActions = [];
  const otherObjects = {};
  for (const objId of response.conversation.objIds) {
    if (objId.key === 'messages') {
      messages.push(response[objId.key][objId.id]);
    } else if (objId.key === 'modActions') {
      modActions.push(response[objId.key][objId.id]);
    } else {
      if (!otherObjects[objId.key]) {
        otherObjects[objId.key] = [];
      }
      otherObjects[objId.key].push(response[objId.key][objId.id]);
    }
  }
  return this._r._newObject('ModmailConversation', {
    messages,
    modActions,
    ...otherObjects,
    ...response.conversation
  });
}

There has to be a better way to map all the objects correctly

from snoowrap.

Is there a useful difference between ModmailConversationAuthor and RedditUser? If not, we could just make the author a RedditUser.

To me, it seems like ModmailConversation could just expose a an objects list containing messages and anything else that reddit provides. That way we wouldn't be removing information about the order that the objects appear in, and we wouldn't need to separately filter out messages from modActions.

It seems like it would also be nice to have the messages property of a conversation be a Listing so that someone can fetch more messages easily, but we can deal with that later. (I'm assuming reddit sometimes truncates the messages in a specific conversation if there are too many, but I haven't tested this.)

from snoowrap.

Is there a useful difference between ModmailConversationAuthor and RedditUser? If not, we could just make the author a RedditUser.

ModmailConversationAuthor would catch the following fields from user, conversation.participant, and conversation.authors which are specific for the user in the subreddit's modmail. So putting them in RedditUser seems bad to me.

Examples:
user

{
  "user": {
    "recentComments": {
      "t1_ei5ih1f": {
        "comment": "...",
        "date": "2019-03-09T17:57:38.692577+00:00",
        "permalink": "https://www.reddit.com...",
        "title": "...."
      },
      "t1_ei5yec4": {
        "comment": "...",
        "date": "2019-03-09T21:18:55.217443+00:00",
        "permalink": "https://www.reddit.com.../",
        "title": "HCS / Voice Attack Cheatsheet"
      }
    },
    "muteStatus": {
      "isMuted": false,
      "endDate": null,
      "reason": ""
    },
    "name": "Name",
    "created": "2014-02-09T15:59:30.643957+00:00",
    "banStatus": {
      "endDate": null,
      "reason": "",
      "isBanned": false,
      "isPermanent": false
    },
    "isSuspended": false,
    "isShadowBanned": false,
    "recentPosts": {
      "t3_abcdef": {
        "date": "2018-06-17T20:47:52.055160+00:00",
        "permalink": "https://www.reddit.com...",
        "title": "If you were to play without 3rd party tools, what would you need to know?"
      },
      "t3_abcdeg": {
        "date": "2018-07-13T14:43:05.259408+00:00",
        "permalink": "https://www.reddit.com...",
        "title": "Title"
      }
    },
    "recentConvos": {
      "abcdef": {
        "date": "2019-03-11T16:41:45.496676+00:00",
        "permalink": "https://mod.reddit.com/mail/perma/abcdef",
        "id": "abcdef",
        "subject": "Edit removal of remove content"
      }
    },
    "id": "t2_abcdef"
  }
}

conversation.authors:

{
  "conversation": {
    "authors": [
      {
        "isMod": true,
        "isAdmin": false,
        "name": "SpyTec",
        "isOp": false,
        "isParticipant": false,
        "isHidden": false,
        "id": 12345678,
        "isDeleted": false
      },
      {
        "isMod": false,
        "isAdmin": false,
        "name": "SpyTecAgain",
        "isOp": true,
        "isParticipant": false,
        "isHidden": false,
        "id": 12345678,
        "isDeleted": false
      }
    ]
  }
}

conversations.<id>.participant:

{
  "conversations": {
    "77iiq": {
      "participant": {
        "isMod": false,
        "isAdmin": false,
        "name": "Abcdefg",
        "isOp": true,
        "isParticipant": true,
        "isHidden": false,
        "id": 12345678,
        "isDeleted": false
      }
    }
  }
}

Speaking of author, should messages and modActions arrays be their own listing of class objects as well? Particularly modActions as it has an enum for the actions:

To me, it seems like ModmailConversation could just expose a an objects list containing messages and anything else that reddit provides. That way we wouldn't be removing information about the order that the objects appear in, and we wouldn't need to separately filter out messages from modActions.

Right, so instead change the code to this, which takes all the objects associated with the conversation:

let objects = {};
for (const objId of response.conversation.objIds) {
  if (!objects[objId.key]) {
    objects[objId.key] = [];
  }
  objects[objId.key].push(response[objId.key][objId.id]);
}

It seems like it would also be nice to have the messages property of a conversation be a Listing so that someone can fetch more messages easily, but we can deal with that later. (I'm assuming reddit sometimes truncates the messages in a specific conversation if there are too many, but I haven't tested this.)

So far I haven't encountered anything suggesting there is any pagination of messages. Not seen any code in PRAW yet about such a thing. The Modmail GUI does truncate it, but it's a visual one

But even still, with things like conversation.owner we still need to write code to go through and convert them to snoowrap objects right?

from snoowrap.