Currently (almost?) all of our caches also have equivalent variables used to set the cache initially or for use as a live variable to access/update for the cache.

We should probably look into configuring our cache getting to use 3 methods to retrieve the value (in order of preference):

1. variable
2. cache
3. retrieve from api.stackexchange.com

Once a value is obtained, any preferred methods should also have it set. This will save on disk reads, and also allow for a method of writing to cache on an timer rather than on every change. sx-cache-get should not differentiate between sources, but look at them in sequence and then set those that require setting.

So, now's the time to decide. How do we want to display questions?

Just to be clear. This has nothing to do with writing questions or answers. This is about displaying questions (their entire content, including answers) which we get from the API.

The API offers us the question content in 2 formats:

• Markdown
• Html

The way I see it, we have 2 options:

Derive from org-mode

The current prototype uses this.

Pros:

• The question is a top-level headline and each answer is a level-2 headine.
• This gives built-in org-mode navigation and folding, which is really nice.
• Comments are description lists, which gives them right amount of emphasis in my opinion.
• Despite the screenshot bewow not showing it, we get automatic images display.

Cons

• The content of questions and answers has to be rendered in HTML. This is very stable (the question is well formated), but doesn't really offer decorations (everything just looks like regular text).

Use Markdown-mode

Pros:

• We can display some question properties at the top (title, author, views, etc), in a format similar to e-mail headers.
• Markdown mode is very good at decorating markdown format (duh), so the output is prettier than the html version.
• It also renders faster than the html version.

Cons:

• It's not obvious to me how to represent comments here.
• We have org-like navigation (markdown-mode does this) but not folding. We can patch this in easily, but it's a non-trivial amount of work, and therefore worthy of mention.
• We can have image display, but we'll have to implement it.

A popup-buffer similar to Org capture mode or twittering-mode's status updating mechanism would be ideal for this. I'm personally leaning towards the pop-up buffer (from the bottom), but that's just me.

It would be really nifty to display different types of data in the modeline (or somehow have access to it) as the user browses with n/p. Things like bounty, etc.

'Hot', 'Featured', etc.

In addition to searching for questions #16, we should add an option for searching for questions and answers using tags. Using tags makes it easier for others to find and answer your question.

RET works, though.

Debugger entered--Lisp error: (error "No selection is available")
  signal(error ("No selection is available"))
  error("No selection is available")
  mouse-yank-primary((mouse-2 (#<window 28 on *stack-question*> 833 (281 . 126) 77700094 nil 833 (40 . 8) nil (1 . 12) (0 . 14))))
  call-interactively(mouse-yank-primary nil nil)
  command-execute(mouse-yank-primary)

Expanding (with pp-macroexpand-last-sexp) the following

(sx-assoc-let response
  (when error_id
    (error "Request failed: (%s) [%i %s] %S"
           method error_id error_name error_message))
  (when (< (setq sx-request-remaining-api-requests
                 quota_remaining)
           sx-request-remaining-api-requests-message-threshold)
    (sx-message "%d API requests reamining"
                sx-request-remaining-api-requests))
  items)

yields

(let
    ((error_id           (cdr (assoc 'error_id           response)))
     (error_name         (cdr (assoc 'error_name         response)))
     (error_message      (cdr (assoc 'error_message      response)))
     (favorite_count     (cdr (assoc 'favorite_count     response)))
     (filter             (cdr (assoc 'filter             response)))
     (items              (cdr (assoc 'items              response)))
     (is_accepted        (cdr (assoc 'is_accepted        response)))
     (is_answered        (cdr (assoc 'is_answered        response)))
     (last_activity_date (cdr (assoc 'last_activity_date response)))
     (last_edit_date     (cdr (assoc 'last_edit_date     response)))
     (last_editor        (cdr (assoc 'last_editor        response)))
     (link               (cdr (assoc 'link               response)))
     (owner              (cdr (assoc 'owner              response)))
     (profile_image      (cdr (assoc 'profile_image      response)))
     (question_id        (cdr (assoc 'question_id        response)))
     (quota_remaining    (cdr (assoc 'quota_remaining    response)))
     (reopen_vote_count  (cdr (assoc 'reopen_vote_count  response)))
     (reputation         (cdr (assoc 'reputation         response)))
     (score              (cdr (assoc 'score              response)))
     (tags               (cdr (assoc 'tags               response)))
     (title              (cdr (assoc 'title              response)))
     (upvoted            (cdr (assoc 'upvoted            response)))
     (user_id            (cdr (assoc 'user_id            response)))
     (user_type          (cdr (assoc 'user_type          response)))
     (view_count         (cdr (assoc 'view_count         response))))
  (when error_id
    (error "Request failed: (%s) [%i %s] %S" method error_id error_name error_message))
  (when (< (setq sx-request-remaining-api-requests quota_remaining)
       sx-request-remaining-api-requests-message-threshold)
    (sx-message "%d API requests reamining" sx-request-remaining-api-requests))
  items)

when we only really need

(let ((error_id           (cdr (assoc 'error_id           response)))
      (error_name         (cdr (assoc 'error_name         response)))
      (error_message      (cdr (assoc 'error_message      response)))
      (quota_remaining    (cdr (assoc 'quota_remaining    response))))
  (when error_id
    (error "Request failed: (%s) [%i %s] %S" method error_id error_name error_message))
  (when (< (setq sx-request-remaining-api-requests quota_remaining)
       sx-request-remaining-api-requests-message-threshold)
    (sx-message "%d API requests reamining" sx-request-remaining-api-requests))
  items)

Not a priority, just here for future reference.
The current question list is pretty, but people have different preferences. Ideally, there should be a variable somewhat like the following, which users could configure to get the display they want.

(defcustom stack-question-list-format
  '(score answers title newline "     " date tags newline)
  ""
  :type '(repeat (choice symbol string))
  :group 'stack-question-list)

Because, of the way tabulated-list-mode is designed, the newline part has to be done is a very hacky way, which is the only thing that makes this non-trivial.

It'd be nice if Trello and GitHub issues could be integrated.

Inspired by #79 and #77.

I'll start working on this tomorrow. It should make tabs and search a lot simpler! Just set 3 variables and activate the mode.

See https://travis-ci.org/vermiculus/stack-mode/jobs/39595687#L241

I cannot reproduce this locally. See also https://github.com/vermiculus/stack-mode/blob/master/stack-core.el#L176.

Question/answer search

User must be able to archive questions for offline viewing

#30 makes a good case for explicitly requesting all of the fields that we use in each request. This should be implemented. Perhaps this should be implemented in the default filters.

To avoid bulk write-to-file for read/unread status etc, keep cache data as a variable rather than in file while active.

Use idle-timer to write to file.

We need a sx-question-list-mark-as-read command. Probably bound to m.

As a point of reference:

(questions ;; List to store all question info
 (site1 ;; One sublist per site
  (starred q1 q2 q3 ... qn) ;; List of starred questions on site
  (hidden q1 q2 q3 ... qn) ;; Questions to never see again
  (read ;; Read/unread status of the questions
   (q1 . q1-date)
   (qn . qn-date))
  (saved  ;; Offline storage of questions.  Defaults to any starred questions
   (q1 last-updated . (q1-data))
   (q2 last-updated . (q2-data))))
 (site2 ...))

read has been implemented by Bruce-Connor.

Retrieving favorites/starred per site is /me/favorites
q#-data would just be a write-to-cache/file copy of the question. last-updated being when the question was last stored.

Alternately for each stored question:

(q1 last-updated starred . (q1-data))

As in gitter:

(cl-defun sx-method-call (method
                          &key id
                               submethod
                               keywords
                               (filter 'default)
                               auth
                               (url-method "GET")
                               (site sx-default-site))
  ...)

This will also tie in to #78 and #75.

sx-request-make will only need to be as below to accomodate these changes.

(defun sx-request-make (url keywords url-method)
  ...)

There's certainly no absolute answer here.
I noticed you mentioned org-mode in the readme. I too thought about using an org-mode structure, and I've even started designing something of the type (I'll show you soon). The question I was left with is how to render the content of questions/answers.

The API offers two versions of a question's content: markdown and html.

We could take the markdown version and just place it inside a #+begin/end_src markdown block in org-mode (we could use faces to hide the actual #+begin/end_src delimiters). This would get markdown font-locking on the question and answer content so it should be a decent enough start.

The other option would be to render the html version. I think the w3m package is capable of rendering html in part of a buffer. That would probably be nicer to look at, but it'd be a bit more involved to implement.

Detected in #71. Applies to this answer: http://emacs.stackexchange.com/a/543/50

See http://stackapps.com/a/3891/11299 as an example.

I get the above message whentrying to start list-questions on a fresh machine.
Couldn't get a backtrace.

Full contents of the messages buffer:

[stack] Request: "https://api.stackexchange.com/2.2/questions"
Contacting host: api.stackexchange.com:443
progn: End of file during parsing

At least, I think that's the issue. 😉

Consider the following:

(sx-encoding-clean-content "I\342\200\231m a multibyte string.")

The following definition may help, but at least seems to not be enough:

(defun sx-encoding-clean-content (string)
  "Cleans STRING for display.
Applies `sx-encoding-normalize-line-endings' and
`sx-encoding-decode-entities'."
  (sx-encoding-decode-entities
   (sx-encoding-normalize-line-endings
    (string-as-multibyte string))))

Moving to the question buffer and using toggle-enable-multibyte-characters will fix:


Is there a buffer variable we can set to display these properly or (better, perhaps) a transformation we can use in *-clean to convert these to standard ASCII characters? Thinking of console use, I'm not sure most terminals provide extended character sets.

Just like we can mark questions as read, we should be able to mark them as hidden.

Hidden questions are then forever filtered out from the questions list.

I'm seeing this to milestone 0.1 as a suggestion. We can postpone it.

Right now, it's assumed that everything went perfectly. The backend JavaScript should be augmented to detect errors reported in the hash; see https://api.stackexchange.com/docs/authentication (heading Errors).

If sx-method-call is not given a filter, it sets filter to none,
but sx-auth--filter-p errors if it is not given a sequence.

When I was reading up on the SE API and authentication, I remember a page staying very clearly that they don't like bots that affect the content.
Given how emacs users have a tendency to optimize, and how easy it is to write a bot using sx, I think it might be wise to warn our users about this.

The best place to do so should be our "authentication successful" page. Since that's where they're given the power.

Things we should say:

The SE doesn't like automated scripts which affect the content of the websites.

• It's OK to define commands or abbrevs for comments you post very often.
• It's not OK to write a bot that down votes questions based on some criteria.

An identifier such as "mo" should be good, but I can't figure out how this data structure works. Perhaps a doc string should be added with this :)

There are about 2628000 seconds in a month.

This is very much a higher-level check and sx-request should simply not be concerned with it.

Reference: https://github.com/vermiculus/stack-mode/blob/0dd95e3a3d4ee52f52a585388c3ba65e045c305b/sx-request.el#L190

Just thought of one more thing to display in the question list mode-line. The current page (out of the total universe of questions). This would likely have to be estimated (or does the API tell us this?).

Proper testing for a compressed response is required. If a response comes back empty (i.e. {}), json-read-from-string will return nil. If the response was uncompressed to begin with, the current code tries to uncompress it and chokes.

A solution using the magic bytes is given here.

We probably should have an initialization command to access Stack-Mode similar to mu4e etc. Calling this would initialize the cache, ensure auth token is available etc, and then provide shortcuts to the various commands.

For the future, it will be useful to have a variable which, when incremented, invalidates the current cache. This would be useful in case we decide to change the cache format.

The value would then have to be stored with the cache and checked at cache-get I believe.

This can probably be postponed until the time when we actually change the cache format.

Right now, we search for \n\n to find the end of the headers. This is 'OK' since it works, but it would be far better to use the built-in url-http-end-of-headers.

An example of use can be seen in url-dav.el in the url package.

As indicated in #69, using (site . site) in keyword arguments will only pull the first of meta_site or main_site when retrieving favorites. To retrieve both the keyword must be appended to url.

This may need investigation on stackapps.

Comments are easy, question and answers not so much.

The problem is deciding whichparagraphs NOT to fill.
markdown-match-pre-blocks might help.

When trying to view a question after listing them:

M-x sx-list-questions
RET

I get the following error (with debug-on-error)

Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
  replace-regexp-in-string("
\n" "\n" nil)
  (concat (replace-regexp-in-string "
\n" "\n" (cdr (assoc (quote body_markdown) data))) "\n")
  sx-lto--body(((title . "How to wrap given text around region") (link . "http://emacs.stackexchange.com/questions/3499/how-to-wrap-given-text-around-region") (question_id . 3499) (creation_date . 1415891915) (last_activity_date . 1415903002) (score . 2) (answer_count . 2) (view_count . 32) (is_answered . t) (owner (link . "http://emacs.stackexchange.com/users/318/progo") (display_name . "progo") (user_type . "registered") (user_id . 318) (reputation . 138)) (tags . ["elisp" "text-editing"])))
  (cons (sx-lto--body data) sx-lto--body-src-block)
  (cons (quote :value) (cons (sx-lto--body data) sx-lto--body-src-block))
  (list (quote src-block) (cons (quote :value) (cons (sx-lto--body data) sx-lto--body-src-block)))
  (cons (list (quote src-block) (cons (quote :value) (cons (sx-lto--body data) sx-lto--body-src-block))) (if comments (progn (list (list (quote plain-list) (quote (:type descriptive)) comments)))))
  (let ((comments (mapcar (function sx-lto--comment) (cdr (assoc (quote comments) data))))) (cons (list (quote src-block) (cons (quote :value) (cons (sx-lto--body data) sx-lto--body-src-block))) (if comments (progn (list (list (quote plain-list) (quote (:type descriptive)) comments))))))
  sx-lto--question-answer(((title . "How to wrap given text around region") (link . "http://emacs.stackexchange.com/questions/3499/how-to-wrap-given-text-around-region") (question_id . 3499) (creation_date . 1415891915) (last_activity_date . 1415903002) (score . 2) (answer_count . 2) (view_count . 32) (is_answered . t) (owner (link . "http://emacs.stackexchange.com/users/318/progo") (display_name . "progo") (user_type . "registered") (user_id . 318) (reputation . 138)) (tags . ["elisp" "text-editing"])))
  (cons (sx-lto--question-answer data) (mapcar (function sx-lto--answer) (cdr (assoc (quote answers) data))))
  (cons (list (quote :title) (cdr (assoc (quote title) data)) (quote :level) 1 (quote :tags) (mapcar (function identity) (cdr (assoc (quote tags) data)))) (cons (sx-lto--question-answer data) (mapcar (function sx-lto--answer) (cdr (assoc (quote answers) data)))))
  (cons (quote headline) (cons (list (quote :title) (cdr (assoc (quote title) data)) (quote :level) 1 (quote :tags) (mapcar (function identity) (cdr (assoc (quote tags) data)))) (cons (sx-lto--question-answer data) (mapcar (function sx-lto--answer) (cdr (assoc (quote answers) data))))))
  sx-lto--question(((title . "How to wrap given text around region") (link . "http://emacs.stackexchange.com/questions/3499/how-to-wrap-given-text-around-region") (question_id . 3499) (creation_date . 1415891915) (last_activity_date . 1415903002) (score . 2) (answer_count . 2) (view_count . 32) (is_answered . t) (owner (link . "http://emacs.stackexchange.com/users/318/progo") (display_name . "progo") (user_type . "registered") (user_id . 318) (reputation . 138)) (tags . ["elisp" "text-editing"])))
  (org-element-interpret-data (sx-lto--question data))
  (insert (org-element-interpret-data (sx-lto--question data)))
  (save-current-buffer (set-buffer (sx-question--display-buffer window)) (erase-buffer) (insert (org-element-interpret-data (sx-lto--question data))) (org-mode) (show-all) (view-mode) (current-buffer))
  (let ((sx-lto--body-src-block (if sx-question-use-html nil sx-lto--body-src-block)) (inhibit-read-only t)) (save-current-buffer (set-buffer (sx-question--display-buffer window)) (erase-buffer) (insert (org-element-interpret-data (sx-lto--question data))) (org-mode) (show-all) (view-mode) (current-buffer)))
  sx-question--display(((title . "How to wrap given text around region") (link . "http://emacs.stackexchange.com/questions/3499/how-to-wrap-given-text-around-region") (question_id . 3499) (creation_date . 1415891915) (last_activity_date . 1415903002) (score . 2) (answer_count . 2) (view_count . 32) (is_answered . t) (owner (link . "http://emacs.stackexchange.com/users/318/progo") (display_name . "progo") (user_type . "registered") (user_id . 318) (reputation . 138)) (tags . ["elisp" "text-editing"])) #<window 10 on *Backtrace*>)
  sx-question-list-display-question(nil t)
  call-interactively(sx-question-list-display-question nil nil)
  command-execute(sx-question-list-display-question)

Is this a known current issue?

Currently, we are assuming that the user is authenticated as we're adding functionality. This is good, but we need to be mindful of users that will not authenticate. We should provide some means by which to not request authenticated data when we are not authenticated.

Thoughts?

Currently, we're using () because [] get's font-locked by markdown. The way to avoid that would be to use an overlay instead of font-lock-face.

I'd love to have this, but I've no idea how to set it up. If anyone can take the initiative to do this, I'd be very grateful.

This will prove indispensable in the future if/when this project gets mainstream attention.

I kept these under the same issue since they are only trivially different.

In the screenshot below, you can see a few issues with sx-question-mode--fill-and-fontify:

• line breaks after headings should be preserved
• line breaks matching ^\s*\[.+\]:\s+ should be preserved

For this package to be useful, it's imperative that API requests be authenticated. This bumps the the limit on API requests up considerably—from a measly 300 to 10,000.

Currenly, sx-encoding-clean-content is being called whenever content is inserted by the question-list and question-mode.
That's not ideal, as we'll have to keep doing this as we insert more and more stuff.

The ideal way would be for sx-request to go through the entire response it gets and call sx-encoding-clean-content on any string it finds.
This implementation is non-trivial, which is why I haven't put time into it yet.

I believe it is because we do not have a last_editor.

This diff seems to fix the issue:

    Modified   sx-question-mode.el
diff --git a/sx-question-mode.el b/sx-question-mode.el
index 20d3035..2cc95b1 100644
--- a/sx-question-mode.el
+++ b/sx-question-mode.el
@@ -229,7 +229,9 @@ DATA can represent a question or an answer."
         (when .last_edit_date
           (format sx-question-mode-last-edit-format
                   (sx-time-since .last_edit_date)
-                  (sx-question-mode--propertized-display-name .last_editor))))
+                  (when .last_editor
+                    (sx-question-mode--propertized-display-name
+                     .last_editor)))))
        'sx-question-mode-date)
       (when .title
         ;; Tags

This however displays incorrect information; the real reason for the bug should be found.