Support for carrying along external CSV/JSON files about htmltools HOT 22 OPEN

+1 to finding a reasonable solution whatever that may be.

from htmltools.

I think that this could be handled by programmatically creating an htmlDependency. The file would be copied to the lib dir but other than that should work as you are hoping. Does that make sense?

from htmltools.

@jjallaire I'm actually not sure if the JavaScript that needs to get at the CSV/JSON would have a way to know how to get a path to the htmlDependency. Any suggestions?

from htmltools.

Here is a thought. Does it make sense to add dataDependencies akin to htmlDependencies? It could be a vector of paths of data dependencies which would be copied to a data folder, so that a user can access foo.json as data/foo.json. This can be programmatically injected.

from htmltools.

One issue is that we want to keep all of the dependent files in the
_files directory. This is for several reasons:

(1) To make it clear that those supporting files belong with the
corresponding filename

(2) To make it easy to copy the dependencies (just one directory). Windows
Explorer even automatically copies _files when you copy the main file!

(3) For compatibility with the behavior of web browsers when the do a "Save
Web Page"

So it would really need to be _files/data (which I don't think
the JS code could a-priori know to reference). Perhaps the best way to do
this is to just create a JS dependency that includes the JSON data inline
(and uses it to define a variable).

On Thu, Aug 7, 2014 at 12:02 PM, Ramnath Vaidyanathan <
[email protected]> wrote:

Here is a thought. Does it make sense to add dataDependencies akin to
htmlDependencies? It could be a vector of paths of data dependencies
which would be copied to a data folder, so that a user can access foo.json
as data/foo.json. This can be programmatically injected.

—
Reply to this email directly or view it on GitHub
#13 (comment).

from htmltools.

One more thought...

Because the JS code can introspect the DOM, we could add a kind of htmlDependency that writes a <script> to the document with a predictable ID. Your JS code would just need to pull the URL for the dependency from the <script> tag at runtime, e.g.

htmlDependency(name="foo_data", data="foo.csv") 

renders to:

<script id="foo_data" type="text/csv" src="filename_files/foo.csv"/>

and could be referenced in JavaScript:

 var url = document.getElementById("foo_data").src

from htmltools.

@jmcphers This would be really nice if it can be made to work. We can even add a utility function in HTMLWidgets that would do the unpacking on the JS side

window.HTMLWidgets.getDataLink = function(filename){
  return document.getElementById(filename + "_data").src
}

from htmltools.

@jcheng5 @jjallaire @jmcphers Any further thoughts on how such a mechanism can be implemented. I am running into this issue frequently while working with widgets that rely on external data. Any pointers on how to resolves this would be appreciated.

from htmltools.

@jmcphers, would it be pretty easy for us to add this to htmltools?

Note that if possible it would be nice if this could be done without amending the contents of the html_dependency list itself (so we don't run afowl of this check: rstudio/rmarkdown@c5fa018). Note that the check has been eliminated but that was only a few days ago. I'm thinking that the "data" argument could resolve to a script before the html_dependency list is created.

from htmltools.

@jmcphers and @ramnathv, would another approach entirely be to just inline the <script> tag (since we're already generating static html?). This wouldn't use the dependency mechanism at all but rather would be a utility function that does the inlining.

from htmltools.

@jjallaire I was able to work around this by exploiting the fact that htmltools copies the entire folder for a dependency. So for now I am good. But a more generic solution would be useful for widget package authors.

from htmltools.

do you have an example?

from htmltools.

I just realized that this problem is more complicated than I had thought. Note that unlike htmlDependencies which correspond to a widget, the dataDependencies I am talking about are associated with each instance of a widget. So, they are not known apriori.

We might be able to inject them dynamically, but this creates an added complication because in the shiny context, the widgetOutput functions are NOT aware of the widget instance/data and hence these dynamic dataDependencies will not be available.

A simple solution to the problem in my mind is to force users to put any external files they are referring to in their javascript code in a special folder, say www and access it using a simple path. So for example a file www/foo/bar.csv will be accessed as foo/bar.csv. htmltools can be adjusted to always copy contents of the www folder to the temp-folder it copies the index.html file to.

Although restrictive, this is a practice people are familiar with while writing shiny applications. So it should not be hard to adhere to.

from htmltools.

I don't think the www solution will work because (a) htmltools assumes that
all dependencies can be turned into self-contained (i.e. base64 encoded)
assets; and (b) htmltools also assumes that all dependencies can
alternatively be placed in a directory alongside the html file (e.g.
_files) or an arbitrary user-supplied "lib_dir".

So references to dependencies need to be embeddable and aren't addressable
using a directory name known up front. I think the way to beat this is the
idea of an embedded script tag.

On Tue, Aug 12, 2014 at 11:32 PM, Ramnath Vaidyanathan <
[email protected]> wrote:

I just realized that this problem is more complicated than I had thought.
Note that unlike htmlDependencies which correspond to a widget, the
dataDependencies I am talking about are associated with each instance of
a widget. So, they are not known apriori.

We might be able to inject them dynamically, but this creates an added
complication because in the shiny context, the widgetOutput functions are
NOT aware of the widget instance/data and hence these dynamic
dataDependencies will not be available.

A simple solution to the problem in my mind is to force users to put any
external files they are referring to in their javascript code in a special
folder, say www and access it using a simple path. So for example a file
www/foo/bar.csv will be accessed as foo/bar.csv. htmltools can be
adjusted to always copy contents of the www folder to the temp-folder it
copies the index.html file to.

Although restrictive, this is a practice people are familiar with while
writing shiny applications. So it should not be hard to adhere to.

—
Reply to this email directly or view it on GitHub
#13 (comment).

from htmltools.

That makes a lot of sense @jjallaire. Given that pandoc only embeds scripts, styles, images and videos while using the self-contained mode, the only option seems to be turning everything into a script.

We could either do embedded scripts, or create a data.js file on the fly that copies the data and coverts it into a javascript variable that can be accessed by the application.

from htmltools.

I have a use case that expands on this thread (and several other related threads - this seemed like the most appropriate place to continue discussion).

I have an htmlwidget where the associated JS library requires files to be written to disk and these files are later loaded dynamically based the user's interactions with the app (using jsonp). Because of this approach, any kind of dependency attachment or data attachment as has been proposed will be inadequate. These are assets that need to be made available along with the generated index.html, but that make no sense being embedded in the page in script tags because that's not how the JS library that loads the files on demand operates.

Because of this, I need to be able to know the directory of html_print beforehand so I know where to write these files. I want to be able to specify this directory beforehand so that I can both print and potentially modify these files in subsequent R actions prior to viewing.

I made a slight modification to htmltools and htmlwidgets that allows meet this need. Basically, I'm allowing www_dir to be an optional parameter to html_print instead of it being hard coded inside html_print. The default stays the same, and checking is done to make sure it's in the temporary directory. In htmlwidgets, I modified it so you have the option to specify www_dir at the time of creating the widget, and this is then passed to html_print (should probably add in the same temp directory checking here as well). Everything stays completely transparent to any other user, it's just an extra internal option for an htmlwidget author to take advantage of that has a use case like this. Any concerns about this?

The one thing about this use case that might be bothersome is that there's no way to have the standalone option in this case. But that's just how it is and I'm sure there are other JS libraries that will have this same requirement.

Modifications:

master...hafen:feature/customdir

ramnathv/htmlwidgets@master...hafen:feature/customdir

from htmltools.

I think there are a couple of issues to navigate here:

1. I think the dependency needs to make a strong assertion that it's
   incompatible with self_contained: true and that needs to result in a clear
   error message during rendering. This is especially so because
   self_contained is the default behavior.
2. I don't know how this will work with Shiny applications which run in
   read-only directories (e.g. deployed to a server) as those applications are
   not able to write to the filesystem. As a result Shiny "resolves" HTML
   dependencies by calling addResourcePath on the system.file pointed to by
   the dependency. @jcheng5 what are the implications of dependencies that
   must be written to the filesystem.

On Tue, Nov 8, 2016 at 4:50 AM, hafen [email protected] wrote:

I have a use case that expands on this thread (and several other related
threads - this seemed like the most appropriate place to continue
discussion).

I have an htmlwidget where the associated JS library requires files to be
written to disk and these files are later loaded dynamically based the
user's interactions with the app (using jsonp). Because of this approach,
any kind of dependency attachment or data attachment as has been proposed
will be inadequate. These are assets that need to be made available along
with the generated index.html, but that make no sense being embedded in the
page in script tags because that's not how the JS library that loads the
files on demand operates.

Because of this, I need to be able to know the directory of html_print
beforehand so I know where to write these files. I want to be able to
specify this directory beforehand so that I can both print and potentially
modify these files in subsequent R actions prior to viewing.

I made a slight modification to htmltools and htmlwidgets that allows meet
this need. Basically, I'm allowing www_dir to be an optional parameter to
html_print instead of it being hard coded inside html_print. The default
stays the same, and checking is done to make sure it's in the temporary
directory. In htmlwidgets, I modified it so you have the option to specify
www_dir at the time of creating the widget, and this is then passed to
html_print (should probably add in the same temp directory checking here
as well). Everything stays completely transparent to any other user, it's
just an extra internal option for an htmlwidget author to take advantage of
that has a use case like this. Any concerns about this?

The one thing about this use case that might be bothersome is that there's
no way to have the standalone option in this case. But that's just how it
is and I'm sure there are other JS libraries that will have this same
requirement.

Modifications:

master...hafen:feature/customdir
master...hafen:feature/customdir

ramnathv/[email protected]:feature/customdir
ramnathv/htmlwidgets@master...hafen:feature/customdir

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#13 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGXx3Za109Jl3vtVG4FLnz5QAvyOaoNks5q8EX9gaJpZM4CVGuI
.

from htmltools.

Thanks @jjallaire. These are valid issues to consider. However, it appears that self contained is not the default behavior for widget dependencies. This can be verified by creating an htmlwidget (I tried a few package to make I wasn't looking at an exception) and then in the RStudio viewer pane open the inspector to see that there are links in the document head to the widget's dependencies, instead of them being embedded in the page.

Since the path to where these are written seems to vary depending on where you are (terminal, RStudio, R Markdown notebook, etc.), I'm wondering if it would be possible to have a function that registers the output location prior to printing so that I can take advantage of the location and write other data files out. Or, similarly to what has been proposed, a way to attach data files that are to be copied to the widget's directory, but with the option to not embed anything in the html file as a result. This latter approach would be less attractive in my case because I'd rather know the path beforehand and write files once rather than write once and then copy (they can get large).

Anyway, for now, I have found a way to get around this on my end of things without touching htmltools or htmlwidgets, but it would be more robust if I could know where the widget will be written prior to calling print. I'll circle back with some examples / compelling use cases.

from htmltools.

Self contained is the default behavior for R Markdown documents.

The path is never known at print time because the page rendering
environment determines that. In various cases there actually is no _files
directory:

1. In self_contained R Markdown documents
2. In Shiny applications (where widget dependencies are served directly
   from the path they are located via a call to shiny::addResourcePath

So in short "the path where dependencies are written" is not really
something in the scope of htmlwidgets. In some cases (printing at the
console and R Markdown with self_contained: false) there is a _files
directory but in other cases there is no.

I think the ultimate solution here is two-fold:

1. Allow your dependencies to be copied and/or served via
   shiny::addResourcePath; and
2. Document that self_contained must be false in R Markdown

On Fri, Nov 11, 2016 at 8:50 PM, hafen [email protected] wrote:

Thanks @jjallaire https://github.com/jjallaire. These are valid issues
to consider. However, it appears that self contained is not the default
behavior for widget dependencies. This can be verified by creating an
htmlwidget (I tried a few package to make I wasn't looking at an exception)
and then in the RStudio viewer pane open the inspector to see that there
are links in the document head to the widget's dependencies, instead of
them being embedded in the page.

Since the path to where these are written seems to vary depending on where
you are (terminal, RStudio, R Markdown notebook, etc.), I'm wondering if it
would be possible to have a function that registers the output location
prior to printing so that I can take advantage of the location and write
other data files out. Or, similarly to what has been proposed, a way to
attach data files that are to be copied to the widget's directory, but with
the option to not embed anything in the html file as a result. This latter
approach would be less attractive in my case because I'd rather know the
path beforehand and write files once rather than write once and then copy
(they can get large).

Anyway, for now, I have found a way to get around this on my end of things
without touching htmltools or htmlwidgets, but it would be more robust if I
could know where the widget will be written prior to calling print. I'll
circle back with some examples / compelling use cases.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#13 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGXx4EyPfgsdU3VzACPGsrAi-1rKZckks5q9RtngaJpZM4CVGuI
.

from htmltools.

Another use-case for this I'm starting to explore is including WebAssembly (.wasm) files as part of an htmlwidget / shiny app. So keen to get this rolling again if possible?

from htmltools.

@SymbolixAU Can you elaborate on how you are using .wasm files? A concrete example would be super useful.

from htmltools.

admittedly I'm still learning how to use WebAssembly, but an example workflow I want to do is have a C program

extern int foo();

int foo() {
  return 42;
}

int main() { 
  foo();
}

Compile it to wasm

(module
 (table 0 anyfunc)
 (memory $0 1)
 (export "memory" (memory $0))
 (export "foo" (func $foo))
 (export "main" (func $main))
 (func $foo (; 0 ;) (result i32)
  (i32.const 42)
 )
 (func $main (; 1 ;) (result i32)
  (i32.const 0)
 )
)

Then load it into javascript using the WebAssembly api

<script>
WebAssembly.instantiateStreaming(fetch('myprogram.wasm'), importObject)
.then(results => {
  // Do something with the results!
});
</script>

where the fetch() function reads the .wasm file

from htmltools.