Comments (23)
@jneira thank you for your explanation. I decided to try to use plain responseLBS helper.
A very poor man's version of static file serving that plugs nicely into servant Raw tape:
-- LBS - Data.ByteString.Lazy
-- T - Text
data StaticSettings = StaticSettings FilePath
staticAppPieces :: StaticSettings -> [Text] -> WAI.Application
staticAppPieces (StaticSettings root) pieces req respond =
let filePath = joinPath $ map T.unpack (T.pack root : pieces)
contentType = M.mimeByExt M.defaultMimeMap "application/text" (SF.lastDef "" pieces)
in do
fileExists <- jIsReadableAndExists filePath -- needed to work-around in Java
res <- if fileExists
then do
contentBody <- LBS.readFile filePath
contentLength <- jFileSize filePath -- needed to work-around in Java
respond $ W.responseLBS H.status200
[("Content-Type", contentType),
("Content-Length", BS.pack $ show $ contentLength)]
contentBody
else
respond $ WAI.responseLBS H.status404
[("Content-Type", "text/plain")]
"File not found"
return res
Full code: https://github.com/rpeszek/crud-ex-backend-servant/blob/eta-jetty/src-server/Util/WaiStatic.hs
I had to work-around some issues using Java (I will create tickets) and I get exceptions printed when I run it (I will try to narrow down what is causing these) but this does serve files.
Should be not hard to make it a bit better adding some etag and maybe even chunk support.
Not sure how much chunking or using responseStream would stress existing implementation.
Would it make sense to create servant-static-tempfix project for developer testing of servant using this approach?
from eta-hackage.
I am afraid that module was commented in the eta patch for the wai-server package
If i recall correctly i think the cause is the module depends on warp package, and it has a lot of c ffi calls hard to port to java (@rahulmutt could correct me if i am wrong).
Moreover, in the jvm web ecosystem app servers are not frequently used to serve static files but dynamic content.
Anyway you can check a servant api working example for eta which uses a eta wrapper for wai.
from eta-hackage.
Actually, the issue is not even warp
. Turns out warp
is actually not used by servant
anywhere but they have it added as a dependency! The issue is file-embed
that use TemplateHaskell to covert static files to bytestring that are embedded into the executable. If we did a direct translation of this to Eta, it would be the equivalent of wrapping files into string constants in classfiles which doesn't seem like a good idea to me.
So the best solution is to use the JVM's concept of resources. Patch file-embed
to load bytestrings from the resources in the jar vs at compile-time.
from eta-hackage.
Thanks for the info. I use it to serve static content like CSS and JS files.
If converting over something like serveDirectoryFileServer is hard, it can be worked around for now.
In many deployments static files are often served from a separate place anyway.
I will remove my github example as it is useless anyway.
from eta-hackage.
Thanks for the clarification @rahulmutt, It seems servant uses warp only for an example in servant-server and for testing in servant-client. It uses wai-app-static too, and it uses warp to get a static file server executable. We could use only the lib part of it, but we would have to implement ResponseFile in wai-servlet (tracked in this issue)
I dont know it it worths the time to implement cause in java world static files usually are served by the http server directly cause to use the j2ee servlet container has a performance penalty.
from eta-hackage.
@jneira @rahulmutt there is some utility in providing functionality of serving static files just for developer convenience even if it is not very fast and/or is part of a separate patch library.
There are things like asset pipeline in grails, so it is not completely true that Java world does not serve static files (but I do see the argument).
I will need to implement some simple workaround for now to convert my pet project. If there is no better solution by the time I finish this, it could serve as a temporary hack-patch?
I just have seen the commented out stuff in servant-server patch. Very slowly learning hoops.
from eta-hackage.
@rpeszek Temporary patches that just implement the functionality are more than welcome.
from eta-hackage.
@jneira could you clarify your comment on the ResponseFile handling issue you have linked (jneira/wai-servlet#1). Is it about etagging only?
Is ResponseFile handling just performance related or am I missing something subtle here?
Native type-checking tells me that the core of what is missing is (and that matches what you have indicated): Network.Wai.Application.Static (staticAppPieces, defaultFileServerSettings)
From there I see garden path to serveDirectoryFileServer (things compile minus minor stuff like Posix EpochTime).
I do not understand staticAppPieces etag logic well, it has some, but it seems to me that this function could fully encapsulate etagging when serving static file content.
Am I off base here?
There is lots of OS specific stuff here so I do see reasoning to not convert it.
I am treating this a bit as eta-lang learn how to swim. Thank you for your time answering my questions.
from eta-hackage.
After a more carefully reading of wai-static
it turns out that Network.Wai.Application.Static
depends on the modules under WaiAppsStatic. Leaving aside the directory handling , there are two options to serve files content:
- FileSystem
- It outputs at the end a wai
ResponseFile
which is not implemented in wai-servlet. ResponseFile is a type of response in wai with a specific handling of http verbs, headers and body suited to serve files from the file system, for performace reasons and convenience for client apps.serveDirectoryFileServer
andserveDirectoryWebApp
inServant.Utils.StaticFiles
use this way to serve the files content
- It uses
System.PosixCompat.Files
to retrieve info from the file system and create theResponseFile
, we could have to investigate ifSystem.PosixCompat
works in eta without too much modifications.
- It outputs at the end a wai
- Embedded.
- It serves files in memory, and (as @rahulmutt pointed out) it can use Template Haskell to embed files in the executable at compile time. However i think maybe you could use it without template haskell through the basic embeddedSettings.
- And we could patch the template haskell part as suggested by @rahulmutt
- It does not use ResponseFile cause it writes directly the bytes from memory file using ResponseBuilder which is implemented in wai-servlet.
- Used by
serveDirectoryEmbedded
inServant.Utils.StaticFiles
without using the template haskell part if i am not wrong
- It serves files in memory, and (as @rahulmutt pointed out) it can use Template Haskell to embed files in the executable at compile time. However i think maybe you could use it without template haskell through the basic embeddedSettings.
Imo we could uncomment all the code but CommandLine and the Template Haskell part and i think we would try to make the package usable (taking in account the os/posix stuff) . If you want to serve files from the file system we'll have to implement ResponseFile
handling in wai-servlet.
I hope my resume helps to understand the issue.
from eta-hackage.
Trying to install a minimal functional version of wai-app-static
i found some packages which we'll to do eta compatible to use FileSystem:
- cryptonite: module
Crypto.Hash
for calculate hashes. - unix-compat: module
System.PosixCompat.Files
for retrieve info from files. - memory: module
Data.ByteArray.Encoding
to convert to Base64.
Link to the cabal file: https://gist.github.com/jneira/99cef71f79ec61a1134872476eb8d237
Another options would be replace the calls for those packages (that are deeply linked with the os ans c libs) with direct java calls in wai-app-static, as a temporary workaround (?)
from eta-hackage.
Make usable the FileSystem way to handle files will need some work, replacing s.o. calls for java ones and implementing ResponseFile handling in wai-servlet
@rpeszek: if you can use the Embedded option through serveDirectoryEmbedded
in Servant.Utils.StaticFiles
, providing a list of paths/bytestrings with the files content, we could try to make it usable. I think it will not need so much work.
from eta-hackage.
A very poor man's version of static file serving that plugs nicely into servant
Good enough for now!
Would it make sense to create servant-static-tempfix project for developer testing of servant using this approach?
I think the code could be used to patch wai-servlet-static for eta. This is the current standard way in eta for adapt haskell packages. Etlas applies patches from that repo automatically when it installs the package.
In this way you could use directly servant-static without further ado.
from eta-hackage.
@jneira I was away and then absorbed by project work. Sorry for not replying.
2 weeks ago, that code was suffering from #54 (and hopefully no other issues) and I did not have time to test the fix for 54 yet.
EDITED: With latest version of master this code is working fine. I found one more issue which is not relevant to static file serving and I will report on separately (things can hang at startup if several requests to server are issued at once).
from eta-hackage.
@jneira I am finding eta-hackage/servant-server which could be patched by uncommenting parts
of Servant.Utils.StaticFiles module
and I could (EDITED) either patch servant-server or add partially implemented wai-app-static-3.1.6.1 to eta-hackage using the above hack in place of original implementation of staticAppPieces.
EDITED: Since this code is a temporary hack and since it is only a small part of wai-app-static I will try to patch servant-server only. That seems like the most prudent approach.
I am no finding wai-servlet-static in either eta-hackage or etlas-index repos. So I assume this is a typo.
from eta-hackage.
@rpeszek Things hanging up seems like a bug to report. I've been investigating a lot of these hanging bugs recently so please do report them.
wai-servlet-static
doesn't exist as far as I know.
from eta-hackage.
Ooops sorry, i wanted to mean wai-app-static. Patch servant-server is a good option too.
from eta-hackage.
@rahulmutt I created typelead/eta#520 listing various findings.
I cannot be sure that some of these are not related to my WAI.Application logic serving static files. (probably unlikely.)
I will wait with adding this to servant-server patch just in case.
from eta-hackage.
@rpeszek Yes, it does look like it's a result of my recent runtime changes. I'll take a look asap.
from eta-hackage.
@jneira @rahulmutt
Since the deadlock issues appear to have nothing to do with my code, I have created pull request for the patch. #58 with the temporary hack outlined above.
Note that we are missing servant-server-0.10 patches.
I have not tried to patch servant-server-0.10, trying to just apply 0.11 patch to 0.10 does not work so a bit more work would be needed to have 0.10.
from eta-hackage.
This is a great start thanks! The next step is to add the ETag stuff so that the browser doesn't have to download unchanged static content over and over again and should return 304 if the context is unchanged.
from eta-hackage.
It was fun working on this.
Supporting large files using chunks would be another important step. Or maybe instead of continuing piecemeal approach we just implement ResponseFile (as pointed out by @jneira).
I am out of commission for a week or a bit more.
from eta-hackage.
Hi, i've been workin in a version of wai-servlet that supports ResponseFile. It is not fully tested, but it seems that it works for the simple cases, caching files which have not been changed.
The next step would be patch wai-app-static to get a version that works with the FileSystem option as commented above.
from eta-hackage.
Sounds good! Great to see progress on this.
from eta-hackage.
Related Issues (20)
- dhall-to-etlas doesn't build due to ansi-terminal dependency HOT 2
- Issue with compiling language-javascript HOT 1
- Patch cborg-0.2.0.0 HOT 1
- cryptohash-md5 and cryptohash-sha1 large input test HOT 1
- Patch for cql-io HOT 1
- lens-aeson examples throw IllegalStateExceptions HOT 7
- Patch for pandoc 2.5 HOT 2
- failure on Text.Regex.mkRegex HOT 3
- Patch needed to install Reflex? Dependency on haskell-src-exts-1.20.3 HOT 2
- Get Duckling to work HOT 9
- "regex-compat" package behavior is inconsistent with GHCi
- Temporary library -- creation of temp files does not work on windows platform HOT 3
- request: Patch pretty-show-1.6.16 HOT 1
- Patch the Cabal library HOT 1
- directory-1.3.1.0 fix createDirectoryIfMissing HOT 4
- Patch megaparsec package to avoid StackOverflowException's HOT 14
- Patch criterion + support for JMH HOT 12
- Request: patch cryptohash-md5 and cryptohash-sha1 HOT 2
- [Patch Request] MissingH-1.4.0.1 HOT 3
- Patch text-show-3.7.3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eta-hackage.