mikedilger / formdata Goto Github PK
View Code? Open in Web Editor NEWParsing of multipart/form-data
License: MIT License
Parsing of multipart/form-data
License: MIT License
I think it is happening when the buffer drains and refills, but I'm not sure yet.
I'm working on it.
rustc 1.11.0 (9b21dcd6a 2016-08-15)
cargo 0.12.0-nightly (6b98d1f 2016-07-04)
Message:
Compiling textnonce v0.3.4
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20:35: 20:48 error: mismatched types [E0308]
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20:35: 20:48 help: runrustc --explain E0308
to see a detailed explanation
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20:35: 20:48 note: expected type&mut syntex::Registry
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20:35: 20:48 note: found type&mut syntex::Registry
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20:35: 20:48 note: Perhaps two different versions of cratesyntex
are being used?
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/formdata-0.8.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 error: mismatched types [E0308]
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 help: runrustc --explain E0308
to see a detailed explanation
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: expected type&mut syntex::Registry
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: found type&mut syntex::Registry
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: Perhaps two different versions of cratesyntex
are being used?
/home/evgeny/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
error: aborting due to previous error
error: aborting due to previous error
Build failed, waiting for other jobs to finish...
error: Could not compiletextnonce
.
This is where a single multipart section is itself a multipart section. This happens when a file input field is filled with multiple uploaded files.
Strictly speaking, Content-Disposition headers could have their attribute values encoded according to RFC 2231. Most of this was intended for E-Mail headers, not HTTP multipart/form-data headers. However, it's possible.
I've run tests with long filenames with unicode characters, and even ie6 doesn't do any encoding, and the characters come through to UTF-8 just fine.
So I'm opening and then closing this issue.
syntex = { version = "^0", optional = true }
Maybe a more strict requirement?
Error description:
expected `&mut syntex::Registry`,
found `&mut inner::syntex::Registry`
(expected struct `syntex::Registry`,
found struct `inner::syntex::Registry`) [E0308]
This crate works fine with hyper and iron. But it is not sufficient for next-generation non-blocking asynchronous I/O based servers.
Currently there are two places where name
exists within the files array: (1) the first string in the pair, and (2) the name
in the ContentDisposition
header. filename
OTOH is only in the ContentDisposition
header (and thus more difficult to access for readers).
Additionally, when writing out a FormData
, the filename for a FilePart
is taken from the path of that part. But the consumer of the library might want to specify a different filename than the actual one on disk.
I suggest fixing these problems in the following way:
filename
in the FilePart
(this change is required in mime-multipart
)FilePart.filename
the canonical filename.File structure content-type is filled in with Text/Plain at present. We should use the content-type header, if it was provided.
Several occurrences of:
1082:20 error: use of unstable library feature 'core_intrinsics': intrinsics are unlikely to ever be stabilized, instead they should be used through stabilized interfaces in the rest of the standard library (see issue #0)
/formdata-10625c5507708aaf/out/lib.rs:1082 #[derive_PartialEq]
When adding the feature you get an error in textnonce
:
Compiling textnonce v0.3.4
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 error: mismatched types [E0308]
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 help: run `rustc --explain E0308` to see a detailed explanation
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: expected type `&mut syntex::Registry`
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: found type `&mut syntex::Registry`
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20:35: 20:48 note: Perhaps two different versions of crate `syntex` are being used?
/Users/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/textnonce-0.3.4/build.rs:20 ::serde_codegen::register(&mut registry);
^~~~~~~~~~~~~
Using:
rustc 1.11.0-nightly (5522e678b 2016-06-20)
cargo 0.12.0-nightly (4ba8264 2016-06-19)
BufReadPlus::read_until_token() will not perform properly if the token is larger than the underlying buffer capacity, whenever the token spans three or more buffer read segments. E.g., when we find a partial, then we find more partial, then we find the end of the token.
This could be fixed without too much code upheaval, with an extension of the partial concept.
Medium priority, as this is now less likely (the fully-sized stream buffer would actually have to be smaller than the token, not just the remaining stream buffer).
Hello! I don't understand can i build this code https://github.com/sinitcin/microservice
error[E0599]: no method named `deconstruct` found for type `hyper::Request<()>` in the current scope
--> src/post_multipart/post_multipart.rs:30:63
|
30 | let (_, _, headers, _, _, mut reader) = hyper_request.deconstruct();
| ^^^^^^^^^^^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0599`.
error: Could not compile `microservice`.
Please help me =(
Have you considered or would you be open to making this an Iron plugin? It seems like one of the major use cases for this would be with an Iron app, and it would be great to be able to do something like
fn handle(&self, req: &mut Request) -> IronResult {
let form_data = req.get_ref::<FormData>();
// etc
}
Whether the plugin code be in this repo or another that depends on this, Iron doesn't have any official multipart/form-data support yet that I've found, and it would be awesome to see this as a plugin. (Coming from someone writing an app that's going to deal with lots of file uploading)
Among others, replace push_all() with extend()
The nuget client (which I am programming a server for) is sending a request which looks like this:
-----------------------------8d31b628b7d7b04
Content-Disposition: form-data; name="package"; filename="package"
Content-Type: application/octet-stream
PK��
�i9G� T�����
/// File Contens
P�����f
[Content_Types].xmlPK����
-----------------------------8d31b628b7d7b04--
And is causing an Error::Eof.
Updated, see below
formdata-0.8.1/build.rs:15:33: 15:46 error: mismatched types:
expected `&mut syntex::Registry`,
found `&mut inner::syntex::Registry`
(expected struct `syntex::Registry`,
found struct `inner::syntex::Registry`) [E0308]
...once serde-rs/serde#196 is addressed.
This issue was automatically generated. Feel free to close without ceremony if
you do not agree with re-licensing or if it is not possible for other reasons.
Respond to @cmr with any questions or concerns, or pop over to
#rust-offtopic
on IRC to discuss.
You're receiving this because someone (perhaps the project maintainer)
published a crates.io package with the license as "MIT" xor "Apache-2.0" and
the repository field pointing here.
TL;DR the Rust ecosystem is largely Apache-2.0. Being available under that
license is good for interoperation. The MIT license as an add-on can be nice
for GPLv2 projects to use your code.
The MIT license requires reproducing countless copies of the same copyright
header with different names in the copyright field, for every MIT library in
use. The Apache license does not have this drawback. However, this is not the
primary motivation for me creating these issues. The Apache license also has
protections from patent trolls and an explicit contribution licensing clause.
However, the Apache license is incompatible with GPLv2. This is why Rust is
dual-licensed as MIT/Apache (the "primary" license being Apache, MIT only for
GPLv2 compat), and doing so would be wise for this project. This also makes
this crate suitable for inclusion and unrestricted sharing in the Rust
standard distribution and other projects using dual MIT/Apache, such as my
personal ulterior motive, the Robigalia project.
Some ask, "Does this really apply to binary redistributions? Does MIT really
require reproducing the whole thing?" I'm not a lawyer, and I can't give legal
advice, but some Google Android apps include open source attributions using
this interpretation. Others also agree with
it.
But, again, the copyright notice redistribution is not the primary motivation
for the dual-licensing. It's stronger protections to licensees and better
interoperation with the wider Rust ecosystem.
To do this, get explicit approval from each contributor of copyrightable work
(as not all contributions qualify for copyright, due to not being a "creative
work", e.g. a typo fix) and then add the following to your README:
## License
Licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions.
and in your license headers, if you have them, use the following boilerplate
(based on that used in Rust):
// Copyright 2016 formdata Developers
//
// Licensed under the Apache License, Version 2.0, <LICENSE-APACHE or
// http://apache.org/licenses/LICENSE-2.0> or the MIT license <LICENSE-MIT or
// http://opensource.org/licenses/MIT>, at your option. This file may not be
// copied, modified, or distributed except according to those terms.
It's commonly asked whether license headers are required. I'm not comfortable
making an official recommendation either way, but the Apache license
recommends it in their appendix on how to use the license.
Be sure to add the relevant LICENSE-{MIT,APACHE}
files. You can copy these
from the Rust repo for a plain-text
version.
And don't forget to update the license
metadata in your Cargo.toml
to:
license = "MIT OR Apache-2.0"
I'll be going through projects which agree to be relicensed and have approval
by the necessary contributors and doing this changes, so feel free to leave
the heavy lifting to me!
To agree to relicensing, comment with :
I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.
Or, if you're a contributor, you can check the box in this repo next to your
name. My scripts will pick this exact phrase up and check your checkbox, but
I'll come through and manually review this issue later as well.
hey there,
I'm actually looking to do multipart parsing via stdin and don't need the hyper service.
Just wanted to let you know the example does not compile today using hyper 0.4.14.
using the Read Example
error[E0432]: unresolved imports `hyper::server::Request`, `hyper::server::Response`
--> examples\formdata.rs:5:29
|
5 | use hyper::server::{Server, Request, Response};
| ^^^^^^^ ^^^^^^^^ no `Response` in `server`
| |
| no `Request` in `server`
|
= help: consider importing one of these items instead:
httparse::Request
hyper::Request
std::error::Request
core::error::Request
= help: consider importing one of these items instead:
httparse::Response
hyper::Response
error[E0599]: no function or associated item named `http` found for struct `Server` in the current scope
--> examples\formdata.rs:8:24
|
8 | let server = Server::http("0.0.0.0:0").unwrap().handle(handler).unwrap();
| ^^^^ function or associated item not found in `Server<_, _, _>`
BufReadPlus::read_until_token() will not perform properly if the token is larger than the underlying buffer capacity.
This could be fixed without much code upheaval, with an extension of the partial
concept.
@mikedilger We were wanting to add formdata support for Servo(servo/servo#7553). I went through the repository but couldn't confirm if this parser conforms to any spec. It'd be great if you could help us in this regard.
Currently our filename is simply an Option<String>
. A recent commit ensures any specified charset is taken into account and the string in that charset is decoded into utf-8. The language tag is discarded.
Two problems with this:
So the Filename structure will look like:
pub struct Filename {
pub charset: Charset,
pub language_tag: Option<LanguageTag>,
pub bytes: Vec<u8>,
}
and we may want to implement functions which decode the bytes into utf-8 according to the charset as our current code does in charset_decode().
However, this work depends on LanguageTag implementing serde Serialize/Deserialize first, so our UploadedFile struct retains its ability to serialize/deserialize.
For example, a token '---boundary' might be preceded with more hyphens in which case multiple partial offsets could indicate the token. And '--' at the end of a buffer might be followed by '--boundary' in the next buffer in which case the 2nd hyphen was the partial (but our code only checked one partial possibility).
Currently we have only variable processing, file processing is commented out. It needs implementation.
Error enum may have more errors than we presently use, it needs a rethink. Also as a library, it should implement std::error::Error.
Hello! Can you please push the remaining tags? In case you dont have them all, you can use one of the following.
Either:
bash -c 'IFS=$\'\n\'; for line in `git log --oneline | grep \'\b0\.[1-9]\.[0-9]\+$\'`; do git tag "$(echo $line | cut -d\ -f2)" "$(echo $line | cut -d\ -f1)"; done'
Or:
git tag 0.8.4 a7238ee
git tag 0.8.3 78aa038
git tag 0.8.2 701e7ee
git tag 0.8.1 54860f9
git tag 0.8.0 564dd08
git tag 0.7.10 14de762
git tag 0.7.9 7897552
git tag 0.7.8 51a1308
git tag 0.7.7 1da9137
git tag 0.7.6 f02f77b
git tag 0.7.5 663a061
git tag 0.7.4 2a88b2e
git tag 0.7.3 59f6686
git tag 0.7.2 9e4ccaf
git tag 0.7.1 dec1c4a
git tag 0.7.0 1558e66
git tag 0.6.1 fe1251d
git tag 0.6.0 36ffca0
git tag 0.5.0 db93ea8
git tag 0.4.0 5ef4a32
git tag 0.3.1 5f598d0
git tag 0.3.0 4e52d39
git tag 0.2.1 bb81d97
git tag 0.2.0 7b54f09
Thanks in advance! 😄
Hi @mikedilger !
Thanks again for your library. It's quite nice 👍
I was reading the RFC on application/form-data and saw no mention of Content-Disposition: file
, which your library does seem to parse. The RFC says everything has to be Content-Disposition: form-data
.
But I did see a mention of Content-Disposition: file
in an old HTML spec. Do you have any more information about that ?
Would love to know more, if you have the time 😃
Thanks,
Conrad
Error:
Compiling textnonce v0.2.4
/home/vagrant/rust_projects/hastily.rs/target/debug/build/textnonce-b46a6516de072497/out/lib.rs:97:26: 98:98 error: no method named deserialize_newtype_struct
found for type &mut __D
in the current scope
Cargo.toml contents
[package]
name = "hastily"
version = "0.1.0"
authors = ["vagrant"]
[dependencies]
hyper = "0.7.2"
rustc-serialize = "0.3"
xml-rs = "0.3"
zip = "0.1"
url = "*"
regex = "0.1"
formdata = { git = "https://github.com/mikedilger/formdata" }
Release a version supporting Hyper 0.10. I'll see if that's an quick thing enough to do, I'll send a PR.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.