Comments (2)
quinnj
commented on July 18, 2024
When I started developing this package, I kind of implicitly optimized for the "well-formed" csv file case so as to be able to focus on performance. That's part of the reason there's not as many "warnings" and such. Happy to accept PRs that improve things without sacrificing performance. In my mind, I'd like to have a single CSV.isvalid(file) that could do some thorough checking and give some great information about what might be wrong. without needing to cram it all into CSV.read.
from csv.jl.
quinnj
commented on July 18, 2024
We now have the CSV.validate(file) function which takes the same arguments as CSV.read, but will give informative error information for improperly formatted csv data.
For these two examples:
julia> io = IOBuffer("""A,B,C
1,1,10
6,1""")
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=16, maxsize=Inf, ptr=1, mark=-1)
julia> CSV.validate(io)
ERROR: CSV.ExpectedMoreColumnsError("row=2, col=2: expected 3 columns, parsed 2, but parsing encountered unexpected end-of-file (EOF); parsed row: '6,1'")
Stacktrace:
[1] validate(::CSV.Source{Base.GenericIOBuffer{Array{UInt8,1}},Nulls.Null}) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:26
[2] #validate#40(::Bool, ::Dict{Int64,Function}, ::Array{Any,1}, ::Function, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Type{T} where T) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:38
[3] validate(::Base.GenericIOBuffer{Array{UInt8,1}}) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:38
julia> io = IOBuffer("""A;B;C
1,1,10
2,0,16""")
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=19, maxsize=Inf, ptr=1, mark=-1)
julia> CSV.validate(io)
ERROR: CSV.TooManyColumnsError("row=1, col=1: expected 1 columns then a newline or EOF, but parsing encountered another delimiter: ','; parsed row: '1'")
Stacktrace:
[1] validate(::CSV.Source{Base.GenericIOBuffer{Array{UInt8,1}},Nulls.Null}) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:30
[2] #validate#40(::Bool, ::Dict{Int64,Function}, ::Array{Any,1}, ::Function, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Type{T} where T) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:38
[3] validate(::Base.GenericIOBuffer{Array{UInt8,1}}) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:38
julia> io = IOBuffer("""A;B;C
1,1,10
2,0,16""")
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=19, maxsize=Inf, ptr=1, mark=-1)
julia> CSV.validate(io; delim=';')
ERROR: CSV.ExpectedMoreColumnsError("row=1, col=1: expected 3 columns, parsed 1, but parsing encountered unexpected newline; parsed row: '1,1,10'")
Stacktrace:
[1] validate(::CSV.Source{Base.GenericIOBuffer{Array{UInt8,1}},Nulls.Null}) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:26
[2] #validate#40(::Bool, ::Dict{Int64,Function}, ::Array{Any,1}, ::Function, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Type{T} where T) at /Users/jacobquinn/.julia/v0.7/CSV/src/validate.jl:38
[3] (::getfield(CSV, Symbol("#kw##validate")))(::Array{Any,1}, ::typeof(CSV.validate), ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Type{T} where T) at ./<missing>:0 (repeats 2 times)
from csv.jl.
Related Issues (20)
- Keyword `decimal` not respected for AbstractFloats in CSV.write()
- Can't transfer CSV.jl v0.10.11 from Windows to Linux HOT 2
- CSV.write somehow cannot write file with name `con.csv` in Windows?! HOT 5
- Add Zenodo badge to README HOT 6
- Segfault on Julia 1.9 on Intel Sapphire Rapids during precompilation
- `bufsize` of `write` is defined to be length of row but actually cells
- can not read the csv with large cells written by itself HOT 1
- Formatting broken on Examples page in documentation HOT 2
- CSV.jl fails to precompile on Ubuntu server, v0.10.5 and up. HOT 2
- Error on CSV.read attempt HOT 4
- `emptyvalue` keyword option
- CSV.Chunks splits file into uneven chunks
- CSV.jl errors on nightly
- Incorrect results for `argmax` with multithreaded parsing HOT 1
- CSV is failing PkgEval HOT 4
- Error when combining single row with multiple row CSV file into a DataFrame with pooling on. HOT 1
- `Date` types should not be inferred from column
- CSV is broken in nightly julia
- 1.12.0-DEV.317 ERROR: LoadError: TypeError: in typeassert, expected Tuple{Vector{UInt8}, Int64, Int64, Union{Nothing, String}}, got a value of type Tuple{Memory{UInt8}, Int64, Int64, Nothing}
- Error when passing as `source` a vector with fewer unique elements than files.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from csv.jl.