Comments (2)
juliohm
commented on August 17, 2024
1
I got it. If the type is correctly inferred when the column is accessed directly, that shouldn't be an issue. Thanks for the quick reply.
from dbftables.jl.
visr
commented on August 17, 2024
The Schema is currently calculated from the header only, so there we don't know yet if missings are present in the table. As far as I know the only way to find this out is by going over all data, which doesn't seem like a very attractive option. How does this issue affect you?
Using the test DBF, it indeed shows all types as possibly missing:
julia> dbf = DBFTables.Table("test.dbf")
DBFTables.Table with 7 rows and 6 columns
Tables.Schema:
:CHAR Union{Missing, String}
:DATE Union{Missing, String}
:BOOL Union{Missing, Bool}
:FLOAT Union{Missing, Float64}
:NUMERIC Union{Missing, Float64}
:INTEGER Union{Missing, Int64}In this case all columns except DATE have missing values:
julia> DataFrame(dbf)
7×6 DataFrame
Row │ CHAR DATE BOOL FLOAT NUMERIC INTEGER
│ String? String Bool? Float64? Float64? Int64?
─────┼──────────────────────────────────────────────────────────────────────────
1 │ Bob 19900102 false 10.21 11.21 100
2 │ John 20010203 true 100.99 12.21 101
3 │ Bill 20100304 false 0.0 13.21 102
4 │ missing 19700101 missing 0.0 0.0 0
5 │ missing 19700101 true missing 1.11111e9 2222222222
6 │ missing 19700101 true 3.33333e9 missing 4444444444
7 │ missing 19700101 true 5.55556e9 6.66667e9 missingHowever when actually getting the data, the copied vector is correctly narrowed:
julia> dbf.DATE
7-element Vector{String}:
"19900102"
"20010203"
"20100304"
"19700101"
"19700101"
"19700101"
"19700101"from dbftables.jl.
Related Issues (11)
- Adding the Package HOT 2
- TagBot trigger issue HOT 12
- release 1.0
- Write DBF files from a Table
- Use correct string encodings HOT 4
- Handle Integers correctly HOT 7
- Assumption that field names are padded with \0 not always true HOT 1
- Header length is not considered when loading DBF; and possibly incorrect when saving it
- Package compatibility caps HOT 1
- switch from DataFrames to Tables HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbftables.jl.