Comments (4)
I'm sorry I wasn't very clear about the pdf. It didn't have a place to put a note next to the pdf link, so I put it right following it in the next section. And I will include the page number next time as well.
I will try the stream option. I don't understand exactly what it's doing, but I'll let you know if that helps. That last table looks like exactly what I want.
Thanks!
from tabula-py.
I tried using the stream=True option for the whole pdf (meaning I used pages='all'), and it seems to work for the other tables as well. I haven't checked all 557, but with some decent spot checking, it looks like it worked. Thanks for your help!
from tabula-py.
@dstone42 Next time, could you point to a specific PDF and page?
Looking at the result you've shared, it contains the first 7 lines as another table. This is because tabula-java's table detection algorithm, and the only option you can avoid would be setting area
option. Generally, PDF doesn't have table notation, so some detection failure may happen.
>>> dfs = tabula.read_pdf("HINTS 5 Cycle 4 Public Codebook.pdf", pages=27)
>>> dfs[0]
IGHSPANLI: High linguistically isolated strata (‘Census tracts in which 30% of the households have no adults over the age of 14 that report speaking English
0 ery well’)
1 ariable Name: HIGHSPANLI
2 ariable Label: High linguistically isolated st...
3 ariable Format: HIGHSPAN
4 riteria to receive Question: N/A
5 riteria description: N/A
6 ack to Table of Contents
7 NaN
8 NaN
9 NaN
>>> dfs[1]
HIGHSPANLI Value\rLabel Unweighted\rSample\rSize ... Unnamed: 11 Unnamed: 12 Unnamed: 13
0 NaN NaN NaN ... NaN NaN NaN
1 NaN NaN NaN ... NaN NaN NaN
2 NaN HIGHSPANLI Label ... NaN NaN NaN
3 NaN 1 NaN ... NaN 7.6 NaN
4 NaN 2 NaN ... NaN 92.4 NaN
[5 rows x 21 columns]
When I tried stream=True
option, it somewhat ignored the first 7 lines.
>>> dfs = tabula.read_pdf("HINTS 5 Cycle 4 Public Codebook.pdf", pages=27, stream=True)
>>> dfs[0]
Unnamed: 0 Unnamed: 1 Unnamed: 2 Unnamed: 3 Cumulative Weighted Unnamed: 4
0 NaN NaN Unweighted NaN Unweighted Sample Weighted
1 NaN Value Sample Unweighted Sample Size Percent
2 HIGHSPANLI Label Size Percent Size (Estimated) (Estimated)
3 1 Yes 347 9 347 19,266,519 7.6
4 2 No 3,518 91 3,865 234,548,678 92.4
Anyway, it is a tabula-java limitation, so I don't know how to avoid it other than using area
option page by page.
from tabula-py.
Actually, I did some random tweaking for parameters, and I don't know why the streaming option works well. Anyway, it's not a bug but tabula-java's behavior. Setting an explicit area would be the last resort.
Other than that, I can help nothing, unfortunately.
from tabula-py.
Related Issues (20)
- Unable to remove note in log : Got stderr: Picked up _JAVA_OPTIONS: -Djava.awt.headless=true HOT 1
- Tabula py Ignores an entire column if it's blank and if it does not contain headerd? HOT 1
- tabula-py CalledProcessError: Command '['java', '-Dfile.encoding=UTF8', '-jar', HOT 3
- dont ignore empty columns in tables spanning multiple pages HOT 1
- Try to install tabula-py HOT 1
- Use JPype instead of subprocess HOT 11
- Add a way to set areas for non-existent pages in template HOT 4
- Exception: RuntimeError: java.lang.UnsatisfiedLinkError: HOT 2
- cant install tabula-py on m1 mac vscode. HOT 1
- Support Python 3.12 HOT 5
- Pls add "orientation" parameter to read_pdf HOT 4
- Security vulnerability in tabula-1.0.5-jar-with-dependencies.jar HOT 4
- [BUG] Encoding still being overridden even after fix to #371. HOT 5
- FutureWarning: errors='ignore' is deprecated and will raise in a future version. HOT 3
- [BUG] issue just running sample code HOT 1
- Table detection in images HOT 1
- [BUG] <FutureWarning: errors='ignore' > HOT 3
- [BUG] Error importing jpype dependencies. Fallback to subprocess. No module named 'org.apache' HOT 1
- [BUG] column parameter of read_pdf currently needs to be list, not generic iterable HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tabula-py.