Comments (4)
Hello,
I doubt that this issue is related to python-goose. However, does this error also occur
when you are running goose?
My guess is that this error is due to permissions and that you don't have
permissions to access the file .../text/stopwords-tr.txt
. However, i'm not entirely sure,
could you please paste the entire stack-trace?
Note:
https://github.com/codelucas/newspaper/blob/master/newspaper/text.py#L67
from newspaper.
Hi @00krishna and @codelucas
I actually ran into the same issue just today and dug a bit deeper.
It turns out that the default configuration extracts the language from the article's source (HTML) and just uses that. So if the source happens to contain <html lang="uk">
the language will be set to uk
-- that was the case for me, anyway. The OutputFormatter
class will try to remove paragraphs with a small amount of words (via remove_fewwords_paragraphs
method) and during that, will try to load in the stopwords file for the particular language extracted from the article's source. In the uk
case, /Users/karl/.virtualenvs/juicer-mk2/lib/python2.7/site-packages/newspaper/utils/../resources/text/stopwords-uk.txt
will be read, which just doesn't exist. Similarly, as @00krishna found out, stopwords-tr.txt
does not exist either.
I'd say it's nobody's fault. The webpage may specify whatever garbage they want as the language and newspaper
will just try to look it up. I think a good solution might be to have mappings from "things that are english" (like uk
, en
, gb
) to just en
, but obviously for all the languages.
from newspaper.
@karls very good idea. I'll add an "english language mapping". Also a good idea to catch the exception of the language code being missing and just log out the error instead of halting all of newspaper.
from newspaper.
This issue has been patched.
Ref: 246ad93
from newspaper.
Related Issues (20)
- Article() not returning anything when fed article links HOT 1
- Where to find and delete all articles? HOT 7
- can't start new thread HOT 2
- the API doesn't work HOT 10
- Is there any optimization in the configuration of newspaper?
- download() halts/stuck forever with a specific URL
- download() halts/stuck forever with a specific URL
- suggestions for calculate_best_node
- Date extraction is faulty HOT 2
- not working for gnews.org
- TIPS FOR FAST IMPROVEMENT
- Project status HOT 4
- Parsing Problem HOT 6
- Consider switching from lxml's clean_html for enhanced security (and possibly performance) HOT 1
- Getting Older News Articles HOT 4
- gnews with user agent returns empty text HOT 1
- Can I get the CSS Selectors ? HOT 2
- TIPS FOR IMPROVEMENT HOT 5
- Good
- ㅇㅇ
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from newspaper.