Comments (2)
Hi, we did on the branch, just wait till the deployment is done
The configuration is the following:
`"""
Basic example of scraping pipeline using SmartScraper
"""
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai.utils import prettify_exec_info
************************************************
Define the configuration for the graph
************************************************
graph_config = {
"llm": {
"model": "ollama/gemma",
"temperature": 0,
"format": "json", # Ollama needs the format to be specified explicitly
# "model_tokens": 2000, # set context length arbitrarily,
"base_url": "http://localhost:11434", # set ollama URL arbitrarily
},
"embeddings": {
"model": "ollama/nomic-embed-text",
"temperature": 0,
"base_url": "http://localhost:11434", # set ollama URL arbitrarily
}
}
************************************************
Create the SmartScraperGraph instance and run it
************************************************
smart_scraper_graph = SmartScraperGraph(
prompt="List me all the news with their description.",
# also accepts a string with the already downloaded HTML code
source="https://perinim.github.io/projects",
config=graph_config
)
result = smart_scraper_graph.run()
print(result)
************************************************
Get graph execution info
************************************************
graph_exec_info = smart_scraper_graph.get_execution_info()
print(prettify_exec_info(graph_exec_info))`, When we make the next deploy you can use it, just wait
![Screenshot 2024-04-29 alle 11 05 35](https://private-user-images.githubusercontent.com/88108002/326381889-22324503-6b28-44de-b522-b20cf8cfb416.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk0MTI2MDQsIm5iZiI6MTcxOTQxMjMwNCwicGF0aCI6Ii84ODEwODAwMi8zMjYzODE4ODktMjIzMjQ1MDMtNmIyOC00NGRlLWI1MjItYjIwY2Y4Y2ZiNDE2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI2VDE0MzE0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTM2YWQ5NTNkYzBhNjZhMjUzMjRkNDllNWFkNjc1NDdmMzg5YTYwZGE0YTkyZjM4MmJjNTRkM2QyMzhkN2U2NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.OAVa_bejcSYCafRHD-TBIPCmh5CcUlDIT4zLuxK8moM)
from scrapegraph-ai.
@VinciGit00 Thank for the prompt response. When will be the next deployment or version release?
if you are referring to scrapegraphai==0.4.1
this version still have the issue.
from scrapegraph-ai.
Related Issues (20)
- TypeError AsyncChromiumLoader.__init__() got an unexpected keyword argument 'headless' HOT 2
- openai.AuthenticationError: Error code: 401 HOT 11
- Problem running the example notebook in the readme HOT 5
- In headless mode, there is content in the browser, but it prompts that there is no HTML content HOT 1
- Problem with scrapegraphai/graphs/pdf_scraper_graph.py HOT 6
- OpenAI Authentication issue at demo site HOT 3
- make the new search graph
- Support for SPA's that has deferred content loading HOT 2
- TypeError: cannot pickle '_thread.RLock' object
- 'SmartScraperGraph' object has no attribute 'model_token' HOT 2
- Adding message parameter support for OpenAI models HOT 5
- Do we have a output parser to get a certain format output HOT 1
- No HTML body content found when trying FetchNode HOT 3
- Erorr : Model provided by the configuration not supported HOT 4
- Could it support other chinese ollama embedding models? HOT 7
- Switch between search engines
- How could I remove part of page content before sending to LLM? HOT 9
- BedRock Malformed input request: #/texts/0: expected maxLength: 2048, actual: 19882, please reformat your input and try agai HOT 7
- Not able to run Anthropic Claude models. HOT 4
- split unit testing from src HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scrapegraph-ai.