This project is a REST API service built with Node.js and Express, leveraging Playwright to fetch the final rendered content of a given URL. The service returns the HTML content of the URL, optionally using a proxy server.
- Node.js and npm installed
- Docker installed
-
Clone the repository:
git clone https://github.com/rbehzadan/fetch-url-api.git cd fetch-url-api -
Install the dependencies:
npm install
-
Start the server:
node server.js
-
Send a request to the API:
You can use
curl, Postman, or any HTTP client to send a POST request to the server.curl -X POST http://localhost:8080/fetch \ -H "Content-Type: application/json" \ -d '{"url":"https://example.com", "proxy":"http://proxyserver:port"}'Note: The
proxyfield is optional.
-
Build the Docker image:
docker build -t fetch-url-api . -
Run the Docker container:
docker run -p 8080:8080 fetch-url-api
-
Send a request to the API:
curl -X POST http://localhost:8080/fetch \ -H "Content-Type: application/json" \ -d '{"url":"https://example.com", "proxy":"http://proxyserver:port"}'Note: The
proxyfield is optional.
Fetches the final rendered HTML content of the provided URL.
Request Body:
url(string): The URL to fetch the content from. (Required)proxy(string): The proxy server to use. (Optional)
Response:
content(string): The rendered HTML content of the URL.
Example Request:
{
"url": "https://example.com",
"proxy": "http://proxyserver:port"
}Example Response:
{
"content": "<html>...</html>"
}This project is licensed under the MIT License.
Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent