

You can use the proxy to scrape images, documents, PDFs or other files just as you would any other URL.Your requests will receive a 429 error if you go over your concurrency limit and a 403 if you go over your plan's maximum requests.
#UNIVERSAL XML SCRAPER VIRTUALMACHINE CODE#
Unsuccessful requests will return with a status code of 500.Successful requests will return with a status code of 200, 404 or 410. Each request will be sent through to the API and the response returned just like the API normally would.The proxy will take your requests and pass them through to the API which will take care of proxy rotation, captchas and retries. To simplify implementation, we offer a proxy front-end to the API. We offer SDKs for NodeJS, Python, Ruby, and PHP.Each request will return a string containing the raw html from the page requested, along with any headers and cookies.
#UNIVERSAL XML SCRAPER VIRTUALMACHINE FREE#
There is no overage allowed on the free plan, if you exceed 1000 requests per month on the free plan, you will receive a 403 error.If you exceed your plan concurrent connection limit, the API will respond with a 429 status code, this can be solved by slowing down your request rate.You can scrape images, PDFs or other files just as you would any other URL, just remember that there is a 2MB limit per request. Make sure to catch these errors! They will occur on roughly 1-2% of requests for hard to scrape websites. In cases where every request fails in 60 seconds we will return a 500 error, you may retry the request and you will not be charged for the unsuccessful request (you are only charged for successful requests, 200 and 404 status codes). Remember to set your timeout to 60 seconds to ensure this process goes smoothly. Each request will be retried until it can be successfully completed (up to 60 seconds).A few things to consider before we get started:

Scraper API is designed to simplify web scraping. If you need help choosing a plan, or have custom requirements then just reply to this email.
