Proxies underpin much of the digital world. They power some of the best-known web applications and tools, providing end users with competitive analysis, SEO and marketing insights, price monitoring, and geo-targeted testing.
As an end user, it can be very difficult to know if the IPs sold by the provider have been ‘ethically sourced’ and are reliable. The reality is that providing a large proxy network with the IP diversity necessary to ensure quality of service requires a huge amount of infrastructure and investment. Most proxy providers you’ll come across are either resellers or resellers of resellers. This can lead to a degradation in performance and leave you with a lack of support. The worst-case scenario is that the provider serves up IPs they have no right to sell. That’s why you must choose a proxy provider that owns its network and is trustworthy.
The very best proxy providers also offer a suite of value-added services, making it much easier to get your applications and scripts up and running. So whether you’re a freelancer looking to understand how well your clients perform in Search or training business intelligence algorithms for an enterprise-grade application, this article will walk you through the best proxy services available in 2023. We’ll explain why we chose them, walk you through some of their standout features, and break down the pricing models.
Comparitech does not condone breaching the Terms of Service (ToS) of any website, service, or application. We recommend users thoroughly review the relevant ToS if they intend to scrape/store any data that falls outside their direct ownership.
If you want a quick overview, here is a summary of the best proxy providers at the time of testing:
What is the best Proxy in 2023:
- Bright Data: The best proxy provider in 2023. With access to over 72 million + IPs and a great range of features and value-added services, Bright Data is a one-stop-shop for all your proxy needs.
- Oxylabs: The largest proxy pool available with over 100 million IPs. Provides a wide choice of options, including residential, mobile, and server center IPs.
- SmartProxy: With over 55 million IPs this is a very solid service with competitive prices.
- IPRoyal: With over 8 million residential rotating IPs, IPRoyal is a great entry-level option with some of the cheapest prices around.
- NetNut: With over 52 million IPs, Netbut has a strong residential network that is let down by a lack of value-added services.
The best proxy services in 2023
You might think that choosing the best proxy provider is as simple as looking for the one with the largest pool of IPs, but it’s not as straightforward as that. A proxy must work for your use case and offer failsafes when it doesn’t. The best proxy platforms provide a host of additional features and APIs, rapidly accelerating the development process and reducing the need to maintain your scripts or applications.
Here’s an in-depth analysis of the best proxies in 2023:
1. Bright Data
With over 72 million IPs and a plethora of out-of-the-box solutions, Bright Data won our vote for the best proxy provider in 2023. While OxyLabs has more residential IPs, we found that Bright Data’s network was more reliable and offered consistently better performance where it mattered most.
Features
Bright Data has a solution for just about every use case you can think of and a few you can’t! From its open source, locally run Proxy Manager, allowing you to customize and manage all your proxy operations from one central location to its APIs and Scraping Browser, Bright Data has you covered.
On top of standard residential, mobile, and data center IPs, Bright Data offers ISP IPs. These are residential IPs hosted on a data center. This combination means you get the best of both worlds – the fast response time of a data center but with the reputation of residential IPs. You also get to keep the IPs for as long as you pay for them if maintaining a static IP is important to your use case.
Bright Data’s dashboard is simple enough to move around quickly, and every feature and function has supporting documentation, dynamic code integration examples and even a quick walkthrough video to help you get up and running. Thanks to the thorough documentation and code examples, we created a Python script for checking search engine results in a matter of minutes.
Pricing
Bright Data isn’t the most expensive option on the list, but it’s not the cheapest. They offer a pay-as-you-go option, which is becoming increasingly common and something we love to see because not every user needs thousands of dollars worth of data every month. PAYG starts at $10.50 per GB.
If you decide to build your application on Bright Data, the $500 a month package brings this down to $8.92/GB or $7.87/GB if you go with the $1,000 a month package. A further 10 percent reduces this if you pay annually.
If you’re interested in the SERP API or any of the other features mentioned below, these all come with their custom pricing. Again, cheaper options exist, but as is always the case, you get what you pay for.
Web Scraper IDE
This hosted integrated development environment takes much of the pain out of building custom solutions, getting you from concept to actionable intelligence as quickly as possible. The pre-built code templates covering a wide range of sites, services, and social platforms, as well as the pre-built JavaScript functions, should
reduce development time substantially.
Web Unlocker
A growing number of top-tier providers now provide such solutions, but Bright Data’s Web Unlocker is undoubtedly one of the most powerful we’ve tested. Anyone attempting to scrape publicly available data will inevitably run into various roadblocks, whether it’s protection in the form of CAPTCHAs or more sophisticated fingerprinting techniques.
With the Web Unlocker, you’ll never have to worry about blocks, restrictions, or CAPTCHAs. The Unlocker fully emulates a real user and works completely in the background, allowing you to send unlimited concurrent requests and get back a perfect response without being blocked.
Web Scraping Browser
When scraping data, you should understand the distinction between headless and GUI browsers. A headless browser operates without a visible user interface, making it a popular choice for data scraping in tandem with proxies. It’s fast and scalable. However, detection systems often flag this method, posing challenges for large-scale data extraction.
That’s where Bright Data’s Scraping Browser comes in. It is a GUI (Graphical User Interface) browser called a ‘headfull’ browser. It operates with a visible interface and can be managed using the Puppeteer or Playwright API. This makes it much less prone to detection by bot-protection mechanisms. Additionally, the Scraping Browser comes with integrated features to sidestep website restrictions. Because it runs on Bright Data’s servers, these browsers are perfect for large-scale web scraping tasks. Users can launch multiple Scraping Browsers without needing a vast internal setup.
Use your preferred browser automation API to spin up and control as many Scraping Browser sessions as you need. Save time and resources with unlocking built into the browser, including CAPTCHA solving, browser fingerprinting, and proxy management, and control it easily via Puppeteer (Python), Playwright (Node.js), or Selenium. Enjoy seamless integration with your existing scrapers. It’s a great option for scraping projects that need website interactions, like hovering over an element, clicking buttons, scrolling, adding text, etc.
You might be a bit confused about the difference between Scraping Browser and Web Unlocker. Put simply, Web Unlocker works with one-step requests and Scraping Browser is designed for when you need to interact with a website to retrieve data. It’s also well suited to projects that require a real browser, scaling, and automated management of all website unblocking actions.
Scraping Browser works like other automated browsers and is controlled by common high-level APIs like Puppeteer and Playwright, but is the only browser with built-in website unblocking capabilities. Scraping Browser automatically manages all website unlocking operations under the hood, including: CAPTCHA solving, browser fingerprinting, automatic retries, selecting headers, cookies, & Javascript rendering, and more, so you can save time and resources
SERP API
Whether you’re building an in-house SEO platform, looking to bring an application to market, conducting PPC research, or understanding how you and your competitors appear in search, Bright Data’s SERP API is a great solution. APIs can take much of the guesswork and maintenance out of scraping data. An API means you don’t need to develop or maintain your parsers. You don’t need to change anything if something changes on the target’s end. You can focus on getting the data you need to make better business decisions. The SERP API supports:
- Google Search
- Bing Search
- DuckDuckGo Search
- Yandex Search
- Baidu Search
- Yahoo Search
- Naver Search
Output can be structured as either JSON or HTML.
Dataset Marketplace and custom datasets
Suppose you want actionable insights but don’t have the appetite for building your own solution. In that case, OxyLabs has attempted to skip straight to the final act by providing premade datasets for a growing number of industries and verticals. They deliver parsed and ready-to-use company data in a format that suits you, such as:
Records cost $0.001 per record, but with the very comprehensive datasets that Bright Data holds, this could become an expensive option. The full LinkedIn database will set you back $418,202, for example with 418m records. The minimum order value is $500, and you can select from specific subsets of data.
Code integration examples & API Playground
Another thing we liked about Bright Data was its code integration examples. And the ability to test the API out with the ‘Playground.’ This might not sound like a big deal, but if you’re just getting started in proxy integration or are unsure how to leverage the power of proxies for your use cases, these sample scripts and sandboxes are a fantastic springboard. Bright Data even populates the examples with your usernames, passwords, and preferences, so it’s literally as simple as copying and pasting the code. It’s not a game-changer, but a very nice touch.
Bright Insights: A dedicated eCommerce platform
Bright Data offers a standalone eCommerce platform, allowing you to track competitor products across thousands of categories. You can quickly identify when products are added, sold, unlisted, or out of stock.
We couldn’t fully test the platform, but it is a powerful feature for those in eCommerce, allowing you to align your products and pricing with market trends and demand. You can also track competitor promotions and ad spend across multiple platforms.
[comparitech_get_deal provider_id=”446759″ data_1=”411957 best proxies” data_3=””][/comparitech_get_deal]
2. Oxylabs
Oxylabs is the largest and best-known provider around. It provides access to the largest proxy pool of any provider with over 100 million IPs.
Pricing
Oxylabs is another provider that has recently started offering pay-as-you-go pricing for its residential pool. PAYG is a great option if you want to dip your toes in or don’t require much data. It’s $15 per GB, though, which makes it the most expensive entry price of any of our top recommendations.
Packages start at $300 per month, bringing the price down to $12/GB. $600 a month will drop the price to $10/GB, and $800 a month will see the price fall to $8/GB. There are mostly custom plans for heavy users, which will decrease the price further, and you can knock another 10 percent off by paying annually.
All in all, Oxylabs is one of the most expensive providers of residential proxies on the list. One could argue that the price is worth it because the quality and diversity of IPs on offer are undeniably excellent.
APIs
As well as the usual resi, DCs, and mobile proxies, Oxylabs probably has the largest choice of stock APIs available of any provider. It offers custom APIs for:
- SERPs (search engine results page)
- e-Commerce
- Real estate
- Web scraper (general purpose)
As mentioned above, APIs take much of the pain out of automated data harvesting because you don’t need to worry about building and maintaining your own parsers. If the target website makes a tweak, it’s not on you to adapt to said tweak… sit back and let the API do its thing. Oxylabs uses a machine learning parser that can adapt to different layouts, dynamic fingerprint browsing, and JavaScript rendering.
The various APIs on offer have a refreshingly straightforward pricing structure. Starting at $49 a month for approximately 17,500 results ($2.80 / 1K results) and going up to $249 per month for 104,000 ($2.40 / 1K results). It would be nice to see the price per request come down a little more as you scaled through the plans, but it’s a reasonable price at both ends of the spectrum, considering how well the APIs worked in our testing.
Web Crawler
Web Crawler is an add-on to Oxylabs Scraper APIs that allows you to leverage the API’s scraping and parsing functions to crawl websites at scale in real time. You simply form an input that determines the crawling scope, specify the scraping parameters, and submit a request to the job initiation endpoint. Web Crawler acts much like a spider and moves between pages until it finds no more new URLs that match the patterns specified by the user. Web Crawler then aggregates the result files and provides them in a format of your choosing or uploads them to cloud storage.
Datasets
Much like Bright Data, Oxylabs offers its own datasets for a variety of sources. These include:
- AngelList startup data
- CrunchBase data
- Craft.co data
- Product hunt data
This is quite an expensive option, starting at $1,000 per month, and is only useful if your use case fits neatly into the predefined scope of the dataset (they do offer a custom service for any public domain), but if it happens to be what you need, the cost could be negligible compared to building and maintaining your own solution.
Built-in Scheduler feature
If you have identical scraping jobs that you execute regularly, OxyLab’s built-in Scheduler feature can be a bit of God send. You can create multiple jobs using the Cron expression to set the time intervals and get automatic data updates sent to your chosen cloud.
[comparitech_get_deal provider_id=”413231″ data_1=”411957 best proxies” data_3=””][/comparitech_get_deal]
3. Smartproxy
SmartProxy is another great option if you’re looking for high-performance proxies for a reasonable price. With over 40 million residential IPs, the pool isn’t quite as diverse as Bright Data and Oxylabs, but our testing found the shared IPs to perform on part for most tasks we could think to throw at it.
As well as residential IPs, Smartproxy also offers the prerequisite Data Center, Mobile, and Dedicated Data Center proxies. It’s also recently introduced an ISP offering, which gives you the speed of data center proxies and the reliability and trust of residential IPs
Pricing
Smartproxy is the only provider on the list to unashamedly offer smaller PAYG plans for non-enterprise users. Pay As You Go plans start at $8.50 per GB. Subscriptions start from $80 per month for 8GB of data ($10 per GB). This quickly drops without need for massive upfront investment:
- 2GB = $7/GB
- 8GB = $6.50/GB
- 25GB = $6/GB
- 50GB = $5.60/GB
- 100GB = $5.20/GB
And that’s just the ‘Consumer’ pricing. Enterprise plans bring the price down further:
- 250GB = $4.80/GB
- 500GB = $4.40/GB
- 1000GB = $4/GB
This new and aggressive pricing strategy firmly blows a lot of the competition out of the water and reduces the barrier to entry significantly. Even just buying 2GB of data means you will be paying less than half the price of Oxylabs. We can only hope other providers take note and follow suit.
Site Unblocker
Like the offerings from Bright Data and Oxylabs, Smartproxy has recently introduced it’s all-singing, all dancing Site Unblockler. This is designed to take the headaches out of scraping by offering a single endpoint to effortlessly integrate into your code.
The feature offers ‘Intelligent Proxy Selection’ to choose the right proxy type for the site you’re targeting. Any blocked IPs are automaticalled retired and replaced in order to ensure a 100% success rate. We tested this on using a Cloudflare protected site (that we own) and it was able to successfully scrape even javascript heavy pages.
A good selection of APIs
Like other top-tier providers, Smartproxy offers a range of APIs to take the hassle out of scraping. Notably, Smartproxy includes a Social Media API, and while this functionality is achievable via methods using Bright Data or OxyLabs, this is the only provider to offer this as an ‘off-the-shelf’ API.
Smartproxy also offers SERP and eCommerce APIs.
No-Code Scraper
There are plenty of businesses and users that want to gather data from the web but have no idea where to start. Smartproxy has taken a big step toward meeting the needs of this growing user base by offering a ‘no-code’ solution. You pick a predefined template (Google or Amazon), feed it some parameters through the point-and-click interface, and you’ll be up and running in no time.
Smartproxy also offers a universal no-code solution that comes in the form of a browser extension. Simply click on the elements within a webpage you want to scrape, and Smartproxy will do the heavy lifting.
In our experience, these no-code solutions can be a bit hit-and-miss, but if your requirements are basic and you don’t have the resources to develop a custom solution, it’s a great way to get actionable insights quickly. In a world of AI, where people increasingly expect to feed a prompt to a chatbot and get the data they need, we hope proxy providers will continue to build on these offerings.
[comparitech_get_deal provider_id=”414199″ data_1=”411957 best proxies” data_3=””][/comparitech_get_deal]
4. IPRoyal
IPRoyal is a relatively small provider compared to the three above, with 8m+ residential proxies, but it’s growing quickly and punches well above its weight in terms of performance and even more so when it comes to price.
The pricing model for pay-as-you-go residential IPs is, as usual, the more you buy, the less you pay. IPRoyal’s model can be quite tricky to understand, but this is only because the UX needs some work. Let’s break it down using their Royal Residential Proxies (their main offering) as an example:
- 1GB will set you back $7
- 2GB = $5.95/GB
- 10-50GB = $5.25/GB
- 50-100GB = $4.90/GB
- 100-250GB = $4.55/GB
- 250-500GB = $4.20/GB
- 500-1,000GB = $3.50/GB
- 1000-3,000GB = $3.15/GB
- 3000-50,00GB = $2.80/GB
- 5000-10,000GB = $2.45/GB
- 10,000+ = $1.75/GB
Of course, by the time you hit the $1.75 mark, you’ve spent $17,500 on proxies, but you can see the economies of scale in action. Even at the more expensive end, the price is very competitive. In fact, it’s the cheapest option on this list; and we found the proxies themselves to perform well for a wide range of services.
Unlike other providers, IPRoyal is quite open about how it sources its residential IPs. Initially, it offered a ‘pawn’ service that allows users to rent (for lack of a better word) some of their unused bandwidth to consumers of IPRoyal. They have since rebranded this business wing to a standalone service called pawns.app.
Their growth has been slow and steady, but they have a way to go before competing with the providers mentioned above.
Features
The service is very much ‘what you see is what you get.’ No fancy APIs or custom scraping solutions… just IPs. But if IPs are all you need and you don’t want to sell your left kidney to be able to afford them, IPRoyal might be right up your street.
They offer the usual: Residential, Data Center, Mobile and Private/Dedicated options. However, unlike some other providers on the list, there are no bandwidth or traffic limitations on their data center offerings, which is nice to see.
The dashboard and UX leave a lot to be desired. If you’re familiar with the AWS or GCP dashboards, it’s a bit like that. Nothing is quite where you think it should be, but it’s entirely manageable once you get used to it. In fact, if you’re looking to plug your proxies directly into a third-party service that requires a standard ‘hostname:port:username:password‘ format, it’s one of the easiest to use. Feed it the type of proxy you need, the location, static or rotating, and how many you need, and it will automatically generate a list for you to copy and paste into your platform of choice. Other providers offer similar functionality, but it’s usually hidden behind a layer of jargon, which means you need to come to the table with an element of knowledge.
[comparitech_get_deal provider_id=”446764″ data_1=”411957 best proxies” data_3=””][/comparitech_get_deal]
5. NetNut
NetNut is a proxy provider based in Tel Aviv, Israel. It has a respectable 52m+ IPs, and performance for most tasks is up there, with some providers at the top of this list.
One of the things we don’t like is the barrier to entry. Prices start at $300 a month for a 20GB plan ($15 per GB) and go up to $4,000 per month for a 1TB plan ($4 per GB). It’s not the price per GB we take issue with; it’s the fact that you have to commit to a minimum monthly spend. This model is becoming increasingly dated as a growing number of businesses want to be able to use proxies on an ad hoc basis. Consumers, even those at the enterprise end of the spectrum, expect to sign up and consume services in a manner that suits them.
Netnut’s dashboard also feels a little dated. It’s hard to pinpoint why, but it’s just a little ‘2008’. Like IPRoyal, it’s a stripped-back service. If you want bells and whistles that make your life easier, look at one of the providers at the top of the list. If you want proxies that perform well, NetNut is a solid choice. “It does what it says on the tin,” as we say in the UK.
The only real downside with NetNut is that you do not get city or state-level proxy selection options unless you subscribe to the semi-pro plan and above. You must spend $1625 or above monthly to use NetNut to access IPs in specific locations. If picking your residential IPs by country will suffice for your particular needs, you can opt to pay as little as $300 per month for 20 gigabytes of data.
[comparitech_get_deal provider_id=”413238″ data_1=”411957 best proxies” data_3=””][/comparitech_get_deal]
Best Proxies: Our testing methodology
Every proxy on our list is put through the same rigorous testing process. We assess speed, success rates, and geo-specific content access. We also test the capabilities of specific features native to that proxy provider. For example, if the provider offers a custom SERP API for analyzing search engine results, we create a basic script to test the performance and reliability of that feature.
Here is a basic overview of our methodology:
- Determine the type of proxy to be tested (e.g., residential, data center, mobile).
- Ensure a controlled testing environment and turn off all other network services that might interfere with the testing process.
- Get a baseline by testing the network without a proxy to get standard metrics.
- Anonymity Testing: Check if the proxy reveals the original IP address.
- Speed and Performance Testing: Measure the connection speed through the proxy. We use a combination of custom scripts as well as FOGLDN Proxy Tester. We use the same countries for each test: UK, USA, Hong Kong, France & Germany. We run 300 residential proxies to access the same sites: https://bbc.com and https://www.comparitech.com. We then take an average of these results.
- Geo-Location Testing: We use a simple extension like BP Proxy Switcher to test the access to geo-restricted websites or services to verify if the proxy represents the claimed location.
- If the proxy offers additional features (like browsers, APIs, playgrounds, or native scraping tools), we test each of them to get a reasonable understanding of their capabilities and test the effectiveness of these features.
- Documentation & Support: Even for a seasoned developer, every proxy service is different; therefore, thorough documentation and support are essential. We read through the documentation as part of the testing process and ask support at least two technical questions to see how quickly they can resolve any issues.
What kind of proxies do I need?
While there are dozens of proxy types out there, for the purposes of data scraping and collection, you can break them down into X categories:
Residential proxies
Residential proxies are by-far the most widely used proxies and are suitable for most uses. That’s because they are IP addresses that belong to real, physical devices. You’re essentially piggybacking on someone else’s IP. They appear as average users to the target servers and usually impossible to detect (unless the proxy user abuses it, which happens a lot and is why choosing a provider with a large and diverse IP pool is important).
Many open source and ‘off-the-shelf’ tools expect a list of proxies in a standard format like host:port:username:password. Some providers make it easier than others to generate proxies in this format (see reviews above for those that do).
Others allow you to connect directly to an API taking the hassle out of generating and testing your residential proxiesResidential proxies and almost always billed per gigabyte of data. You’re paying for the traffic that passes through the proxies, rather than the number of IPs. This is great because it means you can tap into millions of IPs across hundreds of countries and thousands of cities and you only pay for the traffic you consume.The downside of residential proxies is that they can be slow. The host’s network conditions will have a direct impact on the speed of the proxy.
Data Center proxies
These are non-physical IP addresses that are artificially created within datacenters that are owned or leased by the proxy provider. The obvious upshot of this is that they are fast. Really fast. If your use cases relies on moving quickly, data center proxies can be a great option.
A single server might host thousands of IPs. The downside here is that they will all share the server’s IP subnetwork. This means that many datacenter proxies will look similar and it’s not uncommon for target servers to automatically ban enitre ranges if they detect abuse within that range. Because you’re sharing IPs with other users, it’s impossible to ensure the IP range will remain ‘clean’, even if you use them responsibly.
Generally with DC proxies, you ‘rent’ an IP address, which you keep for the duration of that rental period. The upside is that you can appear to originate from the same device and location throughout the period. The downside is that if the IP becomes blocked (and they quite often do), you’re stuck with it (unless the provider allows you to rotate the proxy).
ISP Proxies
This is a relatively new concept but the big providers have latched on to the idea and now each offer their own version of ISP proxy. Essentially, these offer the best of both residential and data center proxies. ISP proxies are hosted on data centers, so you get all the speed. However, they carry real IP addresses allocated by official Internet Service Providers, so they are much less likely to be picked up as data center proxies.
Static vs Rotating proxies
What is a static proxy?
As the name implies, a static proxy (also called a ‘sticky IP’) allows you to access the Internet from a single unique IP address. In most cases, this would be a datacenter IP that gets assigned to you. You can continue to use the static IP address as you need continue to pay for it.
This is hugely useful if you need to appear to be a single user over a prolonged period.
You can find ‘static residential proxies’, but these are often limited to quite a short session duration.
What is a rotating proxy?
Rotating proxy are most commonly associated with Residential proxies and provide you access to the entire pool of IP addresses that meet your criteria (country, state, city etc.). Rotating proxies are swapped out based on your requirements. For example, the IP address might change with every request you make, or at a specific time interval. It’s entirely up to you.
Rotating proxies obviously carry a host of benefits. You can make one request from one IP address, or you can make a million requests from a million different IP addresses.
Shared vs Dedicated Proxies
What is a shared proxy?
Shared proxies are probably the most common type of proxy, particularly when it comes to Residential Proxies. Quite simply, you are sharing the IP address with multiple users at any given time. Shared Proxies rely on sophisticated network archietture and load balancing to make sure everyone gets reasonable performance. Shared proxies are significant cheaper that other options because you are splittng the cost.
What is a dedicated or private proxy?
A dedicated proxy (also referred to a ‘private proxy‘) is an IP that is reserved for you. The term is most often associated with data center or ISP proxies, where you lease an IP for a specific period of time, but you can also get dedicated residential IPs/
The terms ‘shared’ and ‘dedicated’ are usually interchangeable, although marketing departments sometimes like to attach buzzwords to specific products, so make sure you thoroughly review the features to make sure the product matches your requirements.
L’article proxy v2 DRAFT DO NOT PUBLISH est apparu en premier sur Comparitech.
0 Commentaires