Tutorials

How to Use a Proxy with Python Requests

By IPMusk TeamUpdated 5 min read

Pass a proxies dictionary to Requests, using the target URL's scheme as the key and your proxy URL as the value. For SOCKS5, install requests[socks] and use socks5h:// when the proxy should resolve the destination hostname.

Start with the right connection details

You need Python, Requests, and a proxy host, port, username and password. Use the proxy credentials from your provider, not the email and password you use to sign in to its website. At IPMusk, copy these details only after the resource is available in your dashboard.

The examples below use placeholder credentials and IPMusk's Rotating Residential gateway, gate.ipmusk.com:31212. Static ISP and Datacenter proxies use the host and port assigned to each resource instead. Replace all placeholders locally before running a network request. These are configuration examples, not reports of a live proxy test.

Create a virtual environment for your project if you do not already have one. Using python -m pip installs into the Python interpreter you select; use the same interpreter to run your script. The SOCKS extra also works for the HTTP examples, so one installation covers both protocols.

Shell: install Requests with SOCKS support
python -m pip install "requests[socks]"

Send an HTTPS request through an HTTP proxy

Set both dictionary keys if your script may request either HTTP or HTTPS destinations. The key describes the destination URL. The scheme at the start of the value describes the connection to the proxy. An HTTPS destination can therefore use an http:// proxy value: the client asks the HTTP proxy to open a CONNECT tunnel, then negotiates TLS with the destination through it.

Do not change the proxy value to https:// just because your destination uses HTTPS. That value requests TLS to the proxy itself and needs a proxy endpoint that supports it. Keep the scheme supplied by your provider. For an HTTP destination, an HTTP proxy does not add end-to-end encryption; use HTTPS destinations when sending sensitive data.

This example percent-encodes each credential separately so a password containing @, : or / does not break the URL. Replace USERNAME and PASSWORD with the raw credentials, not an already encoded connection string. In a shared project, load them from your local secret store instead of committing them. Avoid printing the proxy URL or a full exception that might include it.

Python: authenticated HTTP proxy
from urllib.parse import quote
import requests

username = quote("USERNAME", safe="")
password = quote("PASSWORD", safe="")
proxy_url = f"http://{username}:{password}@gate.ipmusk.com:31212"
proxies = {"http": proxy_url, "https": proxy_url}

with requests.Session() as session:
    session.trust_env = False
    response = session.get(
        "https://api.ipify.org?format=json",
        proxies=proxies,
        timeout=(10, 30),
    )
    response.raise_for_status()
    print(response.json()["ip"])

The printed value is the exit IP seen by that destination for that request. No specific IP or successful response is promised by this example.

Use SOCKS5 with proxy-side DNS

Requests needs its optional SOCKS dependency for SOCKS proxy URLs. If you installed only requests, run the installation command above in the same environment. An error about missing dependencies for SOCKS support usually means that extra is absent from the interpreter running your script.

Use socks5h:// to have the proxy resolve the destination hostname. With socks5://, resolution happens on your machine. This difference matters when your local DNS cannot resolve a destination or when you want the destination lookup to happen through the proxy. Your machine still needs to locate and reach the proxy gateway itself.

The following is a standalone SOCKS example. Both dictionary keys use the same SOCKS URL because either kind of destination can travel through SOCKS5. HTTPS still supplies destination TLS; SOCKS5 itself does not encrypt the traffic.

Python: SOCKS5 with remote destination DNS
from urllib.parse import quote
import requests

username = quote("USERNAME", safe="")
password = quote("PASSWORD", safe="")
proxy_url = f"socks5h://{username}:{password}@gate.ipmusk.com:31212"

with requests.Session() as session:
    session.trust_env = False
    response = session.get(
        "https://api.ipify.org?format=json",
        proxies={"http": proxy_url, "https": proxy_url},
        timeout=(10, 30),
    )
    response.raise_for_status()
    print(response.json()["ip"])

Keep environment settings from changing the route

Requests normally reads proxy settings from the environment, including http_proxy, https_proxy, all_proxy and no_proxy, with uppercase variants also supported. That is useful on a managed workstation but can make a copied script behave differently on another computer. Check those settings when a request appears to use an unexpected proxy or bypass one.

The examples pass proxies on each request and set session.trust_env = False to make the setup explicit. This also disables other environment-derived settings, including netrc authentication and Requests' environment CA bundle selection. If your organization needs a private CA, configure the approved bundle explicitly with verify set to its path. Do not solve certificate errors by setting verify=False.

Choose one configuration approach for the script and document it. If you intentionally rely on environment proxies, keep trust_env enabled and check NO_PROXY exclusions. Putting a credential-bearing URL in a terminal command can leave it in shell history; keep passwords out of commands you share.

Set timeouts and choose whether the IP should stay

Requests has no default timeout. The tuple timeout=(10, 30) sets a connection timeout and a read timeout in seconds. It is not a 40-second deadline for the whole download: read timeout measures waiting between received data, and a request can take longer overall. Pick values that fit your destination rather than leaving a stalled request open indefinitely.

A Session reuses connections where possible and keeps cookies between requests. It does not tell a rotating proxy provider to preserve an exit IP. If a workflow needs a stable address for several steps, generate a sticky connection in IPMusk and use its generated username unchanged. Sticky options are 1, 3, 10 or 30 minutes; they are not a promise of the same address tomorrow. Use a fixed Static ISP resource when the requirement is a fixed address across days.

Start with a single request to an IP-check endpoint, then try a small request to the destination you actually use. A successful IP check confirms that one route worked. It does not show that every destination accepts the address. Respect the destination's access rules and request limits before increasing volume.

Find the failure before retrying

Check the host, port, scheme and credentials first. Retrying the same malformed URL will not repair it. Keep any diagnostic output free of passwords, proxy URLs and authorization headers.

Python Requests proxy errors and the next check
SymptomWhat to check
407 or a CONNECT tunnel failure mentioning 407Check the proxy username and password, URL encoding, and whether you used the credentials for this product. HTTPS tunnel failures can surface as ProxyError rather than a normal Response.
ProxyError or connection refusedCheck the gateway, port and proxy scheme. Confirm the resource is available and that your local network permits connections to that port.
Missing dependencies for SOCKS supportInstall requests[socks] with the interpreter that runs the script.
Name resolution failureSeparate gateway DNS from destination DNS. With SOCKS, choose socks5h:// if destination resolution should happen through the proxy.
ConnectTimeout or ReadTimeoutCheck reachability or a slow destination. Set bounded timeouts and diagnose with one request before adding retries.
SSLErrorCheck the destination certificate, system clock and required CA bundle. Keep certificate verification enabled.
403 or 429 responseInspect the destination's response and access policy. Fix permissions or reduce request frequency; changing proxy settings is not a general remedy.

A timeout does not prove that a write request failed before the server acted. Do not automatically repeat purchases or other state-changing requests without the API's documented retry safeguards.

Check the configuration you will actually run

Keep the first working script small: one destination, explicit proxy settings, bounded timeouts and certificate verification enabled. Confirm the exit address, then check the real destination's status before parsing its body. raise_for_status() raises for unsuccessful HTTP status codes; it cannot tell you whether a successful page contains the data your application expects.

If you need help with a failing connection, share the protocol, redacted host and port details, error type and time of the request. Keep your username and password private. For the exact connection format and product credentials, use the documentation linked below.