Tutorials

How to Use a Proxy with curl

By IPMusk TeamUpdated 4 min read

Pass your proxy URL with --proxy and the destination URL at the end of the command. Use http:// for an HTTP proxy, or socks5h:// for SOCKS5 when the proxy should resolve the destination hostname.

Get your proxy connection details

Copy the proxy protocol, host, port, username and password from your provider. IPMusk Rotating Residential uses gate.ipmusk.com:31212. Static ISP and Datacenter resources have their own assigned host and port. If your dashboard generates a username for a particular location or session, copy the whole username.

The examples use placeholders and request a simple HTTPS page. Use a small endpoint you control or have permission to test when checking your own configuration. Check which curl build your shell runs with curl --version. This matters if your terminal and a scheduled job pick up different executables.

Make an HTTPS request through an HTTP proxy

Use --proxy-user for the proxy login and keep the credentials separate from the proxy URL. For an interactive run, pass just the username and curl prompts for the password. This keeps the password out of the command you type. For a scheduled job, keep the password in your secret store and supply it through a curl config that only the job can read.

The URL after --proxy describes the connection to the proxy. The last URL is where the request goes. For an HTTPS destination through an HTTP proxy, curl asks the proxy for a tunnel using CONNECT and then negotiates TLS with the destination. An https:// proxy URL would request a TLS connection to the proxy itself. Use it only when your provider supplies that kind of endpoint.

The examples put -q first to ignore the default curl configuration for this test. The empty argument to --noproxy clears the bypass list for this command, including one set by NO_PROXY. Leave out that override when your network requires local or internal destinations to connect directly.

Shell · HTTP proxy with an interactive password prompt
curl -q --proxy 'http://gate.ipmusk.com:31212' \
  --proxy-user 'USERNAME' --noproxy '' \
  --connect-timeout 10 --max-time 25 \
  'https://example.com/'

Use real credentials only on your own machine. With an http:// proxy and Basic proxy authentication, TLS to the destination begins after CONNECT. It does not protect the earlier exchange with the proxy.

Choose where SOCKS5 resolves the hostname

With socks5://, curl resolves the destination hostname on your machine and sends the address to the proxy. With socks5h://, curl gives the hostname to the proxy for resolution. In both cases your machine still needs to resolve and reach the proxy gateway when the gateway is specified by name.

Use socks5h:// when you want the proxy to resolve the hostname of your test endpoint. Keep the destination, account and time limits the same when comparing the two schemes. If one works and the other fails, the difference gives you a DNS question to investigate, since the two resolvers may see different records. SOCKS5 supplies the route. HTTPS supplies TLS to the destination.

Where the destination hostname is resolved with curl and SOCKS5
Proxy URL schemeWho resolves the destination
socks5://curl uses the local resolver
socks5h://The proxy resolves the hostname
Shell · SOCKS5 with DNS at the proxy
curl -q --proxy 'socks5h://gate.ipmusk.com:31212' \
  --proxy-user 'USERNAME' --noproxy '' \
  --connect-timeout 10 --max-time 25 \
  'https://example.com/'

Read the proxy and destination results separately

A page body by itself tells you little about the route. For an HTTP proxy tunneling to HTTPS, write out http_connect and http_code. The first is the HTTP response code for the CONNECT tunnel request. The second is the HTTP status of the transfer. With SOCKS5 there is no HTTP CONNECT response to inspect.

For a separate check of the visible exit address, run your command against an HTTPS endpoint that reports the IP it sees. If useful, compare with a direct request from the same machine and keep the endpoint unchanged. The result describes that one request. A rotating connection can show a different exit address on the next request.

Keep curl’s process exit status alongside the two HTTP fields when you log a test. If CONNECT returned 200 and the transfer then failed before an HTTP status arrived, look at the TLS or timeout error. If the destination replied with an HTTP status, compare that status with what the application expected before changing the proxy.

Start with one request and a finite time limit. The connect timeout covers the connection phase, including the applicable DNS and handshake steps. The maximum time limits the overall transfer. Add retries later only if your application can safely repeat the request.

Shell · Print CONNECT and destination status for an HTTP proxy
curl -q --proxy 'http://gate.ipmusk.com:31212' \
  --proxy-user 'USERNAME' --noproxy '' \
  --connect-timeout 10 --max-time 25 \
  --silent --show-error --output /dev/null \
  --write-out 'CONNECT=%{http_connect} HTTP=%{http_code}\n' \
  'https://example.com/'

curl can finish a transfer successfully even when the destination returns an HTTP error status. Read the status explicitly, or use --fail in a script that treats HTTP errors as failures.

Change one setting at a time when a request fails

  • If curl cannot resolve or reach the proxy, check the proxy host and port copied from your dashboard, then check network access from the same machine that runs the job.
  • If the HTTP proxy returns 407, check the proxy login and whether the account can use that endpoint. A SOCKS5 authentication failure comes from its own handshake and has no HTTP 407 status.
  • If CONNECT succeeds and the HTTPS request returns 403, check the destination's permissions and whether the request is allowed. If CONNECT itself returns 403, check the proxy's connection policy first.
  • If a test appears to go direct, check NO_PROXY and any curl configuration your normal command loads. Keep the proxy and target URLs separate so the intended route is easy to inspect.
  • If certificate verification fails, check the destination hostname and certificate trust. For a private test service, supply its trusted CA certificate with --cacert.

Before sharing a diagnostic log, remove proxy credentials, authorization headers and any private URLs. Start with the curl error message and the two status fields.