Restrict a URL fetcher at the point where it opens a connection, not only when it first parses a string. Define permitted schemes, destinations, and ports; validate DNS results against that policy; prevent redirects or revalidate every new destination; and enforce a separate network egress boundary.
For an image importer, the simplest design may be a short list of approved image providers rather than arbitrary URLs. If arbitrary public URLs are a real requirement, acknowledge the larger validation and network problem instead of disguising it as a regular-expression check.
This is a defensive design review with a proposed isolated test matrix. No network probes, external targets, DNS experiments, or SSRF exploit attempts were executed.
Decide which product you are building
An importer that accepts an asset identifier from one known provider can build the destination itself. An importer that accepts any URL gives the user influence over where a privileged server connects. Those interfaces have different security requirements.
For the worked example, assume a service imports images from two approved providers. The application fixes HTTPS and port 443, sends no user session credentials, and stores the result for later processing. Provider paths are treated as untrusted data and constrained to the provider's documented asset interface. A generic provider-side URL proxy would reintroduce arbitrary fetching one hop later.
OWASP's SSRF prevention guidance distinguishes known destinations from arbitrary external destinations and recommends application and network controls. It warns that validation must account for DNS behavior and redirects. Its distinction is useful before selecting a library: a convenience API that follows every redirect may not match the importer's intended contract.
Inventory the worker's actual network position. A background import service may reach internal services unavailable to the browser that submits the URL. Restrict its credentials and connectivity to the import task; the browser's own cross-origin restrictions do not establish the server's destination policy.
Keep parsing and policy separate
A parser answers questions such as “what host and port does this string represent?” The policy answers “may this worker contact that host and port?” Successful parsing is only the beginning.
Define how the application handles user information, fragments, internationalized hostnames, explicit ports, empty hosts, and unsupported schemes. For a narrow provider integration, rejecting unsupported forms is often easier to review than normalizing every possible form into something acceptable.
Python's URL parsing documentation explicitly cautions that its parsing functions are not input validators. This article does not supply a universal parser or a supposedly complete address blacklist. The selected HTTP client, proxy, and downstream service must interpret the destination consistently.
Prefer one reviewed parsing path and structured fields across internal boundaries. If a second component reparses a reconstructed URL differently, the first component's decision may no longer describe the eventual request. Include the exact library versions and proxy configuration in the implementation review.
Bind the permitted destination to the connection
A hostname can resolve to more than one address, and its answers can change. An address check followed by an independent lookup in the HTTP client creates a gap between the address approved and the address contacted. GitLab's URL-blocker guidance makes this concrete: ignoring the validated destination returned by its blocker loses the intended DNS-rebinding protection.
Validate the resolved IPv4 and IPv6 destinations against the application's allowed networks, then ensure that the client uses a validated address. Apply the same rule to retries and fallback connections. When a destination cannot be evaluated safely, fail the import rather than silently selecting a different route.
Preserve HTTPS identity while controlling resolution. The connection's address, the HTTP host, TLS Server Name Indication, and certificate verification serve related but distinct purposes. The curl documentation on custom name resolution illustrates this with --resolve: the connection address can be supplied while the URL's hostname remains the TLS identity. That mechanism illustrates the distinction; it is not a complete SSRF defense or an instruction to change a production fetcher's configuration blindly.
Disable redirects if the provider contract does not need them. If redirects are required, treat each hop as a fresh destination decision, bound the hop count, and avoid forwarding credentials across origins. A permitted first destination does not grant permission to every destination it names.
Keep egress enforcement outside the URL-validation function. An isolated worker with a tightly controlled outbound path can limit the consequences of an application mistake. Review how DNS, proxies, local service sockets, and alternate protocols interact with that boundary.
A small test plan that measures the right things
Use only controlled services inside an isolated lab. Configure synthetic DNS records and distinguish a permitted service from a forbidden service. The following cases are proposed tests, not observations from this article.
| Case | Expected result | Independent evidence |
|---|---|---|
| Valid provider asset | Import completes within configured limits | Permitted server records the intended request |
| Unsupported scheme or port | Rejected before connection | No server receives a request |
| Host resolves to a forbidden lab address | Rejected | Forbidden service receives nothing; policy denial is recorded |
| Allowed host changes resolution between checks | Connection remains bound to the approved address or fails | Connection trace matches the decision |
| Redirect points to forbidden service | Rejected before the new connection | No request reaches redirected destination |
| Retry selects an alternate forbidden address | Rejected | Retry cannot bypass destination policy |
| Allowed service sends an oversized response | Transfer stops under the size policy | Bounded received bytes and a clear failure result |
Test permitted operations too. A blocked request to a service that was already down cannot establish that the destination policy works. Keep the service logs, client result, and effective policy together so the conclusion can be checked.
Do not substitute live cloud metadata endpoints, internal production hosts, or someone else's website for the forbidden lab service. The property being tested is the policy's refusal, not access to valuable data.
The response is another untrusted input
A permitted connection can return harmful or unexpectedly expensive content. Apply connection and total-duration limits, cap bytes received and decoded image dimensions, and use an appropriate image-processing boundary. Check the content rather than trusting a filename or declared media type alone.
Decide what the caller sees on failure. Returning arbitrary upstream bodies or headers can expose data the importer was never meant to relay. A small error code and an internal correlation identifier may be enough for support.
These limits address resource use and response handling; they do not replace destination restrictions. Keep those findings separate in the review record. The same distinction helps when auditing a URL-fetching MCP tool: a valid tool schema and a successful fetch do not establish that the destination was authorized.
Ship the narrowest interface that meets the requirement, then maintain the policy alongside provider changes and client upgrades. Record the selected parser, resolver, client, proxy, permitted destinations, redirect behavior, and executed tests. That record is much more useful than a claim that the URL “looked safe.”
Documentation reviewed October 28, 2026. The connection and DNS cases remain an unexecuted lab protocol; no SSRF resistance is claimed for a running service.
