README

Link preview for PHP: fetch a user-supplied URL through any PSR-18 client with an SSRF-aware redirect loop, and read title, description, image, favicon and more from oEmbed, JSON-LD, OpenGraph, Twitter Cards and plain HTML - with built-in adapters for eleven sites.

Using an AI coding assistant? llms.txt contains a compact API reference you can share with the model.

Projects using the llm/skills Composer plugin also get this package's agent skill synced into .agents/skills/ automatically on install.

Russian version: README.ru.md.

Requirements

• PHP 8.3-8.5 (8.6 is checked in CI as experimental) with dom, libxml, mbstring, intl, ctype, filter
• psr/http-client, psr/http-factory, psr/http-message
• rasuvaeff/json-ld (the JSON-LD 1.1 processor behind the JSON-LD source), rasuvaeff/result, rasuvaeff/schema (reads the JSON answers of oEmbed, Wikipedia and Rutube internally; pulls rasuvaeff/property-testing-core in at runtime)
• A PSR-18 client and a PSR-17 request factory. The recommended client is rasuvaeff/pinned-http-client (DNS pinning, brings Guzzle's PSR-17 factory); any other works (Guzzle, Symfony HttpClient + Nyholm PSR-7, ...)

Installation

composer require rasuvaeff/link-preview
composer require rasuvaeff/pinned-http-client   # recommended PSR-18 client: pins the connection to checked addresses

Usage

use GuzzleHttp\Psr7\HttpFactory;
use Rasuvaeff\LinkPreview\Fetch\Fetcher;
use Rasuvaeff\LinkPreview\Fetch\UrlPolicy;
use Rasuvaeff\LinkPreview\Fetch\UserAgentMap;
use Rasuvaeff\LinkPreview\PreviewProvider;
use Rasuvaeff\PinnedHttpClient\AddressPolicy;
use Rasuvaeff\PinnedHttpClient\PinnedClient;

$policy = new UrlPolicy();

$provider = new PreviewProvider(
    fetcher: new Fetcher(
        client: new PinnedClient(
            // the addresses a host resolves to are judged by the same rules as the URL
            policy: new class ($policy) implements AddressPolicy {
                public function __construct(private UrlPolicy $policy) {}

                public function allows(string $address): bool
                {
                    return $this->policy->allowsAddress($address);
                }
            },
            timeoutSeconds: 5.0,
            connectTimeoutSeconds: 3.0,
            maxBodyBytes: 2_097_153, // above the Fetcher's limit: an oversized page is the Fetcher's TooLarge
        ),
        requestFactory: new HttpFactory(),
        policy: $policy,
        userAgents: UserAgentMap::unfurler(),
    ),
);

$result = $provider->preview(url: 'https://example.com/article');

if ($result->isOk()) {
    $info = $result->unwrap();   // PageInfo
    echo $info->title, ' - ', $info->image;
} else {
    $failure = $result->error(); // PreviewFailure
    echo $failure?->reason->value;                       // policy_rejected, network, http_status, ...
    echo $failure?->isTransient() ? ' (retry later)' : ''; // see "Failures" below
}

Do not wrap PinnedClient in its RedirectFollower: the Fetcher follows redirects itself and checks every hop. With a plain Guzzle client instead, pass new Client(['allow_redirects' => false, 'http_errors' => false, 'timeout' => 5.0]) - and read the Security section about DNS rebinding.

The pipeline is UrlPolicy -> site adapter (YouTube, Wikipedia, ...) -> Fetcher (policy on every hop, size and content-type limits) -> PageDocument -> extractors -> fixed-priority merge -> Limits (clean text, bounds) -> PageInfo. Every stage returns a Result; nothing throws on bad input from a user or the network.

PreviewProvider implements LinkPreviewer (preview(), fromHtml()). Depend on the interface to put a cache, a rate limit or a circuit breaker in front of it - examples/cached_previewer.php caches previews as PageInfo::toArray() and reads them back with PageInfo::fromArray().

PageInfo

providerName is the service behind the content: oEmbed provider_name, or the site adapter (Wikipedia, GitHub, ...). toArray() returns all fifteen keys; PageInfo::fromArray($array) reads them back (fromArray($info->toArray())->toArray() === $info->toArray()), treats missing keys as null - an array stored by 0.1 still reads - and throws InvalidArgumentException on a wrong type.

siteName is only what a source said — the page's own site name, or else the oEmbed/adapter providerName (always, with or without the option below). To show the host when nothing names the site, opt in: new PreviewProvider(fetcher: $fetcher, hostAsSiteName: true) - the host of url without www., in its ASCII form (xn--mnchen-3ya.de, not a Unicode look-alike). It is opt-in so that a siteName you store or compare is never a guess.

Failures

A failed preview is a PreviewFailure: reason (PreviewFailureReason), status and retryAfterSeconds (for an HTTP status answer) and isTransient(), which says whether the same URL may preview later. The Fetcher reports the same way through FetchFailure / FetchFailureReason.

retryAfterSeconds is the server's Retry-After: delta-seconds as sent, an HTTP-date counted from the response's own Date header (without one an HTTP-date is not read - the package keeps no clock), capped at a day. Retry a transient failure after retryAfterSeconds when it is there, after your own backoff otherwise; do not retry a permanent one.

Sources and priority

Each field is taken from the first source that has it, in this order: site adapter, oEmbed (discovered through <link rel="alternate" type="application/json+oembed">), JSON-LD (schema.org Article, VideoObject, WebPage, Organization), OpenGraph, Twitter Cards, plain HTML. Pass your own extractors: list (highest priority first, after the adapter) to change that. Text is cleaned as it is merged, so a value that is nothing but markup (<b></b>,  ) does not hide the next source's; the image and its dimensions are merged as one field.

oEmbed discovery makes one extra request: the endpoint is an address chosen by the page, so it is fetched through the same Fetcher - same UrlPolicy, same redirect checks - and can never reach what the original URL could not. JSON-LD is read by ProcessorJsonLdReader (behind the JsonLdReader interface, so you can replace it): the block is expanded by the JSON-LD 1.1 processor of rasuvaeff/json-ld, the schema.org properties are mapped back to the short names the extractor reads. See JSON-LD below.

JSON-LD

ProcessorJsonLdReader expands every <script type="application/ld+json"> block with rasuvaeff/json-ld, so inline contexts, aliases ("title": "headline") and compact IRIs work, together with @graph, lists, several types and {"@id": ...} references between nodes of one block. Nothing is fetched from the network: the https://schema.org context (any spelling: http, www, /docs/jsonldcontext.json) is answered from memory, with its schema: prefix; any other remote context makes the block UnsupportedContext and it is skipped. A type or a property outside schema.org is dropped, so a term redefined to a foreign IRI ({"headline": "http://evil.example/x"}) is not read. Input nested deeper than 32 levels is refused before the processor sees it. Tested on 14 blocks collected from real pages (Wikipedia, YouTube, BBC, The New York Times, IMDb, Kommersant, Championat and others).

To read JSON-LD some other way, implement JsonLdReader (nodes(string $json): Result<list<array<string, mixed>>, JsonLdFailure>: schema.org nodes with short property names, no I/O of its own) and pass new JsonLdExtractor(new YourReader()) in extractors:.

oEmbed gives title, author_name, provider_name (as providerName and siteName), thumbnail_url with thumbnail_width / thumbnail_height (a photo: url, width, height), the <iframe src> of html as embedUrl, and type. JSON-LD gives an ImageObject's width / height (a number, digits or a QuantitativeValue) with its url. A side is a positive integer below 100 000; anything else ("100%") is unknown.

All URLs are absolute (<base href> is honoured) and only http/https: javascript:, data:, file: and URLs with credentials are dropped, and the next candidate is tried.

UrlPolicy

use Rasuvaeff\LinkPreview\Fetch\UrlPolicy;

$policy = new UrlPolicy();
$policy->check(url: 'HTTPS://Example.COM:443/a#frag')->unwrap(); // 'https://example.com/a'
$policy->check(url: 'http://[::7f00:1]/')->error();               // UrlRejection::PrivateAddress
$policy->allowsAddress('169.254.169.254');                         // false: the same ranges, for a resolved address

// development stand: allow one network (localhost names count as 127.0.0.1)
$dev = new UrlPolicy(allowedNetworks: ['10.20.0.0/16'], allowedPorts: [80, 443, 8080]);

Internationalised hosts become ASCII (münchen.de -> xn--mnchen-3ya.de). On success check() returns the normalised URL.

allowsAddress(string $address): bool judges one resolved address - a plain IPv4 or IPv6 literal, no brackets, no port - exactly as check() judges a URL with that IP as its host, allowedNetworks included; anything else is false. It is the AddressPolicy of rasuvaeff/pinned-http-client: the bridge is the three-line anonymous class in Usage.

Fetcher

Many sites serve metadata only to crawlers they know: checked live, vk.com answers the default name with an endless redirect loop and championat.com with a login stub. UserAgentMap::unfurler() presents Twitterbot/1.0 everywhere (single hosts can still be overridden): pass it as userAgents: when you read arbitrary sites.

use Rasuvaeff\LinkPreview\Fetch\Fetcher;
use Rasuvaeff\LinkPreview\Fetch\UserAgentMap;

$fetcher = new Fetcher(
    client: $psr18Client,
    requestFactory: $psr17Factory,
    policy: new UrlPolicy(),
    userAgents: new UserAgentMap(byHost: ['t.me' => 'TelegramBot (like TwitterBot)']),
    maxRedirects: 5,
    maxBodyBytes: 2_097_152,
    chainBudgetSeconds: 10.0,
);

$page = $fetcher->fetch(url: 'https://example.com/')->unwrap(); // FetchedPage: url, body, contentType, charset, redirects
$json = $fetcher->fetchJson(url: 'https://api.example.com/x');  // application/json or +json

// an image of a preview: same policy, redirect checks and User-Agent, its own types and size limit
$media = $fetcher->fetchMedia(url: $info->image, acceptedTypes: ['image/*'], maxBytes: 5_242_880);

Redirects are followed by Fetcher, not by the client, and UrlPolicy judges every target before any request is made to it. A failure is a FetchFailure (reason, status, retryAfterSeconds, isTransient(); reasons as in Failures, without the preview-only MissingToken and EmptyPage). fetch() accepts only text/html and application/xhtml+xml. fetchMedia() replaces the Fetcher's maxBodyBytes with its own maxBytes, and the client's own limit still applies: a body over PinnedClient's maxBodyBytes is TooLarge as well. Give media a Fetcher over $pinned->withMaxBodyBytes($mediaMaxBytes) when media may be larger than pages. An empty 2xx body is EmptyBody. It takes media types (image/png) or families (image/*), lower-case, and throws InvalidArgumentException for an empty or malformed list or a limit below 1 - those are programming errors, not input. UserAgentMap matches a host and its subdomains.

Limits

use Rasuvaeff\LinkPreview\Limits;

new PreviewProvider(fetcher: $fetcher, limits: new Limits(titleLength: 120, descriptionLength: 300, httpsImagesOnly: true));

The package returns clean text. For title, description, siteName, authorName and providerName, Limits:

1. scrubs invalid UTF-8;
2. decodes HTML entities - repeatedly, so double escaping such as " comes out as ";
3. removes tags, comments and doctypes, each replaced by a space (Line one.<br>Line two. -> Line one. Line two.); a < that opens no tag (a < b, I <3 PHP) is text;
4. collapses control characters and runs of whitespace to one space;
5. and only then cuts what is too long by characters, with a trailing … that counts toward the limit (defaults: title 200, description 500, site and provider name 100, author 100).

Decoding and stripping repeat until nothing changes, at most four rounds; text escaped deeper than that gets its remaining & and < replaced by full-width ＆ and ＜, so the result is final either way. Applying Limits twice equals applying it once, and the output never contains an entity to decode or a tag. Escaped markup is markup: a title written as <b>bold</b> becomes bold. Every step is linear in the input.

An image, favicon or embed URL longer than maxUrlLength (2048) is dropped, an image together with its dimensions; with httpsImagesOnly plain-http ones are dropped too (the next source is not consulted after the merge). An over-long canonicalUrl is dropped as well.

Site adapters

PreviewProvider picks an adapter by the host of the URL (after the URL policy accepted it) and merges its answer in front of everything else. Hosts are matched label by label: www.youtube.com and m.youtube.com are YouTube, youtube.com.evil.example and notyoutube.com are not. If an adapter fails the generic extractors still run; if the page is unavailable too, a missing token is reported as PreviewFailureReason::MissingToken.

The built-in adapters are not public classes: they come from AdapterRegistry::standard(). Your own adapters (implement Adapter) go in front of them with with(); the first adapter whose domains match the host wins, so an adapter of yours for youtube.com replaces the built-in one.

use Rasuvaeff\LinkPreview\Adapter\AdapterRegistry;

$provider = new PreviewProvider(
    fetcher: $fetcher,
    adapters: AdapterRegistry::standard(
        instagramToken: '<token>',
        facebookToken: '<token>',
        twitchParents: ['example.com'],
    )->with(new MyIntranetWikiAdapter()), // asked first; then the eleven built-in ones
);

An adapter answers Result<Metadata, AdapterFailure> and reaches the network only through its AdapterContext (html(), json() - the Fetcher, under the URL policy). JSON answers of oEmbed, Wikipedia and Rutube are checked against their shapes before they are read: a field of the wrong type makes the answer unreadable (AdapterFailure::InvalidResponse) and AdapterContext::diagnostics() says where (oEmbed $.title: ...) without echoing a value.

Tokens are arguments of standard() (#[\SensitiveParameter]), travel as an Authorization: Bearer header to the API host only (a redirect to another host drops them) and never appear in a URL, a failure or an exception. Without a token, Instagram and Facebook answer AdapterFailure::MissingToken without making any request. new AdapterRegistry() turns adapters off.

The fixtures behind the adapter tests are in tests/fixtures/adapters/: they are written from the services' documented response formats, not live recordings. bin/record-preview-corpus --check re-parses every committed entry (no network); bin/record-preview-corpus --record=<name> --live refreshes one entry's files from the real service, and LINK_PREVIEW_LIVE=1 vendor/bin/testo --suite=Integration compares the real services with what the code expects.

Public API

Security

A link preview fetches a URL chosen by a user from inside your network. What the package does:

• Whitelist policy on the first URL and on every redirect target, checked before the request is sent; fromHtml() checks its pageUrl too.
• Alternative IP spellings, IPv6-embedded IPv4, localhost., userinfo tricks and control characters are rejected or classified as the address they really are.
• The HTML is parsed with libxml (LIBXML_NONET), never executed; external entities are not loaded; scripts are text.
• Response size is bounded; only the content types asked for are read.
• Every URL taken from the page is absolute http(s) without credentials. An oEmbed endpoint named by a page is fetched under the same policy and redirect checks as the page; a JSON-LD @context is never fetched.
• url is always where the preview was fetched from; what the page claims (og:url, canonical) is kept apart in canonicalUrl, so a page cannot make a card point elsewhere.
• Adapter hosts are matched by labels, not substrings; adapter tokens are sent as headers to the API host only and never appear in URLs, failures or exceptions.

What it cannot do, because PSR-18 does not give it the means:

• No DNS pinning in the package. The policy never resolves names; with a plain PSR-18 client a name that resolves to an internal address passes, and an attacker can rebind a name between the check and the connection. Close this in the transport: use rasuvaeff/pinned-http-client as in Usage (it resolves the host, checks every address with UrlPolicy::allowsAddress() and pins the connection; examples/with_pinned_client.php), or send the client's traffic through a trusted resolving proxy (an egress proxy that refuses internal destinations, e.g. Smokescreen).
• The client must not follow redirects (PinnedClient does not; allow_redirects => false for Guzzle, max_redirects => 0 for Symfony). A redirect followed inside the client skips the policy.
• Timeouts belong to the client. PSR-18 has none; configure connect and read timeouts on the client. chainBudgetSeconds is checked between hops only and cannot interrupt a request in flight.
• Body size is checked on the body the client returns; a client that buffers has already read it. Set a limit on the client too (PinnedClient's maxBodyBytes).
• Image URLs are not fetched by the preview. To download one, use Fetcher::fetchMedia(), which applies the same policy.
• Output is clean plain text, not safe markup. Escape it for the context you render it in.

Examples

See examples/ for runnable scripts. Examples are expected to execute without fatal errors and stay aligned with the documented public API.

Development

No PHP/Composer on the host - everything runs in Docker. The composer:2 image has neither ext-intl (IDN hosts, Psalm stubs) nor pcov, so make image builds a local image (link-preview-php:local) that has both:

make image
make install
make build
make cs-fix
make test
make test-coverage
make mutation
make release-check

License

BSD-3-Clause