The Social Crawler Isn't a Browser — and That's Your Bug

The Preview That Doesn't Exist Yet

You publish a page and share it immediately. The preview shows no image, or the wrong one, or a title from an unrelated page.

The tags are correct. You can see them in the source. The problem is that the platform fetching your page and the browser you tested in are doing very different things — and understanding what the crawler actually does explains almost every Open Graph failure.

The Crawler Is Not a Browser

When a link is shared, the platform sends a bot to fetch the page. That bot has significant limitations:

It usually doesn't execute JavaScript. Meta tags injected client-side — by React Helmet, Vue Meta, or any framework doing head management in the browser — are often invisible to it. The tags need to be in the HTML the server sends.

This is the single most common cause of missing previews on JavaScript-heavy sites. The page looks correct when you view source in a browser with JS enabled; the crawler sees the pre-hydration HTML.

It reads a limited amount of the document. Some crawlers stop after a fixed number of kilobytes. Open Graph tags buried below a large block of inline CSS or a long script may never be reached. Put them near the top of the <head>.

It has a timeout. A slow-responding page may be abandoned before the tags are read.

It has its own user agent. If your CDN, WAF or bot protection blocks unknown user agents, it may be blocking the crawlers. Check your logs for the platform crawler user agents and confirm they're getting 200 responses.

It won't authenticate. A page behind a login, a paywall, or geographic restriction returns whatever an anonymous request gets.

It respects robots.txt. A page or an image directory blocked in robots.txt can prevent the crawler from fetching what it needs.

The Image Is Fetched Separately

This trips people up: the crawler makes a second request for the image URL after parsing the tags.

That means the image itself must be:

At an absolute URL. Relative paths in og:image don't work. The full URL including protocol and host.

<!-- Broken -->
<meta property="og:image" content="/images/preview.jpg">

<!-- Correct -->
<meta property="og:image" content="https://example.com/images/preview.jpg">

Publicly accessible. No authentication, no signed URL with a short expiry, not blocked in robots.txt.

Served over HTTPS if your page is. Mixed content gets blocked.

Not behind a redirect chain. Some crawlers follow redirects; some don't. Link directly.

Within size limits. Platforms impose maximum file sizes, commonly around 5–8 MB. A large uncompressed PNG can exceed this.

In a supported format. JPEG and PNG are universally safe. WebP and AVIF support is inconsistent across platforms — if you serve modern formats, provide a JPEG or PNG specifically for og:image.

Adding explicit dimensions helps the platform lay out the preview before fetching:

<meta property="og:image" content="https://example.com/preview.jpg">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta property="og:image:type" content="image/jpeg">
<meta property="og:image:alt" content="Chart showing quarterly growth by region">

The 1200×630 dimensions give a 1.91:1 ratio, which is the widely-used standard for the large card format.

Caching Is Aggressive

Once a platform has fetched your page, it caches the result — often for a long time. Publishing a correction doesn't update the preview.

This produces the specific frustration of fixing a broken preview and seeing no change. The tags are right; the platform is showing you a stored copy.

Options:

Use the platform's debugger tool. Most major platforms provide one that re-fetches and refreshes the cache. This is the intended mechanism and the fastest fix.

Change the URL. Adding a query parameter creates a new cache key. Effective but pollutes your URLs and splits any social engagement metrics.

Wait. Caches expire eventually, but the interval varies by platform and can be days.

The practical implication: verify your tags before the first share. Once a URL has been fetched with broken tags, you're in cache-busting territory. Check with a debugger tool on a staging URL, or on the live URL before you post it anywhere.

Platform Differences

Open Graph is a Facebook-originated standard that other platforms adopted with variations.

X/Twitter reads its own twitter: tags in preference to og: ones, falling back to Open Graph where they're absent. The twitter:card value determines the layout:

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="...">
<meta name="twitter:description" content="...">
<meta name="twitter:image" content="https://example.com/preview.jpg">

Without summary_large_image, you get the small square thumbnail regardless of how good your image is. This is the most common cause of "my perfect 1200×630 image shows as a tiny square."

Note the attribute difference: Twitter card tags use name=, Open Graph uses property=. Most parsers tolerate either, but following each spec avoids edge cases.

LinkedIn caches particularly aggressively and has historically been the hardest to refresh.

WhatsApp has tighter image size constraints than most and may skip images above a certain file size.

Slack and Discord each implement their own unfurling with their own quirks and their own caches.

Messaging apps generally fetch on behalf of the sender, which means a preview may be generated from a request originating in a different country to yours — relevant if you geo-restrict content.

The Minimum Viable Tag Set

The Open Graph Generator produces the standard block:

1. Enter your title, description, image URL and page URL.
2. Generate the tags.
3. Place them near the top of the <head>.

The tags that matter:

<meta property="og:title" content="...">
<meta property="og:description" content="...">
<meta property="og:image" content="https://example.com/preview.jpg">
<meta property="og:url" content="https://example.com/page/">
<meta property="og:type" content="article">
<meta property="og:site_name" content="Example">

og:url should be the canonical URL. This is what platforms use as the cache key and what engagement counts attach to. If the same content is reachable at several URLs, pointing them all at one canonical og:url consolidates shares rather than splitting them.

og:title should be distinct from your <title> tag where it makes sense. A search result title and a social share headline serve different contexts — search titles often include a brand suffix that's redundant in a social card.

Designing the Image

The image is doing most of the work in the preview, and a few constraints apply:

Text must be large. Previews render small, particularly in mobile feeds. Text sized for a desktop view of the image will be illegible.

Keep content away from the edges. Different platforms crop to different aspect ratios. A safe zone with margin prevents important elements being cut.

Don't rely on the description being read. Many layouts truncate it heavily or omit it entirely.

Test at actual display size. Scale the image down to roughly 500 pixels wide and check whether it still reads.

Generate them programmatically if you publish frequently. Templating an image with the article title, an author name and a consistent visual identity is far more sustainable than designing each one by hand, and produces more consistent results.

A Verification Checklist

Before sharing any URL:

1. View the raw HTML — curl the URL or use view-source, not the rendered DOM. Confirm the tags are in the server response.
2. Check the tags are near the top of the <head>.
3. Open the og:image URL directly in a private window. It must load without authentication.
4. Confirm the image is under the size limit and in JPEG or PNG.
5. Check og:url matches your canonical URL.
6. Add twitter:card if you want the large image layout on X.
7. Run the platform debugger for wherever you're sharing.
8. Check your logs for crawler user agents returning 200, not 403.

FAQ

Why doesn't my Open Graph image show up? Most often a relative URL, an image behind authentication, tags injected by client-side JavaScript, or an image exceeding the platform size limit.

Why does my updated image still show the old one? Platform caching. Use the platform's debugger tool to force a re-fetch.

Do Open Graph tags affect SEO? Not directly. They control social link previews, which is a separate system from search snippets.

Why does my image show as a small square on X? Missing twitter:card set to summary_large_image. Without it, the small summary layout is used.

Does the crawler run JavaScript? Generally no. Tags must be present in the server-rendered HTML.

What image size should I use? 1200×630 is the widely-supported standard for the large card format, giving a 1.91:1 ratio.

The Takeaway

Open Graph failures are almost never about the tags being wrong — they're about the crawler not being able to see them or fetch what they point at. Server-rendered tags near the top of the head, an absolute and publicly accessible image URL, and a debugger check before the first share resolves the overwhelming majority of cases.

Generate Open Graph tags free with the Open Graph Generator at sadiqbd.com — no sign-up, instant results.