How to Extract JSON-LD, Open Graph, and Microdata from Web Pages
Web ScrapingStructured DataJSON-LDTechnical SEOMetadataData Extraction

How to Extract JSON-LD, Open Graph, and Microdata from Web Pages

SScraper Studio Editorial Team
2026-11-14
7 min read

Learn how to extract JSON-LD, Open Graph, Twitter Cards, and microdata, then validate and export structured metadata for recurring SEO audits.

Structured metadata gives SEO teams and developers a machine-readable view of a page’s meaning, but it is often distributed across JSON-LD, Open Graph tags, Twitter Card tags, and HTML microdata. This guide explains how to extract JSON-LD from a website, collect the other metadata formats, resolve multiple schema blocks, validate the results, and export a consistent JSON or CSV dataset for recurring technical SEO checks.

Overview

When you scrape structured data, the goal is not simply to copy every tag into a file. A useful extraction workflow preserves the source information, identifies the format it came from, normalizes equivalent fields, and makes missing or conflicting values visible.

A single page may contain several script type="application/ld+json" elements, a set of meta property="og:*" tags, Twitter Card metadata, and microdata attributes such as itemscope, itemtype, and itemprop. These formats can describe overlapping concepts without using identical field names. For example, a page title might appear as a document title, an Open Graph title, a Twitter title, a JSON-LD name, and a microdata property.

A reliable process therefore has four layers:

  1. Fetch: retrieve the HTML that contains the metadata, using a browser-rendering tool when the page generates its tags with JavaScript.
  2. Locate: find JSON-LD scripts, relevant meta tags, and microdata nodes.
  3. Normalize: map different representations into a predictable internal schema while retaining the original values.
  4. Validate and export: check syntax, required fields, duplicates, and changes before writing JSON or CSV.

For larger audits, combine this workflow with a site crawl so that each URL receives the same checks. The guide How to Crawl a Website for Technical SEO Data with a Scraper covers the broader crawl design, while How to Clean, Normalize, and Deduplicate Data from Web Scraping is useful once records have been collected.

What to track

JSON-LD blocks

Start with every script element whose type is application/ld+json. Do not assume that one page has only one schema object. The parsed value may be an object, an array, or an object containing an @graph array. Store each block and record its position in the document if order helps with debugging.

Useful fields to retain include:

  • @context, @type, and @id
  • Page-specific values such as name, headline, description, url, and image
  • Entity relationships such as author, publisher, brand, offers, and review
  • The original JSON-LD text, parsing status, and any error message

Keeping the raw block matters. A normalized record is convenient for reporting, but the original value helps an engineer repair a broken parser or confirm whether a change came from the page or from your transformation rules.

Open Graph and Twitter Card metadata

To extract Open Graph tags, collect meta elements with a property beginning with og:. Common fields include og:title, og:description, og:url, og:type, and og:image. An image can have associated values such as width, height, and alternative text, so preserve repeated tags rather than overwriting them in a simple key-value dictionary.

Twitter Card tags generally use a name attribute, such as twitter:card, twitter:title, twitter:description, and twitter:image. Some sites use both property and name inconsistently. A practical extractor checks both attributes and records which attribute supplied the value.

Microdata

Microdata is embedded in the HTML tree rather than in a standalone script. Find elements with itemscope, read their itemtype, and collect descendants with itemprop. The value may come from different attributes depending on the element: text content for many elements, content for a meta tag, href for a link, or src for an image.

Nested scopes require care. An item may contain another item, and a property can point to that nested item. Represent nested relationships as objects rather than flattening everything into one dictionary. If your audit only needs presence checks, you can also maintain a simpler summary containing item types, property names, and counts.

A normalized field map

Use a field map that separates source-specific data from comparable values. For example:

  • url: canonical page URL, JSON-LD url, and Open Graph og:url
  • title: document title, JSON-LD name or headline, Open Graph title, and Twitter title
  • description: meta description, JSON-LD description, Open Graph description, and Twitter description
  • image_urls: all discovered image URLs, with their source format
  • types: every JSON-LD and microdata type found on the page
  • errors: malformed JSON, missing values, duplicate fields, and unsupported structures

Do not silently select a winner when values conflict. Store all candidates, then apply a documented preference rule for reporting. For instance, you might report a JSON-LD value separately from an Open Graph value and flag differences for review.

Cadence and checkpoints

A structured-data extraction project is most useful when it produces comparable snapshots. Run a baseline crawl after the initial implementation, then repeat it monthly or quarterly depending on how frequently templates, product catalogs, or publishing workflows change.

At each run, capture the crawl date, requested URL, final URL, HTTP outcome, rendering mode, parser version, and extraction status. These details distinguish a genuine markup change from a failed request or an outdated parsing rule.

Use checkpoints that answer operational questions:

  • Did the number of JSON-LD blocks change?
  • Did a page type or entity type disappear?
  • Are previously valid JSON-LD blocks now malformed?
  • Do canonical, Open Graph, and JSON-LD URLs still agree?
  • Did image URLs, titles, descriptions, or product-related values change?
  • Did a template release create duplicate or conflicting properties?

Export a detailed JSON file for debugging and a flat CSV for filtering or comparison. A CSV row can represent one URL, while nested values are stored as JSON text or separate columns. If you need a broader recurring workflow, see How to Schedule and Run Reliable Web Scraping Jobs.

How to interpret changes

Not every difference indicates a defect. A product page may legitimately change its price, availability, image, or review count. Focus first on structural changes and compare them with deployment history, content updates, and URL changes.

Separate findings into four categories:

  1. Collection failures: the page was unavailable, blocked, redirected unexpectedly, or rendered without the content your fetcher needs.
  2. Syntax failures: JSON-LD cannot be parsed, attributes are malformed, or a value uses an unexpected type.
  3. Completeness issues: a required business field or expected entity is absent.
  4. Consistency issues: equivalent values disagree across JSON-LD, Open Graph, Twitter Cards, microdata, or the canonical URL.

When JavaScript adds structured metadata after the first HTML response, compare a plain HTTP fetch with a browser-rendered capture. If the rendered page contains the data, use a tool such as Playwright for that URL class and record the rendering choice in the output. The Playwright web scraping guide provides a starting point for JavaScript-rendered pages.

Also expect legitimate variation between page types. A home page, article, product page, and category page should not necessarily expose the same properties. Build validation rules by template or content type instead of applying one rigid checklist to every URL.

When to revisit

Revisit this extraction workflow on a monthly or quarterly schedule, and immediately after a template migration, CMS change, JavaScript framework update, product-catalog import, or large content release. These events can alter markup even when the visible page appears unchanged.

Before each recurring run, review the field map and confirm that the schema types and properties important to your project are still represented. After the run, compare counts and normalized values with the previous snapshot. Keep a small set of representative URLs for manual inspection: one page for each major template, plus examples with nested entities, multiple images, and known edge cases.

A practical maintenance checklist is:

  1. Fetch representative pages with the same method used in production.
  2. Parse every JSON-LD block and retain raw input alongside normalized output.
  3. Collect Open Graph, Twitter Card, and microdata values without overwriting repeats.
  4. Compare structural counts, types, URLs, and key fields with the prior snapshot.
  5. Review new errors and conflicts before changing parser rules.
  6. Export JSON for investigation and CSV for audit reporting.
  7. Record the parser version and any intentional exceptions.

This approach turns a one-time website schema markup extraction task into a repeatable technical SEO data pipeline. It helps teams detect meaningful changes, preserve evidence for debugging, and update extraction rules when the site—not just the standards—evolves.

Related Topics

#Web Scraping#Structured Data#JSON-LD#Technical SEO#Metadata#Data Extraction
S

Scraper Studio Editorial Team

Technical SEO and Data Extraction Editors

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.