.Net / C#
Scrape URLs
Confused about "Scrape URLs"?
Let us know how we can improve our documentation:
LAST EDIT Jan 26 2021
- On This Page:
- User agents
This endpoint allows you to retrieve open graph information from a URL which you can then use to add images and a description to activities. For example:
1
const preview = await client.og('http://www.imdb.com/title/tt0117500/');Returns the following:
1
2
3
4
5
6
7
8
9
10
11
12
{
"url":"http://www.imdb.com/title/tt0117500/",
"images":[
{
"image":"https://m.media-amazon.com/images/M/MV5BZDJjOTE0N2EtMmRlZS00NzU0LWE0ZWQtM2Q3MWMxNjcwZjBhXkEyXkFqcGdeQXVyNDk3NzU2MTQ@._V1_UY1200_CR90,0,630,1200_AL_.jpg"
}
],
"type":"video.movie",
"title":"The Rock (1996)",
"site_name":"IMDb",
"description":"Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer. A mild-mannered chemist and an ex-con must lead the counterstrike when a rogue group of military men, led by a renegade general, threaten a nerve gas attack from Alcatraz against San Francisco."
}Some URLs from popular services such as Instagram, Youtube, etc. resolve to their CDN and they have signatures on them which expire after some time. There is no clear information on when these links expire. However, if you think your older data requires these links to stay valid, please consider refreshing the open graph data by calling this endpoint and updating your metadata.
User agents
Copied!Confused about "User agents"?
Let us know how we can improve our documentation:
Our scraper uses the following user agent, as well as the latest desktop and mobile user agents: getstream.io/opengraph-bot facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
If you plan to scrape any links to websites you can control, please ensure that this user agent isn't blocked and returns valid opengraph data, for the best results.