The Hard Problem of Rendering Tweets

swyx 2022-06-12

I’ve been unhappy with my tweet rendering strategy for a while - Twitter encourages you to use their heavy JS script to render tweets, which undoubtedly heaps all sorts of tracking on the reader, docks your lighthouse performance score by ~17 points, adds ~4 seconds to Time to Interactive, occasionally gets adblocked (so nothing renders!)

perf impact screenshots

https://pagespeed.web.dev/report?url=https%3A%2F%2Fswyxkit.netlify.app%2Fsupporting-youtube-and-twitter-embeds

https://www.webpagetest.org/result/220612_BiDcVR_5KK/1/details/#waterfall_view_step1

The solution, of course, is to render it yourself, hopefully on the server side.

Solution up front

You can see my Svelte REPL solution here and paste in your own data generated from any curl request:

curl "https://api.twitter.com/2/tweets?ids=1441050138806415369&tweet.fields=attachments,author_id,conversation_id,created_at,geo,id,in_reply_to_user_id,lang,public_metrics,referenced_tweets,text,withheld&expansions=attachments.media_keys,attachments.poll_ids,author_id,entities.mentions.username,geo.place_id,in_reply_to_user_id,referenced_tweets.id,referenced_tweets.id.author_id&media.fields=alt_text,duration_ms,height,media_key,preview_image_url,public_metrics,type,url,variants,width&poll.fields=duration_minutes,end_datetime,id,options,voting_status&user.fields=created_at,description,id,name,profile_image_url,username" -H "Authorization: Bearer $BEARER_TOKEN" | pbcopy

Use https://developer.twitter.com/apitools/api?endpoint=/2/tweets&method=get to construct your curl query; you’ll also need a $BEARER_TOKEN from a twitter developers app you have to set up separately.

The problem

However, a Tweet isn’t just a simple data object. Tweets can have polls, images, videos, quote tweets, likes, retweets, quote tweets, mentions, hashtags, threads/conversations, and on and on. This is a lot of product complexity to model and display correctly.

The Next.js team have put together a nextjs component with some nifty AST parsing and serverside cheerio automation: https://static-tweet.vercel.app/ but even here I found that it doesnt correctly handle video tweets and omits displaying retweets.

(use https://developer.twitter.com/apitools/api?endpoint=/2/tweets&method=get to construct your curl query

Basic display

I started out modeling the component in the Sveltejs REPL: https://svelte.dev/repl/7a576202df06467c957b8ff64dfb2e73?version=3.48.0

Part of this was just a mix of grabbing some relevant Nextjs code, but then making different design choices like taking Twitter’s actual SVG icons and displaying retweets. This was brain numbing and took a couple hours but wasn’t too hard.

More work can be done to add polls, images and videos but I chose to skip that for my basic implementation.

Tweet Body parsing

The text parsing became the tricky part.

Simple text like @swyx on “learning in public” Have a listen: https://t.co/L4VF9a8ukZ https://t.co/QnVqvu8zRc rendered on Twitter is enriched into

<div lang="en" dir="auto" class="css-901oao r-1nao33i r-37j5jr r-1blvdjr r-16dba41 r-vrz42v r-bcqeeo r-bnwqim r-qvutc0" id="id__9kn1r6pzdhd" data-testid="tweetText"><div class="css-1dbjc4n r-xoduu5"><span class="r-18u37iz"><a dir="ltr" href="/swyx" role="link" class="css-4rbku5 css-18t94o4 css-901oao css-16my406 r-1loqt21 r-poiln3 r-bcqeeo r-qvutc0" style="color: rgb(29, 155, 240);">@swyx</a></span></div><span class="css-901oao css-16my406 r-poiln3 r-bcqeeo r-qvutc0"> on “learning in public”
Have a listen: </span><a dir="ltr" href="https://t.co/L4VF9a8ukZ" rel="noopener noreferrer" target="_blank" role="link" class="css-4rbku5 css-18t94o4 css-901oao css-16my406 r-1cvl2hr r-1loqt21 r-poiln3 r-bcqeeo r-qvutc0" style=""><span aria-hidden="true" class="css-901oao css-16my406 r-poiln3 r-hiw28u r-qvk6io r-bcqeeo r-qvutc0">https://</span>buff.ly/3Qc0MIq</a></div>

This is a very basic part of the twitter experience so I wanted to model it properly.

Here is the research I did on available options:

using a series of regexes to do replacements https://stackoverflow.com/a/66127711/1106414
Twitter itself actually publishes a twitter-text library: https://github.com/twitter/twitter-text/tree/master/js but it uses core-js so is unsuitable for frontend usage
use Autolinker: http://greg-jacobs.com/Autolinker.js/api/#!/api/Autolinker

The test code I used was this:

var twitter = require('twitter-text') // https://github.com/twitter/twitter-text/tree/master/js
var Autolinker = require('autolinker')
const text = `
#hello < @world > Joe went to www.yahoo.com
@swyx on “learning in public”
Have a listen: https://t.co/L4VF9a8ukZ https://t.co/QnVqvu8zRc
  `
console.log('----twitter-text')
console.log(twitter.autoLink(twitter.htmlEscape(text)))

var autolinker = new Autolinker( {
    mention: 'twitter',
    hashtag: 'twitter'
} );

var html = autolinker.link( text );
console.log('----autolinker')
console.log(html)

which gets you:

----twitter-text
<a href="https://twitter.com/search?q=%23hello" title="#hello" class="tweet-url hashtag" rel="nofollow">#hello</a> &lt; @<a class="tweet-url username" href="https://twitter.com/world" data-screen-name="world" rel="nofollow">world</a> &gt; Joe went to www.yahoo.com
@<a class="tweet-url username" href="https://twitter.com/swyx" data-screen-name="swyx" rel="nofollow">swyx</a> on “learning in public”
Have a listen: <a href="https://t.co/L4VF9a8ukZ" rel="nofollow">https://t.co/L4VF9a8ukZ</a> <a href="https://t.co/QnVqvu8zRc" rel="nofollow">https://t.co/QnVqvu8zRc</a>
  
----autolinker

<a href="https://twitter.com/hashtag/hello" target="_blank" rel="noopener noreferrer">#hello</a> < <a href="https://twitter.com/world" target="_blank" rel="noopener noreferrer">@world</a> > Joe went to <a href="http://www.yahoo.com" target="_blank" rel="noopener noreferrer">yahoo.com</a>
<a href="https://twitter.com/swyx" target="_blank" rel="noopener noreferrer">@swyx</a> on “learning in public”
Have a listen: <a href="https://t.co/L4VF9a8ukZ" target="_blank" rel="noopener noreferrer">t.co/L4VF9a8ukZ</a> <a href="https://t.co/QnVqvu8zRc" target="_blank" rel="noopener noreferrer">t.co/QnVqvu8zRc</a>

I also found this small library http://blessanm86.github.io/tweet-to-html/ but it seems to use Twitter’s v1 API response which is useless for modern needs.

t.co unfurling - the failed attempt

Twitter’s t.co link shortening is user unfriendly because it adds tracking, latency, and makes the url opaque. (docs, docs)

To unfurl the t.co URL to something more user friendly, we could add an async process to send a ping to the t.co url, and get back the redirect header (you can use the followRedirects in the fetch API). Autolinker does not seem to support this or an async replacement function.

You can read Loige’s blogpost: https://loige.co/unshorten-expand-short-urls-with-node-js/ for the basic intuition and use his tall library:

import { tall } from 'tall'

tall('http://www.loige.link/codemotion-rome-2017')
  .then(unshortenedUrl => console.log('Tall url', unshortenedUrl))
  .catch(err => console.error('AAAW 👻', err))

There is also a url-unshort library to do this with retries and caching:

const uu = require('url-unshort')()

try {
  const url = await uu.expand('http://goo.gl/HwUfwd')

  if (url) console.log('Original url is: ${url}')
  else console.log('This url can\'t be expanded')

} catch (err) {
  console.log(err);
}

I ended up going with tall and postprocessing autolinker:

async function superautolink(text) {
  const urls = []
  var autolinker = new Autolinker( {
      mention: 'twitter',
      hashtag: 'twitter',
      replaceFn : function( match ) {
          if (match.getType() === 'url') {
                  const url = match.getUrl();
                  if (url.startsWith('https://t.co')) urls.push(url)
	  }
         return true
      }
  });
  
  var html = autolinker.link( text );
  for (let url of urls) {
    const unfurl = await tall(url);
    html = html.replaceAll(url, unfurl) // handle https://t.co links
    html = html.replaceAll(url.slice(8), unfurl) // handle raw t.co link text
  }
  return html
}
console.log('----autolinker')
superautolink(text).then(console.log)

which correctly unshortened the URLs

<a href="https://twitter.com/hashtag/hello" target="_blank" rel="noopener noreferrer">#hello</a> < <a href="https://twitter.com/world" target="_blank" rel="noopener noreferrer">@world</a> > Joe went to <a href="http://www.yahoo.com" target="_blank" rel="noopener noreferrer">yahoo.com</a>
<a href="https://twitter.com/swyx" target="_blank" rel="noopener noreferrer">@swyx</a> on “learning in public”
Have a listen: <a href="https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/learning-in-public-with-swyx/" target="_blank" rel="noopener noreferrer">https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/learning-in-public-with-swyx/</a> <a href="https://twitter.com/LastWeekinAWS/status/1535727356849135617/video/1" target="_blank" rel="noopener noreferrer">https://twitter.com/LastWeekinAWS/status/1535727356849135617/video/1</a>

However I found that I needed to run this unshortening in the browser and the tall library requires Node.js’ http module, so back to square one for me.

t.co unfurling - the simple way

A discovery I had in testing these unfurls was the sneaky way that Twitter makes it hard for you to unwrap the url. if you fetch('https://t.co/L4VF9a8ukZ'), you get back <head><noscript><META http-equiv="refresh" content="0;URL=https://buff.ly/3Qc0MIq"></noscript><title>https://buff.ly/3Qc0MIq</title></head><script>window.opener = null; location.replace("https:\/\/buff.ly\/3Qc0MIq")</script> which basically forces an in-place reload rather than using the proper HTTP redirect headers. annoying!

However, assuming Twitter’s redirect response is stable, you can exploit this.

fetch('https://t.co/QnVqvu8zRc')
  .then(res => res.text())
  .then(x => x.match("(?<=<title>)(.*?)(?=</title>)")[0]) // https://stackoverflow.com/a/51179903/1106414
  .then(console.log) // https://twitter.com/LastWeekinAWS/status/1535727356849135617/video/1

et voila…

Rendering images

The API forces you to perform a lookup, which isn’t the hardest thing in the world to do. (The CSS is harder)

aside: Nextjs gets the aspect ratio presentation wrong

Then it’s just a matter of getting some test cases:

4 images: https://twitter.com/swyx/status/1534575878025678848
4 images: https://twitter.com/swyx/status/1534109340072038400
3 images: https://twitter.com/swyx/status/1531330889535602688
2 images: https://twitter.com/swyx/status/1534793827528998913
1 image portrait: https://twitter.com/swyx/status/1535271645954588672
1 image landscape: https://twitter.com/swyx/status/1534998899592949760

I wasn’t super confident in my css grid ability so I blended some css grid with JS to represent the different layouts (particularly the 3-image layout):

<script>
export let tweet
export let data

let grid = {
	4: `
	"foo-0 foo-1" 100px
	"foo-2 foo-3" 100px
	/ 50% 50%;
	`,
	3: `
	"foo-0 foo-1" 100px
	"foo-0 foo-2" 100px
	/ 1fr 1fr;
	`,
	2: `
	"foo-0 foo-1" 200px
	/ 50% 50%
	`,
	1: `"foo-0" 100% / 100%`,
}[tweet.attachments.media_keys.length]


</script>
{#if tweet.attachments}
<div style={`margin-top: 0.5rem; display: grid; grid: ${grid}; gap: 1px; background-color: black; border: 1px solid black; overflow: hidden; border-radius: 5px`}>
	{#each tweet.attachments.media_keys as mediakey, index}
	<img style={`width: 100%; height: 100%; object-fit: cover; grid-area: foo-${index}`} alt="todo" src={data.includes.media.find(fullmedia => fullmedia.media_key === mediakey).url} />
	{/each}
</div>
{/if}

Rendering Video

The next thing to do is video. Nextjs doesnt handle it so we’ll have to figure this out.

For a single embedded video, Twitter provides a bunch of different bitrates:

      {
        "preview_image_url": "https://pbs.twimg.com/ext_tw_video_thumb/1532586917866266625/pu/img/WJt1sEy-ZmJChtCQ.jpg",
        "type": "video",
        "variants": [
          {
            "content_type": "application/x-mpegURL",
            "url": "https://video.twimg.com/ext_tw_video/1532586917866266625/pu/pl/5jCHdoj5JVcBsb4T.m3u8?tag=12&container=fmp4"
          },
          {
            "bit_rate": 950000,
            "content_type": "video/mp4",
            "url": "https://video.twimg.com/ext_tw_video/1532586917866266625/pu/vid/480x852/HMEUvm-JijggZFcf.mp4?tag=12"
          },
          {
            "bit_rate": 632000,
            "content_type": "video/mp4",
            "url": "https://video.twimg.com/ext_tw_video/1532586917866266625/pu/vid/320x568/H51WXWN-QV3UGnie.mp4?tag=12"
          },
          {
            "bit_rate": 2176000,
            "content_type": "video/mp4",
            "url": "https://video.twimg.com/ext_tw_video/1532586917866266625/pu/vid/720x1280/cn2RJQ2ZVY4ScC4Y.mp4?tag=12"
          }
        ],
        "public_metrics": {
          "view_count": 765
        },
        "width": 720,
        "media_key": "7_1532586917866266625",
        "height": 1280,
        "duration_ms": 90000
      }

I considered offering a custom player, but felt that wasn’t worth it. So I just offer a basic video control:

<video controls src="/static/short.mp4" poster="/static/poster.png" preload="none"> </video>

This was the best blogpost I found on the topic: https://blog.addpipe.com/10-advanced-features-in-html5-video-player/

Polls

Here’s a generic search for all polls: https://mobile.twitter.com/search?q=card_name%3Apoll2choice_text_only%20OR%20card_name%3Apoll3choice_text_only%20OR%20card_name%3Apoll4choice_text_only%20OR%20card_name%3Apoll2choice_image%20OR%20card_name%3Apoll3choice_image%20OR%20card_name%3Apoll4choice_image%20&src=typed_query&f=top

Polls can have:

It wasn’t too bad:

<script>
export let tweet
export let data
const pollid = tweet.attachments.poll_ids[0]
const polldata = data.includes.polls.find(poll => poll.id === pollid)
let winningChoice = null
const totalvotes = polldata.options.reduce((a,b) => {
	if (!winningChoice) winningChoice = b
	else if (winningChoice.votes < b.votes) winningChoice = b
	return a + b.votes
}, 0)
</script>

<ul style="position: relative; list-style-type: none; padding-left: 0">
{#each polldata.options as option}
{@const percent = option.votes / totalvotes}
	{@const isWinning = winningChoice.position === option.position}
	<li style="position: relative; height: 1rem; margin-top: 1rem">
		<div style={`height: 1.5rem; border-radius: 5px; background-color: ${isWinning ? `rgba(29, 155, 240, 0.58)`: 'rgba(51, 54, 57, 0.3)'}; width: ${percent * 100}%`}></div>
		<div style="width: 100%; position: absolute; top: 0; display: flex; justify-content: space-between; padding-left: 0.25rem">
				<span>{option.label}</span>
				<span>{#if isWinning}
					🏆
					{/if}{Math.round(percent * 1000)/10}%</span>
		</div>	
	</li>
{/each}
</ul>
<div>
	
	<p class="tweettime tweetbrand svelte-145fr9">
{totalvotes} votes ending {new Date(polldata.end_datetime).toDateString()}
	</p>
</div>

Other Twitter features

There are Twitter Lists, Twitter communities, Twitter Spaces, and others, that I don’t handle well, but at least it doesnt look actively horrible:

To handle this well I would have to write an “unfurl” module to unfurl quote tweets and these other features. Can’t be bothered right now.

The Hard Problem of Rendering Tweets

Latest Posts