?

Log in

No account? Create an account

Downloading the Icelandic evening news

« previous entry | next entry »
Aug. 26th, 2015 | 11:11 am

Just on the off chance that someone else is interested, here's a shell script to download the Icelandic evening news. There's no magic involved, just a bit of automation to figure out the right resource name for a given date; the actual downloading's done using rtmpdump.

#!/bin/bash

DATE=$1
EPOCHSECONDS=`date -d "${DATE} UTC" +%s`
EPOCHDAYS=$((EPOCHSECONDS / 86400))
STREAMID=$((4723254 + EPOCHDAYS))
TAG=${STREAMID: -2:2}
STREAMID2=0

until rtmpdump -V -o ${DATE}.mp4 -n sipvodfs.fplive.net -a sipvod -y mp4:opid/${STREAMID}R${STREAMID2}.mp4
do
        if [ "$?" -eq 2 ]
        then
                break
        fi
        STREAMID2=$((STREAMID2 + 1))
        sleep 1
done

mv -f ${DATE}.mp4 ${DATE}_${TAG}R${STREAMID2}.mp4

Invoke it as e.g. ./dlfrettir.sh 2015-08-25, and you'll eventually end up with a 348 MiB file called 2015-08-25_26R10.mp4.

A few notes:

  • Resources are named e.g. 4739918R0.mp4 on the server; the number before the R is sequentially assigned for each new day, the number after is apparently random. (It's often 0, but not always. Numbers exceeding 20 are rare, and I've never seen it go beyond 30.)
  • Broadcasts remain available online for 90 days or so before they're memory-holed.
  • rtmpdump exits with status 0 upon a successful download, 1 if the resource wasn't found, and 2 if the user interrupts the download.
  • The script, as it is, outputs a lot of debug info; remove the -V flag in the rtmpdump call if you'd rather not have that.

Have I mentioned I really dislike shell scripting? So inelegant. (But it gets the job done.)

Link | Leave a comment | Share

Comments {2}

The Mystery of the Supranational Rabbit

(no subject)

from: porsupah
date: Aug. 26th, 2015 12:17 pm (UTC)
Link

It's at least simpler than the hoops get_iplayer has to jump through. =:) And that's only steadily getting worse, with the BBC migrating their feeds over to a different back end architecture for a new public API that'll involve authorisation tokens. Still, it does continue to work, and they've not taken any active steps to block the utility, unlike Channel 4. (Who at least had a sense of humor about it - if the client was deemed unauthorised, whatever you requested, you'd be served the same episode of the quiz show Countdown)

Reply | Thread

Schneelocke

(no subject)

from: schnee
date: Aug. 26th, 2015 12:30 pm (UTC)
Link

Aye, they're not putting up any real barriers, other than using RTMP (and that's not a barrier anymore these days either, much less an intentional one).

As for the BBC... I'm told a large share of their income's derived from selling their shows abroad, especially to the USA, right? I'm pretty sure the same's not true for RÚV, since there's hardly anyone outside Iceland who'd speak Icelandic in the first place. :) And I imagine that to the extent that they're giving any thought to global audiences at all, they're figuring that viewers from outside Iceland are either Icelandic travellers or expats, or people interested in Iceland, Icelandic culture and the Icelandic language, and all those are folks they most likely want to give access to anyway.

I only wish they'd use a better video codec, or optimize what they have; the videos are inevitably large, and the quality's fairly bad compared to what you'd expect these days. Full HD it ain't.

Reply | Parent | Thread