It’s over a year since my blog post entitled “Quick and Dirty DVR” and I’ve been using and tweaking the code ever since. What I have now is a pretty autonomous and functional video recorder based around that core record script.
What I’ve added
- Scheduling & collision detection
- File management
- Ad detection
- Remote scheduling
The first thing was to get the script to find and record TV shows automatically. To do this, the script needs to know:
- A source of TV listings
- A list of shows that I watch
- The last episode number of a given show that has already been recorded
- When all subsequent shows are on, in the future
The first step is easy – just install the XMLTV package and set up a cron tab:
0 2 * * * /usr/bin/tv_grab_uk_rt --quiet > /share/tv/listings.xml. This will check TV listings nightly.
The list of TV shows is simply a flat text file – one show title per line. I took the decision to allow regular expressions in show titles when the show name starts with “like”. For example, QI and QI XL are largely interchangeable so the entry for QI is “like QI( XL)?”, meaning the “XL” part can be there or not but it will be considered to be the same show.
Then I just have to write something to work out which TV shows I already have. It’s very helpful to have your TV neatly organised for this. All my shows are in the path
/share/tv/[name of tv show]/[name.of.tv.show].s[series]e[episode].[subtitle].[extension], so inside tv.func.php is the function
getLastEpisodeOf($name,$dir) which returns text describing which episodes of that show should be recorded, e.g. “Peep Show after season 8 episode 2”.
Next the listings file needs to be parsed. For a while I had some very dumb parsing which went through each line of the subscriptions file and picked the first available episode of whichever show it was searching for. This was flawed as the scheduler wasn’t aware of other shows being recorded and therefore clashes in scheduling were inevitable. The most I could do was to detect the collisions between shows after the schedule was written and manually correct them.
After a bit of thought I replaced the scheduler with a new script which takes the order in the list of TV shows file as a priority for each show and checks that any new show attempting to be recorded does not clash with a show already scheduled. If this happens, alternative times are checked and used. The presence of “+1” channels makes this a lot more robust. If a show doesn’t have a suitable recording slot, it’s dropped from the schedule and a warning is produced in the crontab output as a comment.
This is an extract of my subscriptions file:
Fresh Meat The Killing Peep Show Wallander Bang Goes the Theory Doctor Who Sherlock Top Gear The Simpsons like QI( XL)? like Have I Got.* News For You The Snowman and The Snowdog The Royal Institution Christmas Lectures
And this is an extract of how the recording schedule looks for the Christmas 2012 period:
# Clash: The Royal Institution Christmas Lectures s1e1 Royal Institution Christmas Lectures 2012: The Modern Alchemist: Air: The Elixir of Life on BBC Four for 61 at 2012-12-26 20:00-21:01 with EastEnders s1e4556 2012-12-26 Wednesday on BBC One for 31 at 2012-12-26 20:30-21:01
# Clash: Tangled on ITV1+1 (Central) for 111 at 2012-12-25 16:10-18:01 with Doctor Who s7e6 The Snowmen on BBC One for 61 at 2012-12-25 17:15-18:16
# No suitable schedule found for Tangled
# Peep Show after Season 08 episode 04
30 22 23 12 * root /home/iain/bin/record 'Peep Show s8e5 Chairman Mark on Channel 4 for 31' # Jeremy is desperate to avoid living with Super Hans and rekindles his relationship with Mark's sister Sarah, moving in with her and her five-year-old son Joshy. In an attempt to deal with a damp patch on his bedroom wall, Mark launches a campaign to be elected chairman of Apollo House's freehold committee. Starring Robert Webb and David Mitchell. The final episode is tomorrow at 10pm.
00 22 24 12 * root /home/iain/bin/record 'Peep Show s8e6 Quantocking II on Channel 4 for 31' # Dobby is offered a job in New York by ex-boyfriend Simon, but is uncertain whether to accept the offer. She suggests to Mark they should go for a weekend break in the country, while Jeremy decides it is time to get a few things off his chest. Comedy, starring David Mitchell and Robert Webb. Last in the series.
# Doctor Who after Season 07 episode 05
15 17 25 12 * root /home/iain/bin/record 'Doctor Who s7e6 The Snowmen on BBC One for 61' # Matt Smith returns as the Time Lord, who is in mourning after losing Amy and Rory to the Weeping Angels and determined to avoid getting mixed up in the universe's problems. But a call for help whisks him back to Christmas Eve 1892, where a trio of old friends and a plucky young governess called Clara need him to take on a chilly menace that comes with the snowfall. Will he be persuaded to abandon his new life as a recluse and defend his beloved Earth once more? Jenna-Louise Coleman joins the Doctor as his new companion - although whether Clara is the same character she played in Asylum of the Daleks remains to be seen - with guest stars Richard E Grant (Withnail & I) and Tom Ward (Silent Witness).
# like Have I Got.* News For You after Season 44 episode 09
00 21 21 12 * root /home/iain/bin/record 'Have I Got News for You s44e10 Christmas Special on BBC One for 31' # Actor Daniel Radcliffe, perhaps still best known to most as boy wizard Harry Potter (despite having laid down his wand 18 months ago) is tonight's host, taking charge of Muggle team captains Ian Hislop and Paul Merton, and panellists Andy Hamilton and Sara Cox.
45 23 25 12 * root /home/iain/bin/record 'Have I Got a Bit More News for You s44e10 Have I Got a Bit More News for You Christmas Special on BBC One for 46' # Actor Daniel Radcliffe, perhaps still best known to most as boy wizard Harry Potter (despite having laid down his wand 18 months ago) is tonight's host, taking charge of Muggle team captains Ian Hislop and Paul Merton, and panellists Andy Hamilton and Sara Cox.
30 19 27 12 * root /home/iain/bin/record 'Have I Got News for You s44e11 Have I Got News for You 2012 on BBC One for 31' # A compilation of highlights from the past year, remembering how Paul Merton, Ian Hislop and a variety of guest hosts and panellists took on the big news stories of 2012. Last in the series.
# Homeland after Season 02 episode 11
00 21 23 12 * root /home/iain/bin/record 'Homeland s2e12 The Choice on Channel 4 for 91' # Feature-length conclusion of the second series of the Emmy award-winning thriller. Carrie thinks about returning to the CIA, but wonders if a career in the intelligence service is really for her, and Nicholas Brody meets with Faber and considers his family's future. Meanwhile, Saul is ordered to undertake a secret mission and Quinn makes a momentous decision. Last in the series.
# The Snowman and The Snowdog
00 20 24 12 * root /home/iain/bin/record 'The Snowman and the Snowdog on Channel 4 for 31' # Animated sequel to Raymond Briggs' classic festive tale The Snowman, telling the story of another youngster's magical Christmas. A boy's snowman and snowdog come to life at the stroke of midnight and take him on an adventure to the North Pole, where he and his new companions meet an assortment of colourful characters, including Santa himself, before returning home - where a wonderful surprise awaits.
# The Royal Institution Christmas Lectures
55 02 27 12 * root /home/iain/bin/record 'The Royal Institution Christmas Lectures s1e1 Royal Institution Christmas Lectures 2012: The Modern Alchemist: Air: The Elixir of Life on BBC Four for 61' # Peter Wothers explores the scientific elements using a periodic table made from audience members at the Royal Institution in London, to help uncover what the medieval alchemists knew about the air people breathe. In his investigation, he reveals these elements can be used to control fire, defy gravity and harness the power of a lightning storm.
00 20 27 12 * root /home/iain/bin/record 'The Royal Institution Christmas Lectures s1e2 Royal Institution Christmas Lectures 2012: The Modern Alchemist: Water: The Fountain of Youth on BBC Four for 61' # Dr Peter Wothers investigates whether drinking water can restore his youth, and discovers how exploding balloons could help solve the energy crisis. The presenter is also joined by Paralympic gold medal-winning cyclist Mark Colbourne as they try to find out what happens when two of the most reactive elements on the periodic table, caesium and fluorine, meet.
00 20 28 12 * root /home/iain/bin/record 'The Royal Institution Christmas Lectures s1e3 Royal Institution Christmas Lectures 2012: The Modern Alchemist: Earth: The Philosopher'\''s Stone on BBC Four for 61' # Dr Peter Wothers explores the elements within the earth and investigates whether it is possible to extract the world's most valuable minerals from them. He discovers how carbon dioxide can be turned into diamonds and attempts the ambitious feat of turning lead into gold. Joined by Professor Sir Harry Kroto, the pair find out what happens when you set fire to a diamond, and establish whether a member of the audience is worth their weight in gold. Last in the series.
Adding a comment line with the show name and previous episode makes it reasonably human-readable and tacking on a comment at the end of the cron line allows you to add in the episode synopsis. Works pretty well.
Smoothing the process
This is nice, but unless you have file name OCD, you won’t have a nice neat folder of TV shows. That’s why in the source of some of those scripts there’s reference to show_organise.php. This looks at any video file in a target folder, parses out the name, episode number and subtitle and then moves it into the TV shows folder in a sub-folder with the name of that show. It then writes a symlink to that file back to a “queue” folder so that there is one location to look for unwatched TV shows. Then is looks at the database file for XBMC and checks which of the shows in the queue folder have already been watched and it deletes the symlink for those shows. The queue folder now represents an unattended churn of just shows that haven’t been watched.
No one likes TV ads
Watching recorded TV is great for being able to fast-forward over the ads, but who wants to have to put their drink down and reach for the remote to do that? I found a reasonably reliable solution for skipping UK TV ads on the mythtv wiki. This ignores most ad detection tactics – looking for a network logo in the corner etc and takes advantage of a feature of UK commercial TV: All ad breaks are preceded and ended with tiny block of silence. The example script then takes the audio from a recording and uses mp3split (designed for splitting continuous audio into separate tracks) and uses the position of those splits to determine which are ads and which are the actual TV show. I tweaked this script and included it at the end of my record script so that it outputs edit decision list files which XBMC will automatically pick up and use in order to skip over ad breaks with no user intervention. Sweet, sweet sanity.
This isn’t quite Sky+ though is it? I mean, with Sky+ you get an app which lets you schedule TV shows to record from your phone. You can however, get pretty close with a bit of Dropbox (or cloud storage service of your choice) trickery.
Step 1: Add your subscriptions file to Dropbox. I’m using the linux CLI client.
Step 2: Symlink from that file back to where the scheduler is looking, e.g.
ln -s ~/Dropbox/subscriptions.txt /share/tv/subscriptions
Step 3: Monitor that file for changes. inotify is the swiss army knife of scripting when it comes for monitoring for file changes but it’s less straightforward if you want to set up what is effectively a service running constantly against a file. I did find a neat little script called when-changed, however which simplifies this massively. I just then have it set up so that whenever my subscriptions or listings file is changed, it re-runs the scheduler.
This is the command which I have set up:
/home/iain/bin/when-changed /share/tv/subscriptions /share/tv/listings.xml -c "/home/iain/bin/tv_priority_schedule | tee /etc/cron.d/tv-schedule /home/iain/Dropbox/tv-schedule.copy.txt" &
This checks the subscriptions and listings files for changes and runs the TV scheduler if it finds any. The output is a crontab file which it writes to cron.d but also creates a copy in my Dropbox. Now I can edit my listings in Dropbox on my phone and see the resulting TV schedule update a short while later, also on my phone.
Of course, if I wanted I could also have it automatically compress recordings and have them dropped into my Dropbox as well for remote watching, but I think that’s for a time when I have a fatter upstream…