The Future of Video

A presentation by @darcy


  • Independant UX Developer, Designer & Advocate
  • Based in Toronto, Canada
  • Twitter: @darcy
  • Github: @darcyclarke


  • Co-Founded Themify
  • Fi (Fantasy Interactive), Jet Cooper, Polar Mobile, Say Yeah!, Playground, Blast Radius, Bnotions, Daily Challenge, Pinpoint Social, My City Lives, Viafoura, Flixel
  • Google, Microsoft, Nike, Samsung, Porsche, Ducati, Disney, Red Bull, CBS, Panasonic
  • jQuery, DSS, Frontend Developer Interview Questions, Watch.js, Repo.js ...

My Last 12 Months

  • Documentation
  • Open Source
  • Standards & Accessibility
  • Progressive Enhancement
  • Feature Detection & Polyfills
  • Build Tools
  • Server-side Rendering
  • Language Abstractions & Preprocessors
  • Typography
  • Responsive Images
  • * Video

We'll Cover

History, Manipulation, Experiences, Encoders, Decoders, DRM & EME


Codecs, Containers & Playback

1986 - Sony release D1

  • First digital video recording format
  • Stored 94 minutes
  • Uncompressed

1988 - Sony & Ampex release D2

  • Offered simultaneous playback
  • Stored composite video over component
  • Still uncompressed

1988 - H.261 is released

  • First member of the H.26x family
  • Most all subsequent video coding standards are based on it
  • Used compression


Coder + Decoder

Compressor + Decompressor

Video Playback 1991 - 2007





The Aftermath...


February 28th, 2007

Opera Proposes <video> element

~ W3C Mailing List Thread Announcment

7 years later



Canvas Filters

Canvas Explosion

WebGL Filters

Three.js WebGL Materials

WebGL Face Detection

Going Further

Interactive Experiences

Realtime Color Histogram

WebRTC Chat Rooms


Arcade Fire: Just a Reflektor

Pharrell Williams: 24 Hours of Happy


Ryerson University: Peer Learning Experience

The Bleeding Edge

Encoders & Decoders

JavaScript Video Encoders





JavaScript Video Decoders



  • Narrow file requirments
  • Picky implementation
  • MPEG1 is a poor, out-dated format

WEBM: vp8-webm-js-decoder

WEBM: vp8-webm-js-decoder

  • Performance on desktop ~20-30fps
  • Performance on mobile <10fps



  • APNG is bloated
  • Generating files is painful

Sprite PNG + JSON: anim_encoder



  • IFO files are parsed to JSON
  • Chapters are generated as WebVTT
  • NAV packets are extracted to JSON
  • The buttons size/position are saved to CSS
  • The menu still frames are saved to PNG
  • VM commands are compiled into JavaScript
  • The video is encoded to WebM

Today I Saw the Future

May 3rd, 2013

Brendan Eich

Mozilla and OTOY deliver the power of native PC applications to the Web, unveil next generation JavaScript video codec for movies and cloud gaming


What makes this interesting?

ORBX.js Features

  • 25% better compression than H.264
  • Adaptive bit-rate (while streaming)
  • Better color depth
  • Better intra-frame coding
  • IP-blind runtime
  • Flexibile
  • Open Source

But there's more

  • ORBX.js will be free forever
  • Will be Open Sourced (potentially, later this summer)
  • Major studios are looking at ORBX to solve video-on-demand watermarking (DRM free)

Future Headaches

Digital Rights Management (DRM)

Encripted Media Extensions (EME)

Defective By Design on DRM

Digital Restrictions Management is the practice of imposing technological restrictions that control what users can do with digital media. When a program is designed to prevent you from copying or sharing a song, reading an ebook on another device, or playing a single-player game without an Internet connection, you are being restricted by DRM. In other words, DRM creates a damaged good; it prevents you from doing what would be possible without it. This concentrates control over production and distribution of media, giving DRM peddlers the power to carry out massive digital book burnings and conduct large scale surveillance over people's media viewing habits.


Encrypted Media Extensions provides an API that enables web applications to interact with content protection systems, to allow playback of encrypted audio and video.

The Future of Video is?

  • Encrypted Media Extensions
  • Further Exploration with Interactive Experiences
  • JavaScript Video Codec (ORBX.js)