The Reluctant Revolution?
My response to the DPP "The Reluctant Revolution"
Firstly I applaud the DPP and their “The Reluctant Revolution” paper, as sharing information is not something the media industry has classically been very good at. Every house having their own unique workflow, then complaining where they don't meet up. An open source workflow forum is a great idea, and something all industries can benefit from.
If you haven’t read it, take a look here: http://t.co/KIQu0er
Problems at the grass /root level:
I feel a big issue for the industry, especially the creative side, is how brittle and fragile the file based systems are. I commonly hear people complaining they have broken a folder structure, or cant read a MXF file for the lack of a decent cross camera and platform viewer. This is before they’ve even got anywhere near an NLE with their, at best, retrofitted plugin support for solid state acquisition formats. If you start throwing ‘prosumer’ cameras in along the lines of DSLR, GoPro and Flip cams, the lack of timecode, UMIDs or any other ‘broadcast’ metadata leaves the process tricky at best, transcode laden and hand cranked on average and broken at worst.
The complexity of delivery formats has column inches, but in reality this is usually handled by a post facility and/or could be outsourced by exporting a timeline native quicktime and dropping it to a dubbing house as 1 off per deliverable. Acquisition is a much larger problem, and one which if done right has a positive effect on the whole production process.
Where is the modern BITC VHS?
Storage Solutions:
Cloud is a fluffy term (groan) but I feel the only reason IT people feel they understand the term is that it’s been hijacked by the virtual x86 machine crowd. This is only one pocket, that I’d call IaaS. Comparing Amazon EC2 against a SaaS product is like orange trees to orange juice. A core value in the provisioning of cloud services is that you only pay for what you use, and it can scale up and more importantly down on demand.
In the Obstacles section of the paper there is a price comparison between cloud, LTO and LaCie (a name now as ubiquitous as the ‘hoover’). There is one key part missing! Management. 1 drive is easy enough to manage, 5 tricky, 10 you’ve no idea and by the time you’re at 100 you may as well bin them. With reality and ob-doc productions seeing extremely high shoot ratios, 100+ TB projects are not rare. Where is the cost of sorting, organising and finding in this equation?
LTO I consider to be more dangerous term than cloud, as while many industries use it to warehouse vast quantities of static data successfully, it is remiss to attach the same badge and trust to stand alone tape drive and umpteen china-graphed tapes sat on a shelf. I would consider the stand alone drive LTO solution as no better than a stack of LaCie disks, as I can’t easy attach a LTO drive to my MacBook and would need two tape drives if I want to copy anything without an intermediate step.
When you start to delve deeper into the TCO of LTO properly by considering: Reclamation - it’s linear after all, so remove a file in the middle of a tape and you need to rewrite that tape to efficiently use the space again. Error checking - backups never fail, only restores do. Last but not least, supporting and maintaining a proper tape management infrastructure. The numbers are not quite so rosy. Insert even light random access (conform maybe) and the equation gets very interesting.
Conspicuous in its absence was ‘spinning disk’ solutions. While I agree vendors tend to oversell banking sector hand-me-down features (dedup etc) that have little use to the media sector, spinning disk is still the key stone of any good tapeless workflow. However I’ll park this point and I know others have a lot more than me to say about it!
Cost of handling data:
Digibeta was a great format, as it could playback your rushes and serve as your delivery format too. In ’94 an A500 would have set you back £37k* (~£55k in todays money) and it was considered over a 10 year life that the TCO was roughly 2x purchase price*. So before stock you could consider that it cost £917 a month in todays money to have that sat in your machine room. While things have moved on since ’94, in a HD tape workflow you now need a one deck to ingest with (DVCProHD maybe) and another to master to (SR?!!) so the cost is still there, even if you choose to sub the work out. Oh how much does addison lee charge? * Phil Crawley - Chief Tech dude and all round good guy @ root6.
For your fibre internet cost, you are getting a transport layer that doesn’t care about ‘formats’, so no risk in ‘buying the wrong one’ and the ability to use it for a multitude of services. 100Mb is nearly realtime (with overheads) when dealing with AVCintra, so no slower than a tape deck yes doesnt need to be couriered or played in at the other side!
The real future in handling video assets as data in orchestration systems and networks. Being able to deliver media from your rushes store, to your graphics house whilst also delivering a selection of edit proxies to your edit workstation. All this is possible from a 3G connection, as there is no need for the data to flow via the machine you are managing from.
Moores law also relates to connectivity, we at Aframe have a 1Gb line for less than I paid (under 1k pcm) for a 10Mb line just 3 years ago...
50Mb Virgin Media
I've been with Virgin Media since Jan 2011, as who wouldnt want 50Mb in the home especially when Media in the cloud is your bread n butter! I we had a honeymoon period of 6 weeks where I'd marvel ar 5MB/s downloads, and enough speed to VPN & VNC into the office like I was sitting my desk at work. It was nice while it lasted...
However feb & march I started noticing pages were slow to load, and the problem got worse until I couldnt stream a 240p (the lowest quality setting) video from youtube. Speed test sites showed up & down performance but that's only half the story, the great info collected from Think Broadband showed the real picture. Their ping monitor keeps a record of the latency on your line to the router, so this is independent of the computer being on at home, and not susceptible to the issues with wifi.
Virgin Medias lines are all marketing hype, plastered over a substandard product with no support. You can chat to the nice guys at @virginmedia (dont bother with the call center) but in reality they can't get anything done, just token gesture partial refunds (can't get all your BB costs back). I'd be more than happy to go back to what I had with Sky, a solid 20Mb line that did what it was supposed to.
So I posted all of the above from my Three mobile 3G dongle, as it's more reliable and faster!
This is my first plot, main thing is max latency is wildly all over the place and a high level of jitter. However the connection was usable, but just slow.
By June this was the norm, however finally it was recognised that my line (and more importantly the edge network it's on) is grossly out of spec.
Check out that average latency, off the 140ms scale. Useless for anything TCP based.
Or it can be just plane dead (1 of 3 days in sep) with no warning or apology.
So as of today I have a bit of everything. Massive latency, packet loss, jitter.
iMeh – My iPad story
Ok I would protest at being called a Apple fanboi, but all the evidence does say otherwise. I do own a Macbook, iPhone (3rd one) and until recently a iPad. I purchase my iPad before it was released in the UK as I was in vegas for business (NAB), it was too shiney to resist, plus @davidpeto was chomping at the bit to get one too.
The screen is impressive, it's quick and beautifully built. But it's tiring to hold as you need to grip it hard as its so slippery, the keyboard is too spaced out to type quickly and the lack of iOS 4 is annoying. You can't store files independently of apps, and no printing. I thought it was going to be like my iPhone but better, but instead it's like a crap laptop, inherited the worst of both platforms.
The straw that broke the back was the SHITE wifi reception, meaning that I could only use it in the lounge really. Why not just use a real computer then? Suffice to say it's gone pretty much unused, so has been moved on. I'm sure the new owner will be less picky and enjoy it. 
Bring back the Apple Cube?
For me there is a hole in the current Mac lineup. The 27" iMac is great, but I dont want or need a new screen and even if I did I'd buy an Eizo. I could bleet on for ages but instead i'll boil it down to my fanboi wishlist.
- 2 PCI slots (one can have GFX card)
- 2 3.5" HDD slots
- SSD space
- 4+ RAM slots
- i7 Quad Core chip
- FW800/1600
- Plenty of USB2
- DVI & display port
Or do us a i7 powered entry level Mac Pro for i7 iMac money.
The death of RAID controllers?
In any data center you will see a raft of disk arrays, the majority either have RAID controllers (hopefully two of them) or are dumb JBODs with a RAID card along the lines of a LSI MegaRAID. The theory is all the same, abstract the physical disks with a RAID algorithm so they appear the host as one large LUN. The filesystem has no awareness of the underlying structure of the stripe or the physical hardware, and the majority of filesystems out there wouldn't do anything with the data if presented anyway. Then there is software raid (LVM, Widnows RAID etc), where the hardware array is replaced with a simple JBOD and HBA, but this simply moves the RAID overhead up the path a notch, onto the host OS and in doing so ties access to one node only.
Enter stage left, we have ZFS and RAIDZ. Heralded as one of the most advanced filesystem available today, ZFS is a real gem, it was a real shame when Apple dropped scraped support for it in 10.6. All striping is handled in the filesystem with pools, and pools of SSDs can be added for acceleration through the use of caching. Jeff Bonwick has written a great blog post about RAIDZ and the RAID5 write hole (the reason we have battery backup on controllers) and also a post on End to End Data Integrity of ZFS, covering re-silvering and how ZFS verifies its structure. The sun x4500/x4540 is the best case of this filesystem in action, a one (4U) box 48 disk storage system with big bandwidth to network, yet no hardware raid cards, just simple LSI SAS HBAs. It's all very clever, however ZFS is a DAS filesystem, not a SAN filesystem.
The first time I thought about the future of RAID controllers, as we know them ,was when testing some DDN gear. I noticed on the back of their 6620 60 disk array the FC ports on the controllers looks suspiciously like qlogic fibre HBA PCIe cards. On further inspection is seems the whole controller is a custom x86 server, and the host connection is just a matter of adding the relevant card (10Gb NIC for iSCSI for example). Their 9900 series goes one step further, using a 4 socket (Dell) server they can run the RAID on 2 of the sockets and allow a guest filesystem to run on the other two. Nice and close coupled, however it still isn't stripe / block level awareness for the filesystem.
Isilon's hardware could be compared to the X4500 in topology at a node level, however they expand their pool of disks outside of just one node into a cluster format, with infiniband stitching it all together. Their 'RAID' means they can protect against not only drive loss, but whole nodes. The power of the whole cluster can be leveraged in rebalancing to protect the data after a failure, in contrast to the time taken to rebuild a 2TB SATA disk in a RAID5/6 array where you are limited by the throughput of that one drive and completing the XOR. Nodes can be easily added and removed from the cluster and it is very easy to manage. The cons are: You can't have a client connected to the IB fabric, they must connect via the Gb Ethernet using normal protocols such as NFS / CIFS / SMB. For higher speed access you need the 10GbE accelerators and you are still bound by the speed limitations of the aforementioned protocols. Isilon is prefect where you have lots of GbE attached nodes wanting to hit the cluster (render nodes), where it falls down is where you want a few really high performance workstations or servers sharing data (uncompressed HD on ingest or edit system) requiring lots of accelerator nodes to be added. They have killed RAID controllers in their system, and harnessed the whole cluster topology into a very capable and easy product.
My final piece is on IBM's GPFS, which like Isilon's OneFS is a cluster filesystem, however their clients access the cluster via propriety protocol, where data is read and written to a pool of Linux servers over 10GbE DCE (Data Center Ethernet) rather than channeled through one server to the rest of the cluster on the IB backend. Therefore a single client can get very high transfer speeds by communicating in parallel, perfect for a network of very high power workstations. They currently still use hardware RAID to present 10 disks (8+2 RAID6) to the GPFS servers as a single LUN, therefore still have the rebuild time issues. However a little birdie tells me soon they will be setting all their arrays to run as JBODs, presenting each drive individually and the filesystem will handle the protection. The key difference between this and Isilon though is that the disks will be presented to all the servers on the fabric, not just one local node, which means the bandwidth is a lot more flexible. It's not announced yet, but I think 2010 could be a very exciting year for large storage systems.
To summarise; DAS solutions we have ZFS, clustered NAS we have OneFS, SAN we are seeing the the controllers being commoditised, and with the arrival of a new GPFS we are even seeing JBOD at a SAN level. There is no need for a layer between the disks and the hosts, so could we kill the JBOD off as well by bringing ethernet or iSCSI right to the HDD controllers? 3.5" disks with a RJ45 and PoE?
Density – More than just CPUs per 42U rack?
There are many new dense (more than two sockets per RU) solutions out there, however they all fall under one the 3 groups:
- Blade - Multiple (8-32) hotswap servers in one host chassis, sharing power and io via plugin modules
- Multi Motherboard - Two or more servers sharing the same PSU and fans, but with independent io
- Half depth - Reduced form factor servers to allow more in one rack, SGI (Rackable) / Verari / IBM iDataPlex
It occurred to me that while the new designs can cram a lot in rack, they don't necessarily make for a dense data center. I decided to look at how many sockets we get per floor tile (600x600) using 42U racks. Taking into account aisles (in a hot aisle - cold aisle config) and switching but not perimeter walkways. For example here is the benchmark using 1U pizza box servers.
1U Conventional Servers
4x 42U racks housing 160 dual socket 'pizza box' server = 320 Sockets
2U in each rack set aside for switching to be fair to the blade and iDataPlex systems
7 Tile Pitch - 4 racks = 14 Tiles
22.86 Sockets per rack tile
