Storage-Interfaces

      No Comments on Storage-Interfaces

Well now that we have established the idea that SATA drives are usually the ideal choice, we have to deal with the next logical question:  How should we go about connecting a whole bunch of these drives to our editing system?  The primary considerations I will be examining are cost, throughput, reliability, and shared access.  The most popular solutions, offered by multiple vendors, are SCSI, Fibre Channel, Ethernet, iSCSI, eSATA and the recently implemented External PCIe.  There are few other proprietary options available, but those are the ones that are widely available.

Let’s start with SCSI, since it is the easiest to dismiss.  While we are discussing the connection of SATA drives, many of the first generation SATA arrays had intergrated controllers and Raid hardware, and then needed a fast connection to the host.  These arrays were designed to replace much more expensive SCSI drive based arrays, so the target customers trusted the SCSI interface, and already had high end SCSI controllers in their systems.  That made SCSI the optimal connection solution for early SATA arrays.  The SATA Raid controller masquerades the entire array as a single SCSI disk, allowing connection to systems through existing SCSI cards.  With up to 320MB/s of bandwidth, a single SCSI channel can efficiently support 5-7 SATA disks without much impact on performance.  The biggest reason to dismiss SCSI as a serious possibility is that eSATA is a better option for most, and the remaining will be much better served by a Fibre Channel interface, allowing for economical upgrading to a full SAN in the future.
Continue reading

Storage-Hard Drives

      No Comments on Storage-Hard Drives

There are many different issues that need to be taken into consideration when evaluating data storage options for post-production.  Cost is a primary factor to consider, since without that issue, there would be little to weight against high end systems.  Performance is key as well, for efficiency as well as creativity.  Reliability is a frequently undervalued consideration, that effects everything from insurance prices to stress levels.  The ability to share data effectively can be an important consideration, since post-production is usually a collabrative effort.

The industry has responded with many different solutions, that vary in concept beyond recognition and in price by many orders of magnitude.  The earliest solutions involved video tape, analog replaced by digital recording.  Hard disks were introduced for random access to data, and now those are slowly beginning to be replaced by solid state flash chips.  Since this site is targeted to PC users, we will focus on hard disk based solutions, and the interfaces with which they can be accessed by a media workstation.

Hard disks are produced with five popular interfaces:  IDE/ATAPI, Serial-ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), and Fibre Channel (FC).  IDE and SCSI interfaces are currently being phased out and replaced by their more capable and flexible Serial varients.  I know little of true Fibre Channel hard disks, but that format is rarely used in this industry.  That leaves only two options, which are now somewhat similar and compatible, SATA and SAS.  With identical connection cables, and both offered in 3.5″ and 2.5″ form factors, it is hard to tell the two options apart visually.  Their interfaces both support 300MB/s, dedicated buses for each drive, and port splitting when that is not required.
Continue reading

Offline/Online Paradigm

      1 Comment on Offline/Online Paradigm

There are those out there, primarily DV and HDV shooter/editors that don’t understand the fundamental concepts of the offline/online process for high resolution post-production.  This used to be required for HD content, but computer technology has advanced to the point that 2K and 4K Digital Intermediates are the primary workflows requiring an offline/online post solution.  That being said, the release of the Red One, and a few other new digital cinema cameras, are pushing new people into this unfamiliar territory.  I originally wrote the following piece in a post on reduser.net and decided to post a polished version here as well.  It is designed to introduce DV and HDV editors to the principles and workflow of the online conforming process.

I will try to explain with this analogy.  Most HDV cameras allow you to capture HDV recordings to downconverted DV files in realtime, for backwards compatibility with older NLEs.  Imagine you shot footage in HDV, but you want to offline in DV, and online in HDV.  You capture your HDV footage into a DV project one full tape at a time, and are left with bunch of DV avi files.  You edit the entire piece, and export the final audio, and a DV guidetrack.  Now that you have the final edit in DV, you are ready to conform it at HDV quality.

There are a number of approaches you could take to accomplishing this.  You could throw away the project file and recapture everything in HDV to do an eye match to the DV guidetrack, withno other reference.  That would obviously be a tedious process. Alternatively, you could offline all your media in Premiere Pro, and recapture every entire tape, starting and ending at the exact same timecode values as your DV captures were, giving you an identical set of AVIs, that are at the higher HDV resolution.  This is more efficient, but still requires a lot of storage if your online format is 100 times larger than your offline (not true with HDV) and capture time.  Once finished capturing, there would be very little more work to do. (Resize titles, etc.) You could do a “Remove Unused” in Premiere to totally skip any tapes that were not used in the final edit.  This could save you some space, but only if you have a lot of captures you never ended up using at all.  The next option is to use the project manager to create an offline trimmed project, and recapture only the required parts of clips from HDV (usually with handles).  This process works in Premiere even with high end HD-SDI projects, but will not yet work with data centric workflows.  The alternative that SHOULD work for the Red RAW 4K workflow, among other tapeless formats, is very similar.  In our DV to HDV example, we export an EDL from our DV project and import that EDL into an HDV project to recapture only the needed material.  This allows us to easily capture only the required segments of higher resolution footage, saving tima and disk space.  The new captures will already have a project sequencing them in the correct order, so all that remains is to redo any titles and graphics or effects at the higher resolution, and the project will be ready to send to color correction or whatever the next step in your specific workflow is.

Silicon Imaging SI-2K Camera

      No Comments on Silicon Imaging SI-2K Camera

The Silicon Imaging SI-2K is a single sensor CMOS camera that mimics 16mm film camera optics.  It uses a bayer pattern to derive RGB images from a single native 2048×1152 pixel sensor.  These images are transfered uncompressed over a Cat6 ethernet cable to an Intel based computer system, which compresses the image data using Cineform RAW wavelet based compression.  The resulting data rate is low enough to be saved to a single 2.5″ hard disk.

When used with just the sensor, the SI-2K is an amazingly small and light imaging device for use in the tightest places, but it must be tethered to a computer system within 100 yards.  Usually the lens is much larger than the imager.  If an all-in-one form factor is desired, there is a full-sized camera body that can be attached to it, which contains a fully functional PC, running Windows XP Embedded.  The data is then saved on removable hard disks, that can be hot swapped via the internal USB interconnect.  With 300GB 2.5″ hard disks available now, you should be able to get 4 or 5 hours of footage onto a single drive.

Once the Cineform RAW footage is on disk, it can be edited in that codec on most popular NLEs with Cineform’s Neo2K product.  For best performance, you can use Prospect2K in Premiere Pro to edit with accelerated playback and rendering.  The RAW files are debayered in real time by the software, and edits or exports are rendered to regular Cineform 422 or 444 files, which work seemlessly together.

The SI-2K, when processed at the highest quality, generates a 10bit log RGB image at up to 2048×1152 pixels.  The colorist I work with has been very pleased with the dynamic range the images retain by the time they reach him in color correction.

Its small size gives this camera a few unique advantages.  Like the Iconix, it can capture stereoscopic footage without the use of a beam splitter, since the cameras can be mounted so closely together, side by side.  ParadiseFX has also recently developed an interesting application for the cameras, in their 18K wide 360 degree surround 2K360 video capture system, utilizing nine cameras running in parallel.  It has very limited and specialized uses, but will be much cheaper than any similar solution.  The next logical step is the inverse of this, with a ring of cameras pointing inwards, allowing time to be frozen while the perspective changes, like in the famous shots from The Matrix.

There are a few issues to be dealt with in using the camera though.  No matter how you configure the capture setup, you are using a Windows PC as your capture device.  This entails almost all of the same possible pitfalls of any other direct to disk recording system.  Your camera has to “boot up” and could lockup, crash, or even theoretically get a virus, if you network it to offload capture files.  Eventually you will probably have to reformat and reinstall the software as performance declines.  Silicon Imaging says they are currently working on a USB Flash based solution to that problem.  On the positive side, it makes it much easier to update the drivers and software, to add functionallity to the system as it continues to be developed by Silicon Imaging.  For example support to capture directly to Quicktime wrapped .mov files was recently added.

This is all around, a very creative and innovative imaging solution from Silicon Imaging, but they are still developing it into a smoother running product.  It has many possible uses that it is uniquely well suited for, but one needs to be ready to compensate for its limitations as well, as with any imaging solution.

Intel vs AMD

      No Comments on Intel vs AMD

It was only two or three years ago that AMD had the workstation solution of choice, over the bigger Intel.  AMD’s Opteron series was the first to offer many features that were especially important to video editors, from native 64 bit processing and multiple cores, to hypertransport frontside bus and integrated memory controllers.  Opteron’s with these features were  released about a year before Intel’s Xeons could catch up.  By the time Intel released Nocona core Xeons with 64bit support in mid 2005, AMD was selling Opteron’s with dual cores, and so on the race went.  The only advantage the Xeon’s were able to hold onto was their higher clock speed, but it was well known that Opteron’s were far more powerful at a given clock speed.

Then in the summer of 2006, Intel released two major processor upgrades back to back.  The Dempsey cores (5000 series) were a maxed out variation of the Pentium 4 “Netburst” architecture, and finally brought dual cores to the Xeon line.  The 3.73 Ghz was well above AMD’s 2.6 Ghz and the 1066Mhz FSB final topped AMD’s 1Ghz.  Only one month later, Intel released its entire new line of CPUs for all platforms, based on their totally new “Core2” design.  The Woodcrest series of Xeon’s (5100 series) were clocked lower, were supposed to be much more efficient per clock cycle, along the lines of the Opterons.  Woodcrest had everything to finally close the gap between Xeons and Opterons, with dual 64 bit cores runnning more efficiently and already at higher clock speeds, with a 3Ghz model available.  AMD had very little in the way of improvements in their response, and were totally unprepared when Intel released their next update less than 6 months later.

The Clovertown (5300 series) CPUs were simply two Woodcrest chips in a single socket, making it a Quad Core CPU.  This allowed a regular Xeon motherboard to support 8 discrete processing cores, clearly doubling performance in high end applications.  I had the privilege to use a Clovertown system for about a month when they were first released, and it was without question the fastest computer I have ever used, by a long shot.

AMD’s response was a new line of CPUs with a new numbering scheme, but no new major features.  Then recently, a year after Intel brought Quad Core CPUs to market, AMD released their long awaited Barcelona line, which were native quad core CPUs.  I have yet to see any version of those for sale nearly a month after release, and almost every review and benchmark has been negative.

We are now a month away from Intel’s next refresh of their CPU line, and are looking forward to more L2 cache, 1600Mhz FSB, and much lower prices.  AMD seems to have nothing in sight with which to compete with, which is unfortunate for both Intel and AMD users, since competition usually drives prices down for all users.  On the positive side, Intel doesn’t seem to be using their monopoly on the ultra high end to dramatically inflate prices. 

Xeons are still lacking AMD’s integrated memory controller and Hypertransport link, but those are scheduled to be included in Intel’s next major redesign, “Nehalem” in late 2008.  It will be interesting to see what AMD brings to the table by then.  Stay tuned for details when Intel releases their new line of CPUs next month.

Blackmagic HDLink & Video Hub

      No Comments on Blackmagic HDLink & Video Hub

Besides their Mac and PC compatible I/O components, Blackmagic makes a couple of utility devices that greatly increase the options available to a small post facility in using SDI video gear.  Technically, the Multibridge used to fall into this category as well, until they added PCIe support to it, totally extending its functionality.

The HDLink is a small box that converts HD-SDI video signal to DVI or HDMI, for monitoring video signals at full resolution with a computer LCD screen or projector.  It supports regular 1080 and 720 in most variations, and RGB 444 over dual link SDI.  It also has stereo sound output for audio monitoring.

These HDLinks are in widespread use at a number of the facilities I work at, and at under $500, they are within the reach of independent editors or anyone else who needs an economical solution for monitoring SDI.  They do get hot though, so they should be powered down when not in use, and they don’t last forever.

The new HDLink Pro adds 3Gb/s SDI for 2K support on a single SDI cable, but lacks dual link support, which is still the most popular RGB standard.  It also adds 6 Channel Audio de-embedding to analog RCA ports for 5.1 surround sound monitoring.  I have not yet had the opportunity to try one of these out yet, but I look forward to doing so as I find myself doing more 2K work, especially now that I have a 30″ LCD to connect it to.

 Separately but related, Blackmagic’s Workgroup Video Hub is a 12×24 SDI router.  It supports SD and HD, but currently not the 3Gb/s SDI standard that Blackmagic has been rolling out this year.  With a bit of creativity, it can easily support dual link connections, but that lowers the number of devices you can connect by half.  All patching is controlled in software via USB, which is connected to a single system, that hosts a network utility to share control with all users.  Their utility for patching as well designed and fleshed out at this point.  You can also patch through a built-in downconverter to SD, and it has a Still Store, ideal for color bars or logos.

We use these at three facilities I work at, and they bring a dramatic increase to the flexability and efficiency of your workflow.  They are not economical for use by a single individual, but will be well worth it in work areas shared by 3-10 people.  They greatly ease the sharing of high end video equipment.  For example, I can output any edit workstation’s SDI playout to either: a projector on an HDLink, an LCD on an HDLink, an HD CRT, our vectorscope, or any combination thereof.  Using a Video Hub can have a significant effect on your environment if you frequently find yourself reconfiguring your peripherals when switching between different projects.

Matrox Axio LE

      No Comments on Matrox Axio LE

Matrox’s Axio line of Premiere Pro based hardware solutions rewrite the application’s features much more so than most of the competing products.  With a focus on broadcast television output, as opposed to feature films or other less standardized workflows, Axio is highly optimized and accelerated for SD and HD, but supports no other frame sizes.  Axio LE is their newest  product, positioned at the mid level between the Axio SD and Axio HD.  The Axio LE also has a recent firmware upgrade that allows 10 bit capture and processing of SD and HD uncompressed footage.

The one Axio feature that I use the most, is Matrox’s somewhat proprietary HD compression format, MPEG I-Frame HD.  It supports full size and frame rate HD files at a constant bitrate, between 50 and 300Mb/s, depending on your needs.  I use 100 or 150Mb/s for projects going directly to broadcast video, and 50Mb/s is useful for offline edits that will be reconformed uncompressed.

Matrox also allows native editing of many different formats that Premiere does not otherwise support, including DVCProHD, P2 MXFs, XDCam MXFs, and HDV including 24fps.  In theory these all play in real time on the same timeline.  Real world results are a bit different, and I have experienced many performance problems when mixing these formats, but individually they all play back great.  My usually solution to this mirrors the Cineform workflow, to convert everything into one high performance codec, specifically Matrox’s MPEG I-Frame HD codec.  Matrox’s wide file support allows me to make all of these conversions without leaving Premiere, although using a separate project for asset preparation is highly recommended.  This conversion process is also accelerated by Matrox’s faster than realtime exports of all supported formats.
Continue reading

Cineform ProspectHD

      2 Comments on Cineform ProspectHD

Cineform was one of the earliest companies to create solutions catering to the emerging HDV workflows, lowering the budget requirements for projects desiring to produce higher resolution images.  Their solution AspectHD involved a separate high quality wavelet compression format that is scalable to even higher resolutions than HDV.  Combining this compressed format with a realtime rendering and effects engine in Adobe Premiere created a very efficient workflow.  By adding support for the AJA Xena HD-SDI cards, they had a professional-level I/O solution in ProspectHD.

The original ProspectHD supported compression of 1080p/i 10bit 422 data into a variable bitrate codec that only required around 15MB/s or 1GB/min.  Lowering the datarate had two significant benefits, both related to lowering the cost of disk storage.  Data rates at that level can be supported by a single hard disk drive, removing the need for expensive arrays, and much more HD content can be stored on a given drive.  The catch was that the amount of processing power needed to compress the data required the fastest CPUs, which at the time, meant dual processor AMD Opteron systems.  In the two years since then, processing power has greatly increased, and many more capable options are available.

Capture and playback over HD-SDI is supported in realtime using Premiere Pro.  Motion and opacity effects are realtime, as well as limited color correction and transitions.  The performance of these features scale with the power of the system, with many tasks being multithreaded to utilize multi-core CPUs.

The Cineform codec itself has many inherent advantages.  Wavelet codecs can easily be viewed at lower resolutions in an efficient manner.  As far as quality goes, Cineform is hard to beat, especially for their file size.  Another advantage I find significant is that their files perform well in After Effects, unlike many other compressed formats I have tried.
Continue reading

AJA Xena Cards

      No Comments on AJA Xena Cards

AJA currently has three main HD I/O cards for use in a PC, the Xena HS, the Xena LHe and the Xena 2Ke.  The LH and 2K come in PCIe and PCI-X varients, which are otherwise identical to my knowledge.  All of these AJA cards support 10bit color and 23.976/24p frame rates.  (The Xena HD was AJA’s first HD card for PC, and was identical to the HS except that the HS now includes Standard Def SDI support)
Continue reading

Blackmagic Multibridge

      No Comments on Blackmagic Multibridge

I was never all that impressed with original Multibridge from Blackmagic Design.  It was not an I/O interface for your computer, it was just a Analog to SDI convertor, both ways.  When the Multibridge Extreme was released, and with every major revision since then, computer I/O was added in a brilliant way.  Basically they embeding their Decklink electronics within the breakout box, and then devised a way to tap directly into the PCIe bus.  I don’t believe they created the original concept of External PCIe, but they were the first by a long shot to market a product utilizing the concept.  Caldigit and Ciprico, among others, have recently developed drive arrays with an external RAID controller that uses the same basic connection.
Continue reading