Film-Tech Forum ARCHIVE: EXT 4 FORMATTED DRIVES FOR DCP S

my profile | my password | search | faq & rules | forum home

»	Film-Tech Forum ARCHIVE » Operations » Digital Cinema Forum » EXT 4 FORMATTED DRIVES FOR DCP S (Page 2)

This topic comprises 3 pages: 1 2 3

Author

Topic: EXT 4 FORMATTED DRIVES FOR DCP S

Mattias Mattsson
Film Handler

Posts: 90
From: Göteborg, Sweden
Registered: May 2007

posted 11-04-2014 03:49 PM

That's strange..

I forgot to mention another thing that I have come across recently.

Newer Dolby versions (4.8.x) seems to have problems with unaligned partitions. All sorts of problems arises when a partition has an odd (or low?) starting sector. Including what you describe above, Carsten. Making the starting sector 2048 seems to resolve these issues.

I also agree that for distribution you should use the lowest common denominator that works. Which probably is to follow the recommendations of ISDCF: MBR, ext3, inode size of 128, etc..

| IP: Logged

Carsten Kurz
Film God

Posts: 4340
From: Cologne, NRW, Germany
Registered: Aug 2009

posted 11-04-2014 06:47 PM

I don't think there are Doremis in the wild with different kernel versions, are there? Maybe aside from their IMS. Does anyone know what kernel Dolbys DSS200 is running, and wether that changed/got updated over the course of their software upgrades?

Some Doremi software notes mention kernel versions 2.6.18-2 or 2.6.22-6, and the last change was in 2.2.5 to accomodate PCIe hotplugging in support for the IMB external interface.

GPT is officially supported since 2.6.25.

- Carsten

| IP: Logged

Mattias Mattsson
Film Handler

Posts: 90
From: Göteborg, Sweden
Registered: May 2007

posted 11-05-2014 04:26 AM

This is for the latest Dolby release. I haven't really kept track of the kernel versions but I think they have been updated along with previous firmware versions.

Linux showstore-1 2.6.35.13 #1 SMP PREEMPT Wed Sep 24 21:29:57 PDT 2014 i686
Dolby D-Cinema System Software Version 4.8.2 (23)

| IP: Logged

Carsten Kurz
Film God

Posts: 4340
From: Cologne, NRW, Germany
Registered: Aug 2009

posted 11-06-2014 02:17 PM

quote: Marcel Birgelen
There are two popular (one freeware, the other open source) products supporting EXT2 under Windows. Ext2-ifs is known to have this limit, but ext2fsd, while it used to be rather wonky, now works fine with inodes of 256 bytes.

Those servers with a windows based ext driver are old servers and will usually not receive any updates on that level anymore. At the time they were bundled with the server software, the server manufacturers probably tested all available options and decided for the most stable ones.

Just a couple of minutes ago I received notice from a cinema operating a Doremi and one of those windows based servers. Two discs from a professional distribution service they received could be ingested by the Doremi, but would not be recognized by the windows based server. This clearly looks like an inode size issue.

People should read and follow http://isdcf.com/papers/ISDCF-Doc3-Filesystem-Structure.pdf

- Carsten

| IP: Logged

Mattias Mattsson
Film Handler

Posts: 90
From: Göteborg, Sweden
Registered: May 2007

posted 11-06-2014 03:16 PM

quote: Carsten Kurz
Some Doremi software notes mention kernel versions 2.6.18-2 or 2.6.22-6, and the last change was in 2.2.5 to accomodate PCIe hotplugging in support for the IMB external interface.

Interesting, just checked a Doremi server with 2.6.4:

Linux dcp2000.dc.doremilabs.com 2.6.18.2 #5 SMP PREEMPT Mon Aug 26 16:11:57 CEST 2013 i686

Where does Doremi mention a later kernel version?

| IP: Logged

Carsten Kurz
Film God

Posts: 4340
From: Cologne, NRW, Germany
Registered: Aug 2009

posted 11-06-2014 05:24 PM

http://support.doremitechno.org/images/drmftp/ftpservice/Software/SW_Release_Notes/DCP2000_Software_Release_Notes_0 00288_v4_7.pdf

'9.7.4 System Changes
Upgrade to Linux kernel 2.6.22.6.'

'9.2.4 System Changes
- System: • Linux kernel is downgraded to version 2.6.18-2'

So they had a later revision kernel on older 0.5x software versions, then downgraded to 2.6.18-2 with later software releases. No reason given in this document. But seems obvious that Doremi stuck to this kernel and preferred to build enhancements into it on their own - e.g. GPT. Still it seems Doremi does not support discs larger than 2TB.

- Carsten

| IP: Logged

Antti Nayha
Master Film Handler

Posts: 268
From: Helsinki, Finland
Registered: Oct 2008

posted 11-13-2014 02:39 PM

quote: Dave Macaulay
I would format external drives destined for content storage/moving as ext2. The journaling feature of ext3 isn't very valuable for an archive drive in my opinion.

That’s exactly that the ISDCF thinks, too:

quote:
Allow EXT2 for storage format partitions. EXT2 is EXT3 without a journal, and a journaled file system is of questionable value for a distribution format.

So the consensus seems to be that ext3 offers no advantage over ext2 in this use.

But is there any real disadvantage in using ext3 compared to ext2? Does journaling affect performance or something?

| IP: Logged

Carsten Kurz
Film God

Posts: 4340
From: Cologne, NRW, Germany
Registered: Aug 2009

posted 11-13-2014 07:00 PM

No. ext3 really is just ext2 with journaling as extra. Don't use it, makes no difference for the server.

- Carsten

| IP: Logged

Antti Nayha
Master Film Handler

Posts: 268
From: Helsinki, Finland
Registered: Oct 2008

posted 11-15-2014 12:51 PM

Carsten, let me rephrase: how exactly do you guys deduce "don't use ext3" from "ext2 vs ext3 makes no difference"?

Looking into it, I found some benchmark comparisons:

Those are about sequential read/write performance, which is what matters when dealing with large files such as those found on DCP’s. So it looks like ext2 is faster – particularly when writing – which makes sense given the lack of journaling overhead. Now, that does sound like a valid reason for choosing ext2 over ext3 to me, particularly when you're writing/copying lots of DCP's.

Supposedly, journaling makes the file system more resilient to unclean shutdowns/unmounts. But that's only a factor when you're writing to the drive, which is why it's not considered useful for transport drives – did I get that right?

| IP: Logged

Dave Macaulay
Film God

Posts: 2321
From: Toronto, Canada
Registered: Apr 2001

posted 11-15-2014 02:52 PM

Yes. The ext3 journaling is primarily to protect data integrity when writing. Since distro drives are copied as clones and verified, journaling is pretty much useless there... and totally useless for reads into the server storage. The journaling activity takes time on each block transfer, slowing down transfer speed.

| IP: Logged

Mattias Mattsson
Film Handler

Posts: 90
From: Göteborg, Sweden
Registered: May 2007

posted 12-14-2014 08:18 AM

Dolby does indeed differentiate the kernel version depending on server hardware. This is the kernel version used on a DSS220 running 4.8.2(23):

Linux showstore-1 3.2.23 #1 SMP Wed Sep 24 21:45:42 PDT 2014 i686
Dolby D-Cinema System Software Version 4.8.2 (23)

They probably need a more modern kernel on DSS220 because they are using the Linux software RAID (mdraid) on these machines. Also interesting is that this kernel for DSS220 lacks the RT-Preempt Patch that the kernel for DSS200 seems to have.

| IP: Logged

Marcel Birgelen
Film God

Posts: 3357
From: Maastricht, Limburg, Netherlands
Registered: Feb 2012

posted 12-15-2014 02:49 AM

quote: Antti Nayha
Those are about sequential read/write performance, which is what matters when dealing with large files such as those found on DCP’s. So it looks like ext2 is faster – particularly when writing – which makes sense given the lack of journaling overhead. Now, that does sound like a valid reason for choosing ext2 over ext3 to me, particularly when you're writing/copying lots of DCP's.

I think the main motivator in choosing the file system should still be compatibility. If you need to churn out a lot of disks in a limited time frame, you could easily come up with an affordable rig which would allow you to create multiple disks in parallel.

| IP: Logged

Antti Nayha
Master Film Handler

Posts: 268
From: Helsinki, Finland
Registered: Oct 2008

posted 12-20-2014 06:33 AM

But surely the real-world compatibility is equal? That is, how could any conceivable system support only ext3 and not ext2?

| IP: Logged

Carsten Kurz
Film God

Posts: 4340
From: Cologne, NRW, Germany
Registered: Aug 2009

posted 12-20-2014 06:46 PM

I am not a specialist there, but as far as I understood, ext3 is just 'some additional information' - the journaling info. As such, it is more or less impossible for ext2 to be incompatible with ext3 supporting systems. An ext2-only capable system would simply ignore the journaling information, which would cause no harm if fs and journal are synced.
And afaik there is no Linux system that doesn't support both. The only way to exclude ext2 would be to reject ext2/non-ext3 filesystems explicitly and intentionally for specific reasons, i.e. because the specific system would be configured to consider a non-journaling fs as outdated or unsafe.

I don't know if the journaling overhead figures given above are also true for large contiguous files as used for DCPs. Anyway - you may do your own testing on that, writing a typical DCP on the same system to an ext2 formatted disc, and then again to the same disc formatted as ext3.

- Carsten

| IP: Logged

Antti Nayha
Master Film Handler

Posts: 268
From: Helsinki, Finland
Registered: Oct 2008

posted 12-20-2014 07:10 PM

quote: Carsten Kurz
The only way to exclude ext2 would be to reject ext2/non-ext3 filesystems explicitly and intentionally

I'm not quite convinced that even that is possible…

And sequential read/write = large continuous files.

| IP: Logged

All times are Central (GMT -6:00)

This topic comprises 3 pages: 1 2 3

Printer-friendly view of this topic

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.