HDMI 1.4 does NOT support checkerboard

[DmitryKo]

I've recently discovered that according to Wikipedia, HDMI 1.4 does support a "frame packing" format, which I understand is the cuincunx or "checkerboard" pattern. This was quoted from HDMI 1.4 Press Briefing and the DevCon Presentation.

This "frame packing" format is not currently present in a list of steroscopic 3D formats in the HDMI 1.4 FAQ and the HMDI 1.4 Installer Training Practice (however, according to the latter, HDMI does provide support 120 Hz (and 100 Hz) signalling for 720p and 1080p with a "high-speed HDMI cable", enabled by CEA-861-E update which probably features shortened timings much like the CVT Reduced Blanking formula).


So I've sent an inquiry to HDMI LLC and today I've got a reply: the "frame packing" format is a mandatory format for any 3D-capable device. So any 3D-ready television must support TI's 1080p60 "checkerboard" format and existing HDMI 1.3 devices which support 1080p60 should have no problem playing back stereo3D content on any future 3DTV device

That solves the PS3 stereo gaming mystery - the games will render at 720p120 then scale it to the 720p60 or 1080p60 checkerboard format 8) And PC users with a powerful video card will be able to enjoy uncomplressed Full HD 120 Hz over a HDMI connection.

I still give HDMI LLC two thumbs down for not publicizing enough these features which are quite important for stereoscopic gaming.


Update: from the freely available HMDI 1.4 3D spec, "frame packing" is actually a special full-resolution top-bottom stereo format. So in the end, HDMI 3D does support 120 Hz frame sequential displays with mandated support for 1080p24 and 720p50/60 frame packing; 1080p60 frame packing, side-by-side full and line alternative formats are all optional though, and checkerboard format is not suppored, although side-by-side half format does support quincunx sampling.

[BlackShark]

Are you sure frame packing is checkerboard ?
It doesn't really make sense to me as this format can't be stored efficiently. A pixel based transformation is required everytime content is decoded from video file/bluray/broadcast.
It's like if they made HD-ready with only 1080i mandatory, 720p being optionnal. It just makes no sense to me.

I have no idea what this "frame packing" actually is but i would have expected the images to be packed at full resolution within some metadata.

[DmitryKo]

this format can't be stored efficiently

Nobody stores anything using these structures, it's just a display link format.

I haven't seen any explicit explanations but I would assume the best frame packing format would be checkerboard, as to ensure backward-compatibility with existing stereoscopic 3DTVs, like 1080p DLP sets from Samsung and Mitsubishi and 720p plasmas from Samsung, and existing software and devices that use checkerboard DLP. Since frame-packing 3D formats 1080p24, 720p50/p60 and 1080p50/p60 are mandatory for 3D display devices in HDMI 1.4, that would make any future 3DTV backwards compatible, no matter the display technology it employs (active shutters, passive polarized and dual projection, or autostereoscopic).


3D Blu-ray content will be coded at 1080p48 using Multiview Video Coding, an extension to H.264 aka MPEG-4 Part 10 Advanced Video Coding (MPEG-4 AVC), using new Stereo High profile which enccodes 2 progressive or interlaced stereoscopic video streams with a typical 50% overhead. And gaming applications will render to a 120 Hz 1080p or 720p framebuffer, then perform a stretch and/or checkerboard encoding; considering that the framebuffer contains ~7.9 and ~3.5 MBytes of 32-bit RGBA data, this operation should take less than 1/1000 s on modern video cards and gaming consoles. If that is not enough, Silicon Image has already sampled a HDMI 1.4 transmitter chip and receiver chip which support the "frame packing" encoding right in the silicon (but curiously it does not cite support for CEA-861-E with 100/120 Hz timings).

[BlackShark]

It seems to me like frame packing isn't checkerboard.

Although i am still not sure about the exact meaning of frame packing, one thing is certain is that according to these docs a stereo image is counted as a single frame.
Blu Ray will be recorded and transmitted as 2x1080p24, not 1080p48.
Similarly games are already rendered as 2x60Hz frame buffers. It makes a lot more sense than rendering 1x120Hz frame buffers.
These figures may correspond to the same amount of pixels or require roughly the same bandwidth, but they have very different meanings.

I think this confusion comes from our current 3D monitors which use technologies not designed for 3D : from the same video source, for example a game @ 60Hz, nvidia requires 120Hz transmission, Zalman uses interlaced 60Hz, iZ3D transmits polarisation angles as colour data, etc...
The stereo frames have to be transformed into the native display pannel format before transmission, whereas it should be transferred untouched and only transformed at the final step inside the display. This is also the reason why hardware image transformation is not implemented in most 3D monitors (how do you change the amount of colour if it screws up the iZ3D polarisation, how do you upscale interlaced stuff without de-interlacing and then re-interlacing ? (i consider checkerboard pattern = a form of interlacing). And if a re-interlacing filter is already in the display, why not transmitting the images normally in the first place ?

It just makes more sense to send untouched L and R images with some identification metadata attached to them and consider L+R = 1 stereo frame, which is what i believe "frame packing" actually means.

[DmitryKo]

i am still not sure about the exact meaning of frame packing

Frame packing is obviously packing two stereo video frames into one single video frame, resulting in a "half-resolution" stereo image.

Blu Ray will be recorded and transmitted as 2x1080p24, not 1080p48

It does not make much practical difference (even though there are some technical differences in regard to video signal timings), other than reducing processing time somewhat when using a format that is "native" to the stereo display technology used.

For example, a 2-engine display device like HDI laser 100" TV or dual projectors is relatively better served by 1080p24 side-by-side format, passive polarized is better served by line-interleaved format, and shutter-glassed 3D LCD TVs is better served by 48 alternating frames. Each of these displays could accept any other format - it would just introduce some processing delay, and that's not really improtant for pre-rendered content. And even then, there will always be some delay between the frames, no matter what format is used, as you just can't simultaneously transmit two stereo images over a scan-line serial display link such as the one currently implemented in HDMI 1.4.

Similarly games are already rendered as 2x60Hz frame buffers. It makes a lot more sense than rendering 1x120Hz frame buffers

No. Current video chips do not render two stereo frames at once to two different framebuffers - that would double the complexity since the chip would have to process twice as much geometry data. The two views are simply rendered in a sequence and presented as needed. So until stereoscopic 3D is the default rendering mode in most games and video chip engineers are forced to implement parallel 2-way stereo rendering (which naturally will be better served by a 2-way transmission format such as side-by-side), gaming is better served by frame sequential which will result the least processing delay.

if a re-interlacing filter is already in the display, why not transmitting the images normally in the first place

Because 120 Hz signals and side-by-side signals were not defined in the CEA-861 by the time first 3DTVs were introduced, which is exactly every 1080p DLP TV produced in the last 2 years. These sets will remain in homes for 5-10 years, and so we are left with this legacy format for the nearest future.

untouched L and R images with some identification metadata attached to them and consider L+R = 1 stereo frame, which is what i believe "frame packing" actually means.

No. What you have described is just some "full resolution" format like side-by-side or frame sequential, as opposed to "lossy" formats which pack two full-resolution frames into one full-resolution frame with a loss of some spatial information.

It seems to me like frame packing isn't checkerboard.

If "frame packing" were just some "container" for 3D stereo which supports many picture formats, why list it among other formats like frame sequential, side-by-side and line interleaving?

Here are all known methods for packing 2 stereo frames into one 1080p frame, resulting in two "half resolution" frames:
1 checkerboard (quincunx) pattern
2 vertical (line) interleaving
3 vertical subfield (interlaced video signal)
4 side-by-side half resolution
5 top-bottom half resolution

(In theory there is also column (horizontal) interleaving and horizontal subfield, but I guess they are not compatible with video processing workflow).

All of the above are listed as separate encoding methods in the HDMI 3D spec (top-bottom wll be included in a future revision), except for checkerboard pattern - which makes it a logical assumption that "frame packing" is HDMI acronym for checkerboard. If you find flaws in this logic, please elaborate.

[BlackShark]

i am still not sure about the exact meaning of frame packing

Frame packing is obviously packing two stereo video frames into one single video frame, resulting in a "half-resolution" stereo image.

I agree with the first part (2 frames within one frame), but not with the second part (meaning half resolution)

If "frame packing" were just some "container" for 3D stereo which supports many picture formats, why list it among other formats like frame sequential, side-by-side and line interleaving?
Here are all known methods for packing 2 stereo frames into one 1080p frame, resulting in two "half resolution" frames:
[..list..]
All of the above are listed as separate encoding methods in the HDMI 3D spec (top-bottom wll be included in a future revision), except for checkerboard pattern - which makes it a logical assumption that "frame packing" is HDMI acronym for checkerboard. If you find flaws in this logic, please elaborate.

I do not believe frame packing is a container for many stereo formats, i believe it is a container for a single stereo format.
From the outside a packed frame looks similar to a big side by side full-res frame : something like (for example) beginning of frame sync stuff, 2840x1080 data,end of frame sync stuff (i don't know the details of an actual signal transmission, i just know there are sync stuff required at regular moments of the transmission). But from the inside the actual data contains additional sub frame components precisely identifying the left and right views as 1920x1080 (sub)frames.

What makes me believe this is that I remember the last time i read the term "frame packing" was at the doom9 forums when x264 developers were explaining why storing H.264 in an AVI file requires tricks. There was an issue with B-frames (interpolated frames requiring loading data from multiple input frames to produce one output frame). The AVI container was not designed to hold such frames, so the B-frames data has to be "packed" with metadata within other frames in order to make the data stream compliant with the AVI file format.
For the file, the individual frames become much much bigger but it isn't a problem for avi filters and media players as they are capable of handling the amount of data, Since the data is not read but just passed through to the next image processor the only thing actually using the packed data being the codec filter, which role is to unpack the frames.

My guess is that hdmi 1.4 3D frame packing uses a similar technique, packing the amount of data of a stereo pair of images within what should usually be considered as one frame. And since hdmi1.4 supports ludicrously big resolutions (4k if i remember correctly) the chips should be able to transmit and recieve these big frames fine. Allowing lower cost for developing hdmi 1.4 chips (you don't need to redesign the entire input chip because of stereo 3D frames, you only need to add an unpacking module to the current chip.

Blu Ray will be recorded and transmitted as 2x1080p24, not 1080p48

It does not make much practical difference (even though there are some technical differences in regard to video signal timings), other than reducing processing time somewhat when using a format that is "native" to the stereo display technology used.
For example, a 2-engine display device like HDI laser 100" TV or dual projectors is relatively better served by 1080p24 side-by-side format, passive polarized is better served by line-interleaved format, and shutter-glassed 3D LCD TVs is better served by 48 alternating frames. Each of these displays could accept any other format - it would just introduce some processing delay, and that's not really improtant for pre-rendered content. And even then, there will always be some delay between the frames, no matter what format is used, as you just can't simultaneously transmit two stereo images over a scan-line serial display link such as the one currently implemented in HDMI 1.4.

I agree with you on the latency reduction theme but the issue with transmitting native images to the display is that it doesn't work in the workflow of modern digital TVs. It might worked well in the old days of analog CRT monitors driven directly by the computer, but modern TVs are overloaded with image treatment chips and algorithms to improve colour, generating subframes to artificially increase the framerate, denoising, deblocking and comb filters to reduce compression artifacts, watch multiple channels with picture in picture etc...
If you want to be able to reuse previous generations of image treatment chips (save R&D time and money) instead of redesigning new ones, it makes much more sense to transmit a single standard full frame (unpacking the frames in the display hdmi input chip) rather than transmitting any native display format.

Similarly games are already rendered as 2x60Hz frame buffers. It makes a lot more sense than rendering 1x120Hz frame buffers

No. Current video chips do not render two stereo frames at once to two different framebuffers - that would double the complexity since the chip would have to process twice as much geometry data. The two views are simply rendered in a sequence and presented as needed. So until stereoscopic 3D is the default rendering mode in most games and video chip engineers are forced to implement parallel 2-way stereo rendering (which naturally will be better served by a 2-way transmission format such as side-by-side), gaming is better served by frame sequential which will result the least processing delay.

It seems we are talking about two different parts/functions of the frame buffers.
It is true trat current 3D drivers do render the frames sequentially, but the framebuffers themselves only work once both frames are ready (i'm talking about the swap between back buffer and front buffer). I also believe that with unified shader architectures that all DX10/DX11 GPUs use, it should be possible to make the games render both frames simultaneously : I mean simultaneously from the game's point of view, since i know there are not enough units in the GPU to complete a single operation on an entire 2D frame within a single GPU clock cycle.

[DmitryKo]

a packed frame looks similar to a big side by side full-res frame ... the actual data contains additional sub frame components precisely identifying the left and right views as 1920x1080 (sub)frames

Side-by-side does not need any additional frame identification tags - left half of the picture is always the left frame, right half is the right frame, and they are perfectly in sync. It's that simple.

Please understand that this is not a broadcast format or container format where new features should be added in a forward-compatible way. It's a device-to-device link which can negotiate the best transmission format - you either support stereo 3d formats or you only support standard HDTV formats, there is no need for tags and compatibility hacks.

storing H.264 in an AVI file requires tricks. There was an issue with B-frames ... AVI container was not designed to hold such frames, so the B-frames data has to be "packed" with metadata within other frames in order to make the data stream compliant with the AVI file format

Cool. Still I don't see how all these technical quirks apply to a video-only bitstream as used in display link inteface such as DVI/HDMI which transmits complete frames/fields.

My guess is that hdmi 1.4 3D frame packing uses a similar technique, packing the amount of data of a stereo pair of images within what should usually be considered as one frame. And since hdmi1.4 supports ludicrously big resolutions (4k if i remember correctly) the chips should be able to transmit and recieve these big frames fine

Again, how does it differ from side-by-side?

modern TVs are overloaded with image treatment chips and algorithms to improve colour, generating subframes to artificially increase the framerate, denoising, deblocking and comb filters

The image processing latency doesn't matter for video content since it does not require user interaction - the processing delay could be up to several seconds and no-one will ever notice.

For user-interactive application, most TV displays now feature a "game mode" which disables most of the additional processing beyond the bare minimum.

it makes much more sense to transmit a single standard full frame (unpacking the frames in the display hdmi input chip) rather than transmitting any native display format

I don't see why. If the processing delay increases from 30 ms to 60 ms, noone would notice, and there is always an option of using 2 or more chips since the costs will be covered by a stereoscopic price premiums.

I also believe that with unified shader architectures that all DX10/DX11 GPUs use, it should be possible to make the games render both frames simultaneously : I mean simultaneously from the game's point of view

Again, the best transmission format is defined by the properties of the display device, not the intrenal quirks of the rendering engine.

Suppose you are looking around the corner and one "eye" only sees a wall while the other sees the complete scene - the rendering times would be very different, no matter if the stereo L/R frames were rendered sequentially or in parallel. But you just can't transmit one frame if the other frame has not finished rendering - they both have to be presented for transmission to begin, and lowering the delay is only possible with higher framerates. Choosing a different video link format doesn't really change that.

[BlackShark]

a packed frame looks similar to a big side by side full-res frame ... the actual data contains additional sub frame components precisely identifying the left and right views as 1920x1080 (sub)frames

Side-by-side does not need any additional frame identification tags - left half of the picture is always the left frame, right half is the right frame, and they are perfectly in sync. It's that simple.
[...]
Again, how does it differ from side-by-side?

The issue with side by side is that it's so simple that they would have labelled it "side by side full resolution" (to show the difference with side by side half resolution)
If it were the checkeerboard they would have labelled it "checkerboard" or "TI DLP3D format" or "cuincunx" (by the way, you're the only personI know who uses this name, where did you find it ?)
My guess is that the Packed format is neither of those two solutions but actually sending the left and right eye images as two separate sub-frames, then both packed within the technical boundaries of what previous hdmi recievers would believe to be a frame.

Please understand that this is not a broadcast format or container format where new features should be added in a forward-compatible way. It's a device-to-device link which can negotiate the best transmission format - you either support stereo 3d formats or you only support standard HDTV formats, there is no need for tags and compatibility hacks.

On the contrary HDMI has a lot of forward compatible transmission requirements.

Hdmi 1.4 chips must be able to decode streams encoded by prevous hardware using all hdmi revision all the way from 1.0/1.1/1.2/1.3a/1.3b to 1.4.
Now imagine the complexity of the input and output chips if completely different image formats were used for every new revision. This isn't some software codec filters you can just store on a hard drive and load whenever appropriate : this is a chip that is permanently wired. So Forward compatibility is mandatory.
The issue here is not getting stuff to work : it's money, R&D money : engineers that have to completely redesign the chips at every new revision it's just not acceptable.
Forward compatibility allows to reuse huge portions of the R&D work previously done and only spend what is absolutely necessary on supporting the new features.

Using subframes clearly identified as left and right would also be much more forward compatible than checkerboard or side by side, as identifying the frames is the best way to provide future support for depth frames (for 2D + depth format, like Philips WOWvx technology), more than 2 views for other autostereo displays : (4 view displays, 8 view displays, like alioscopy displays), virtual data frames for defining occlusion frames (for 2D + depth + occlusion), etc...

Cool. Still I don't see how all these technical quirks apply to a video-only bitstream as used in display link inteface such as DVI/HDMI which transmits complete frames/fields.

The idea is that HDMI does not transmits pixels, but it's actually just binary data packed within boundaries that the hdmi chips call beginning and end of frames. within that data you can store pixel information in many different formats (8 bit, 10 bit, RGV, YUV, standard gammut or xvYCC gammut, 4:4:4 or 4:2:2/4:2:0/4:1:1 subsampled, etc...). so you could hide metadata or even ascii text within that data the Hdmi chips won't care, they'll just decrypt the data it believes to be the frame and hand it over to the TV chips to do the actual image treatment.
In the same way compressed frames can be packed within other compressed frames in order to allow the avi format to be forward compatible, the same can happen with an HDMI packed frame, granting a huge amount of forward compatiblity for 3D displays that would otherwise be impossible to reach otherwise.

Remember the press releases when the hdmi 1.4 version was originally announced : the goal was not to standardize stereo displays but all 3D displays as a whole. Even for formats that are not yet known (what the hell does an occlusion frame looks like ? nobody knows but philipps kept talking about them when mentionning improvements for autostereo displays).

modern TVs are overloaded with image treatment chips and algorithms to improve colour, generating subframes to artificially increase the framerate, denoising, deblocking and comb filters

The image processing latency doesn't matter for video content since it does not require user interaction - the processing delay could be up to several seconds and no-one will ever notice.

For user-interactive application, most TV displays now feature a "game mode" which disables most of the additional processing beyond the bare minimum.

it makes much more sense to transmit a single standard full frame (unpacking the frames in the display hdmi input chip) rather than transmitting any native display format

I don't see why. If the processing delay increases from 30 ms to 60 ms, noone would notice, and there is always an option of using 2 or more chips since the costs will be covered by a stereoscopic price premiums.

You are taking the problem upside down.
The normal behavior of a TV is with all the gadgets and gizmos the TV manufacturer added.
The optionnal behavior is the "game" mode that circumvent all the extra stuff and just displays the image as raw as it can be to reduce latency.

So it makes much more sense to make it mandatory to send the source native format (two separate L and R images) to the display and then let the display do it's magic at lower cost and lower complexity, while keepingother transmission formats (sidebyside/checkerboard/interlaced/frame sequential/etc...) as an optionnal transmission method : and this is exactly what (in my opinion) the hdmi 1.4 standard is doing.

I also believe that with unified shader architectures that all DX10/DX11 GPUs use, it should be possible to make the games render both frames simultaneously : I mean simultaneously from the game's point of view

Again, the best transmission format is defined by the properties of the display device, not the intrenal quirks of the rendering engine.

Suppose you are looking around the corner and one "eye" only sees a wall while the other sees the complete scene - the rendering times would be very different, no matter if the stereo L/R frames were rendered sequentially or in parallel. But you just can't transmit one frame if the other frame has not finished rendering - they both have to be presented for transmission to begin, and lowering the delay is only possible with higher framerates. Choosing a different video link format doesn't really change that.

That is true and that was my point. Unless we had a misunderstanding ?

My point was no matter what the best transmission method is for the display, you still need to wait for the game engine to finish rendering both frames before sending them to the display. So why transforming it in the GPU (which takes time) if within the exact same amount of time you could just send the image pair the game rendered in it's original format and let the display do it rest instantly with a dedicated chip ?

[DmitryKo]

My guess is that the Packed format is neither of those two solutions but actually sending the left and right eye images as two separate sub-frames, then both packed within the technical boundaries of what previous hdmi recievers would believe to be a frame.

In other words, you guess that in the frame packing format each pixel represents two stereo pixels? IMHO that would be "pixel packing", not "frame packing".

by the way, you're the only personI know who uses this name, where did you find it ?

I can't remember, it's just the Latin term for a very similar 5-dot "star" pattern.

If it were the checkeerboard they would have labelled it "checkerboard" or "TI DLP3D format"

I don't know. Why a 1080p side-by-side format is marked "RealD proprietary" in Sony, JVC and Samsung press releases?

On the contrary HDMI has a lot of forward compatible transmission requirements. Hdmi 1.4 chips must be able to decode streams encoded by prevous hardware using all hdmi revision all the way from 1.0/1.1/1.2/1.3a/1.3b to 1.4.

No. It's not forward compatibility, it's rather backward compatibility.

Forward compatibility would require a HDMI 1.0 device to be able to link with any valid HDMI 1.4 source, which is not possible since HDMI 1.3 doubles the maximum pixel rate and older HDMI 1.0 devices will not be able to connect at this rate.

Now imagine the complexity of the input and output chips if completely different image formats were used for every new revision. This isn't some software codec filters you can just store on a hard drive and load whenever appropriate : this is a chip that is permanently wired

Look, I have an engineer degree in electronics and IT, and what you are saying is a bit naive. "Wiring" (tracing) is just the final step, you first need to define the logical schema of your device. It is true that tracing is a very complex process with undetermined results and is not guaranteed to complete in a working device or even to complete at all. But no-one "completely redesigns" everything with each revision of the hardware - there are component libraries which contain tested and proved schemes, a kind of production ready "building blocks". And complex chips are always designed as a system-on-a-chip, that is breaking the logic down to with micro-blocks connected through on-chip component buses. It would be simply impossible to trace a complete electrical scheme otherwise, given the complexity of modern VLSI chips. Also, almost every non-RISC processing device is designed with user-updateable microcode ROM (IBM 360 and Intel Pentium/Core are the most famous examples), and dedicated DSPs (digital signal processors) almost universally use user-loadable microcode to implement their processing algorythms.

As for HDMI, the only serious improvement came with revision 1.3 which doubles the single-link video bandwitdth, and you don't need to "completely redesign" your chip for that, you just need to use better silicon manufacturing process and maybe do a re-trace, but it's part of the routine work of preparing chips for production on a given fab.

Forward compatibility allows to reuse huge portions of the R&D work previously done and only spend what is absolutely necessary on supporting the new features.

Again, you confuse backward compatibility with forward compatibility.

In forward compatibility, an older device will be able to accept data intended for a newer system which offers new features, even though the older device will not recogize new features that it does not support. An historical example is color TV and stereo FM transmission formats, which were designed with forward compatibility in mind, so that monophonic FM receivers could accept stereo FM broadcasting, and black-and-white televisions could accept color signal, even though they would play it back in mono and black-and-white.

If color TVs would accept black-and-white signal, but black-and-white TVs would not accept color signal, that would only be backward compatibility.

Using subframes clearly identified as left and right would also be much more forward compatible than checkerboard or side by side, as identifying the frames is the best way to provide future support for depth frames (for 2D + depth format, like Philips WOWvx technology), more than 2 views for other autostereo displays : (4 view displays, 8 view displays, like alioscopy displays), virtual data frames for defining occlusion frames (for 2D + depth + occlusion), etc...

The idea is that HDMI does not transmits pixels, but it's actually just binary data packed within boundaries that the hdmi chips call beginning and end of frames. within that data you can store pixel information in many different formats (8 bit, 10 bit, RGV, YUV, standard gammut or xvYCC gammut, 4:4:4 or 4:2:2/4:2:0/4:1:1 subsampled, etc...). so you could hide metadata or even ascii text within that data the Hdmi chips won't care, they'll just decrypt the data it believes to be the frame and hand it over to the TV chips to do the actual image treatment.

It's all cool, but dear Blackshark, your stated goal is to reduce complexity and save research budgets, yet you seek to achieve it by reimplementing every single known pixel format supported by HDMI using some new, different data structures, which do require redesigning the physical link interface to a much broader extent than any previous HDMI revision, and provide no new features that cannot be implemented with current revision of the interface. Do you see any irony in that?

Just let the physical link do its job of transmitting pixels and do all the stereoscopic frame processing on the software side, just as it happens with other line-interleaved, sequential and side-by-side formats. And if you need a complete departure from HDMI, there is VESA-approved DisplayPort standard, which offers twice the bandwidth and extensible micro-packet architechture.

Remember the press releases when the hdmi 1.4 version was originally announced : the goal was not to standardize stereo displays but all 3D displays as a whole. Even for formats that are not yet known (what the hell does an occlusion frame looks like ? nobody knows but philipps kept talking about them when mentionning improvements for autostereo displays).

WOWvx uses a fairly well documented format, compared to the mystical "Real-D side-by-side" - see the 3D Display Interface White Paper by Philips (see page 10 for a description of the Declipse format).

You are taking the problem upside down.
The normal behavior of a TV is with all the gadgets and gizmos the TV manufacturer added.
The optionnal behavior is the "game" mode that circumvent all the extra stuff and just displays the image as raw as it can be to reduce latency.

No, I'm not. Serving as a computer display is not the main purpose of a TV set. If this processing does offer subjectively "better" image quality for the majority of consumerss, who only watch DVD video and SDTV broadcasts on their new panels, why not turn it on it by default?

Also, please tell me how an inherently progressive, fixed-pixel Full HD display like LCD or plasma panel should be able to display 1080i, 720p or 576i signal whithout first performing deinterlacing or image stretching and color space conversion.

it makes much more sense to make it mandatory to send the source native format (two separate L and R images) to the display ... this is exactly what (in my opinion) the hdmi 1.4 standard is doing.

No. Frame packing is the only mandatory format for playback devices in the HDMI 3D spec.

So why transforming it in the GPU (which takes time) if within the exact same amount of time you could just send the image pair the game rendered in it's original format and let the display do it rest instantly with a dedicated chip ?

Because the GPU is much more powerful than a $5 dedicated HDMI transmitter chip, 150 GBytes/s of bandwidth and 3 TFLOPS on the latest Radeon 57xx, so an additional filtering operation would be practically instant. The transmitter is better left at what it's designed for, transmitting pixels.

[BlackShark]

My guess is that the Packed format is neither of those two solutions but actually sending the left and right eye images as two separate sub-frames, then both packed within the technical boundaries of what previous hdmi recievers would believe to be a frame.

In other words, you guess that in the frame packing format each pixel represents two stereo pixels? IMHO that would be "pixel packing", not "frame packing".

That's not what i meant.
Packing can mean pretty much anything. the only thing certain is that it sends the left and right eye images together as a whole (packed). So this rules out blind frame sequential (pageflipping like nvidia 3D vision does).

My pick would be full resolution separate Left than right images (what i called sub-frames) sent one after the other but packed within the data of the same frame and separated with metadata clearly identifying the beginning and end of each left and right images. From the inside it looks as frame sequential, with twice the amount of actual full frames, but since the left and right eye frames are not declared as frames but packed as sub-frames, from the outside it looks as if if were one single bigger frame, keeping the framerate identical with a 2D video

I never thought about pixel packing, it could be a possibility ! Instead of making the frame artificially larger, it would allow to keep the exact identical framerates and frame size, the video characteristics that would be inflated would be the bit depth.

Line packing would be equivalent to full resolution (double) interlacing, but it is very unlikely as it adds transform complexity for no reason
Side by side packing would be equivalent to full resolution side by side, but it also is unlikely as it would just have been called full resoluton side by side
Current checkerboard is unlikely as it is only half resolution, with Panasonic and the BluRay association pushing for full resolution 1080p it is in my opinion absolutely not an option.
Checkerboard could be extended to support full resolution, it is a possibility. But the complexity of unpacking checkerboard is greater than unpacking full frames which is why this format is in my opinion less likely than untouched full resolution frame transmission.

I don't know. Why a 1080p side-by-side format is marked "RealD proprietary" in Sony, JVC and Samsung press releases?

You can see what the RealD format looks like when playing AVATAR the game. It's not exactly side by side, it looks more like a hybrid between a checkerboard and side by side, and there are additional color bars at the bottom of the frame.
I'm not really interested in this format but it obviously aims at allowing subsampling and maybe also resizing without having to "unpack" the checkerboard patten. The sensio chip is still required in the TV to change the signal to the native screen format at the end.

I'm not sure if it looks better than simple side by side, it maybe does but i'd rather have full res frames sent to the display and only have the image degradation at the absolute final step inside the display.

It's not forward compatibility, it's rather backward compatibility.

Forward compatibility would require a HDMI 1.0 device to be able to decode any HDMI 1.3 stream, which is not possible since HDMI 1.3 doubles the maximum pixel rate and older HDMI 1.0 devices will not be able to connect at this rate.
[...]
In forward compatibility, an older device will be able to accept data from a completely new device that has some novel features. An historical example is color TV and stereo FM transmission formats, which were designed with forward compatibility in mind, so that monophonic FM receivers could accept stereo FM broadcasting, and black-and-white televisions could accept color signal. If color TVs would accept black-and-white signal, but black-and-white TVs would not accept color signal, that would only be backward compatibility.

Forward compatibility does not implies to be able to decode the entire stream, it just means the device will work normally as previously without the new features. The difference it also mentionned on the Wikipedia page you quoted a few posts above.

B&W TVs do not understand the color information and cannot not display it, yet it's still called forward compatibility.
HDMI 1.0 Televisions cannot handle the hdmi1.3 bandwidth (i think they also cannot handle HDCP if i remember correctly), yet these TVs are still capable of doing the handshake with hdmi 1.3 devices and negotiate a format they understand.
So you can still plug a HDMI1.0 TV on a Sony Playstation 3, you won't get 1080p resolution and you won't be able to watch hdcp protected BluRays but the device will still work as if the Sony PS3 had an HDMI 1.0 emitter.

HDMI does provide both backward and forward compatibility so i understand it's easy to mix the two together.
Backward compatibility corresponds to using a Hdmi 1.3/1.4 TV on a Hdmi 1.0 emitter (the new device can use the old format)
Forward compatibility corresponds to using a Hdmi 1.0 TV on a Hdmi 1.3/1.4 emitter (the old device can use the new format as if it were the old format)

Look, I have an engineer degree in electronics and IT, and what you are saying is a bit naive. "Wiring" (tracing) is just the final step, you first need to define the logical schema of your device. It is true that tracing is a very complex process with undetermined results and is not guaranteed to complete in a working device or even to complete at all. But no-one "completely redesigns" everything with each revision of the hardware - there are component libraries which contain tested and proved schemes, a kind of production ready "building blocks". And complex chips are always designed as a system-on-a-chip, with micro-blocks connected through coponents buses - it would be simply impossible to trace a complete electrical scheme otherwise. Also, almost every non-RISC processing device is designed with user-updateable microcode ROM (IBM 360 and Intel Pentium/Core are the most famous examples), and dedicated DSPs (digital signal processors) almost universally use user-loadable microcode to implement their processing algorythms.

As for HDMI, the only serious improvement came with revision 1.3 which doubles the single-link video bandwitdth, and you don't need to "completely redesign" your chip for that, you just need to use better silicon manufacturing process and maybe do a re-trace, but it's part of the routine work of preparing chips for production on a given fab.

I knew there were these kinds of things but you make it sound like if it were much easier than i thought it was.
But even if it were not as hard as i imagined, I guess it's still a lot of work !

If you were in charge of designing the electronics of a TV what would you prefer :
1 - design two chips for unpacking/repacking the 3D frames and then be able to reuse all the other chips you already designed years ago almost unmodified (image treatment, resizer, framerate interpolater, OSD menus etc...) maybe at the cost of implementing the chips twice : one for each eye view, until you make new improved versions of these chips.
or
2 - keep the 3D frames in native format and have to modify all your other chips (modify the very building blocks that allowed you to gain time and effort) so that they support both 2D and 3D native image formats... and re-do that every time you design a TV system with a different native format

My guess is every single TV designer will answer #1 without hesitation..
If you answer #2, i guess your boss would believe you are bluffing and just want a pay rise

Using subframes clearly identified as left and right would also be much more forward compatible than checkerboard or side by side, as identifying the frames is the best way to provide future support for depth frames (for 2D + depth format, like Philips WOWvx technology), more than 2 views for other autostereo displays : (4 view displays, 8 view displays, like alioscopy displays), virtual data frames for defining occlusion frames (for 2D + depth + occlusion), etc...

The idea is that HDMI does not transmits pixels, but it's actually just binary data packed within boundaries that the hdmi chips call beginning and end of frames. within that data you can store pixel information in many different formats (8 bit, 10 bit, RGV, YUV, standard gammut or xvYCC gammut, 4:4:4 or 4:2:2/4:2:0/4:1:1 subsampled, etc...). so you could hide metadata or even ascii text within that data the Hdmi chips won't care, they'll just decrypt the data it believes to be the frame and hand it over to the TV chips to do the actual image treatment.

It's all cool, but dear Blackshark, your stated goal is to reduce complexity and save research budgets, yet you seek to achieve it by reimplementing every single known pixel format supported by HDMI using some new, different data structures, which do require redesigning the physical link interface to a much broader extent than any previous HDMI revision, and provide no new features that cannot be implemented with current revision of the interface. Do you see any irony in that?

Just let the physical link do its job of transmitting pixels and do all the stereoscopic frame processing on the software side, just as it happens with other line-interleaved, sequential and side-by-side formats.

You seem to have misunderstood what i was trying to say.
My intention was not to redefine the way hdmi chips communicate pixel data but rather to be able to reuse all these exact same data structures but and only add the required metadata for 3D hidden within these same data structures (that was the reference to the compressed video frame packing). Allowing the hdmi chip to transmit the two frames believing there were only one. And letting a separate add-on part of the hdmi chip or a separate chip do the actual required frame unpacking before delivering the two Left and Right separate frames to the TV for treatment and display.

You are taking the problem upside down.
The normal behavior of a TV is with all the gadgets and gizmos the TV manufacturer added.
The optionnal behavior is the "game" mode that circumvent all the extra stuff and just displays the image as raw as it can be to reduce latency.

Well, why not if this processing does offer subjectively "better" image quality for the majority of consumerss, who only watch DVD video and SDTV broadcasts on their new panels?

Also, please tell me how an inherently progressive, fixed-pixel Full HD display like LCD or plasma panel should be able to display 1080i, 720p or 576i signal whithout first performing deinterlacing or image stretching and color space conversion.

The answer is easy : they deinterlace, resize and convert the colorspaces, trying as best as it can ( politically correct expression for the word "failing") to make the image look as good as possible.

Now let me return the question : if you've got a 1080p progressive source and 1080p progressive TVs, would you be happy to downscale it to 1080i or 720p only to be upscaled again to 1080p because previous hdmi specs were not designed for such resolutions ? (That's what happened before the hdmi 1.3 update)
Now the same question with stereo3D TVs.
If you've got a Dual 1080p source (Blu-Ray 3D) and that both Hollywood (especially James Cameron) and Tv manufacturers (especially Panasonic) are pushing for Full3D 1080p per eye displays, would you be happy to downscale your images to 1920x1080 checkerboard packing only to be unpacked and upscaled again to dual 1080p ?

Hdmi 1.3 and 1.4 easily have the bandwidth to do full resolution @ 24Hz (The requirements for 3D are stereo 720p50 720p60 and 1080p24). so why in the world would it use checkerboard ???

However checkerboard does make sense as an optionnal transmission method that can be negociated between the source (a PS3 game) and a DLP3D compatible display.
Since it does not require any special interaction inside the hdmi chip, all the image treatment is done inside the TV (current DLP3D TVs only do basic treatment) i guess checkerboard can be added without modifying the hdmi 1.4 chips.

it makes much more sense to make it mandatory to send the source native format (two separate L and R images) to the display ... this is exactly what (in my opinion) the hdmi 1.4 standard is doing.

No. Frame packing is the only mandatory format for playback devices in the HDMI 3D spec.

As written before : I believe the HDMI 1.4 mandatory "Frame Packing" is sending the two separate L and R full resolutions images packed within special data into one single bigger frame (the frame sent to the display is bigger than the actual display size). I believe it is not checkerboard like you seem to suggest.

I think the only person who could clear this up is your contact at the HDMI group who could give us more details on this "packed frames" format

[DmitryKo]

full resolution separate Left than right images (what i called sub-frames) sent one after the other but packed within the data of the same frame ... it looks as frame sequential, with twice the amount of actual full frames ... from the outside it looks as if if were one single bigger frame, keeping the framerate identical with a 2D video

I believe the HDMI 1.4 mandatory "Frame Packing" is sending the two separate L and R full resolutions images packed within special data into one single bigger frame (the frame sent to the display is bigger than the actual display size).

(Sigh) I'm sorry, but "subframes" that somehow sum up to a full relolution stereo frame but only consume standard video bandwidth and are not checkerboard, line-interleaved, interlaced or side-by-side... this just doesn't sum up in a consistent way - unless you involve voodoo magic, or some Star Trek-esque design that uses subspace transmission!

As for a "bigger" frame, that would require filtering down and such image would look worse than just linear interpolated checkerboard; try to open some picture in Paint.NET and rescale it to say 66%, it would look too blurry.

My intention was not to redefine the way hdmi chips communicate pixel data but rather to be able to reuse all these exact same data structures but and only add the required metadata for 3D hidden within these same data structures (that was the reference to the compressed video frame packing). Allowing the hdmi chip to transmit the two frames believing there were only one. And letting a separate add-on part of the hdmi chip or a separate chip do the actual required frame unpacking before delivering the two Left and Right separate frames to the TV for treatment and display.

Yeah, and I want my 20 Mbit DOCSIS connection to be "packed" into a 10 Gigabit pipe

I think the only person who could clear this up is your contact at the HDMI group who could give us more details on this "packed frames" format

It's the "Site Admin" You can try it yourself, the contact form is here: http://www.hdmi.org/contact/" onclick="window.open(this.href);return false;

You can see what the RealD format looks like when playing AVATAR the game. It's not exactly side by side, it looks more like a hybrid between a checkerboard and side by side, and there are additional color bars at the bottom of the frame

Could you please post a screen capture, preferably with other supported methods as well?

Forward compatibility does not implies to be able to decode the entire stream, it just means the device will work normally as previously without the new features

Of course, but if the new system uses different transmission frequency and/or bandwidth, which the older device is not aware of, it will not be able to decode anything, right?

HDMI 1.0 Televisions cannot handle the hdmi1.3 bandwidth (i think they also cannot handle HDCP if i remember correctly), yet these TVs are still capable of doing the handshake with hdmi 1.3 devices and negotiate a format they understand.

Right, this is because HDMI 1.3 is backward compatible with HDMI 1.0.

The difference is, HDMI 1.0 would be forward compatible with HDMI 1.3 if an 1.0 device could connect at 1.3 rates such as 1080p120 but would drop some lines or frames that exceed pixel clock rates defined in 1.0 spec. As it is implemented, 1.0 is not forward compatible with 1.3 because an older device just won't even accept 1080p120 clock rates and there will be no data coming.

Backward compatibility corresponds to using a Hdmi 1.3/1.4 TV on a Hdmi 1.0 emitter (the new device can use the old format)
Forward compatibility corresponds to using a Hdmi 1.0 TV on a Hdmi 1.3/1.4 emitter (the old device can use the new format as if it were the old format)

Correct, but 1.3 source should transmit using new features (such as increased resolutions or frame rates, 48-bit pixels or extended color space) and 1.0 sink should still accept it; when an 1.3 source transmits in 1.0 format, it's backward compatibility.

If you were in charge of designing the electronics of a TV what would you prefer :
1 - design two chips for unpacking/repacking the 3D frames...
2 - keep the 3D frames in native format and have to modify all your other chips

Blackshark, I said it all before. You don't really have to modify anything, since DSPs use user-loadable algorythms. One example is synth workstations and digital DAWs - they have multiple DSP processors which can be loaded with quite differect effect algorythms.

As for stereo image processing, the processing chips would just have to do twice the work for an increased latency - and if you really care about low latency, just use 2 chips and cover it with the increased the price of the unit, since you will certainly charge consumers some premium for stereoscopic features.

Look at Canon DSLRs - once a new semi-professional camera comes out, it features twin processing chips for increased shooting speed and/or more sensor resolution, since the price premium for a high end model will cover the costs, and when a new consumer camera comes out, it usully features one new processing chip with increased performance, so to keep the prices low on this sensitive market. This way they avoid releasing new Digic chip with every new sensor that increases pixel resolution.

they deinterlace, resize and convert the colorspaces

Correct, but this was a rhetorical question.

If you've got a Dual 1080p source (Blu-Ray 3D) and that both Hollywood (especially James Cameron) and Tv manufacturers (especially Panasonic) are pushing for Full3D 1080p per eye displays, would you be happy to downscale your images to 1920x1080 checkerboard packing only to be unpacked and upscaled again to dual 1080p

That is probably not the most optimal scenario, but again, most people are happy with checkerboard on their DLP TVs.

Hdmi 1.3 and 1.4 easily have the bandwidth to do full resolution @ 24Hz ... so why in the world would it use checkerboard

As I said above, because 120 Hz were not defined and 3D Blu-ray spec was not finished back in 2006, when these DLP3D sets were introduced. Again, newer TV sets will use their native formats since HDMI 3D spec maintains that display devices must support every format; playback devices must only support frame packing, but I'd suggest that at least similarily branded players will support a format native to their brand stereoscopic TV displays.

[BlackShark]

Alright so here is the misunderstanding about frame packing.

You think a packed 1080p stereo frame fits within the same bandwidth as a normal 1080p frame, I don't.
I believe that one packed 1080p frame contains the two full blown 1080p frames each at normal resolution and bit depth.
YES ! it is twice the size !
YES ! it is different from frame sequential !
YES ! it is different from Side by Side full resolution !
And YES ! There is enough bandwidth !

From the specs I read, all the mandatory formats use Frame Packing. From the hdmi PDF there are 3 of them.
-frame packed 1080p @ 24Hz -> "for movies"
-frame packed 720p @ 50Hz for 50Hz TVs "for games"
-frame packed 720p @ 60Hz for 60Hz TVs "for games"
And the "for broadcast" format has yet to be defined but if your suppositions about CVT Reduced Blanking are correct, it will probably be frame packing 1080p @50/60Hz. If I remember correctly what you said in the other thread there should be enough bandwidth with CVT-RB right ?

All the rest :
side by side squashed
side by side full resolution
field alternate
frame alternate
line alternate
2D+depth
other frame packing
the lately added top/bottom

All of them are 100% optionnal.

Neither devices or displays have mandatory support for these formats, the only ones mandatory are the 3 frame packed.

I think the only person who could clear this up is your contact at the HDMI group who could give us more details on this "packed frames" format

It's the "Site Admin" You can try it yourself, the contact form is here: http://www.hdmi.org/contact/" onclick="window.open(this.href);return false;

What a disappointment ! I thought you had a top secret mole within the hdmi group


About the RealD format
Here are the RealD frames from Avatar the game
(CAUTION the "swap eyes key is switched ON, so the eye views are reversed !)

[DmitryKo]

Here are the RealD frames from Avatar the game

Khmmm... interesting. It's actually a variation of the checkerboard format, though the difference will only show for objects rendered at screen depth, such as menus and HUD elements - such items would look identical in both eyes down to per-pixel level, unlike true checkerboard where interpolation could introduce some errors because the sample points are different.

The PRs say there's some additional color filter processing - I can't see it.

You think a packed 1080p stereo frame fits within the same bandwidth as a normal 1080p frame, I don't.
I believe that one packed 1080p frame contains the two full blown 1080p frames each at normal resolution and bit depth.

Sorry, I don't think you provided valid reasons why any new format would be better than frame sequential and side-by-side.

I'd still suggest frame packing is either some variant of the checkerboard (maybe the RealD format above), or some form of pixel packing, say 48-bit pixels which are actually two 24-bit L+R pixels.

if your suppositions about CVT Reduced Blanking are correct, [the broadcast format] will probably be frame packing 1080p @50/60Hz

This is not related (and I'm not going to spend $500 just to buy a working copy of HDMI 1.4 and CEA-861-E standards).

The broadcast format will have to be forward compatible since the infrastructure does not support 1080p50/60 broadcasting yet, so it should be based on 720p50/60 and 1080p24/25 with some "half resolution" encoding. So far it looks like the frame format will be top-bottom and HDMI spec will be soon amended for that.


PS. Please take a look at the free HDMI 1.3a specification and see how this interface is built around continously sending invididual pixels. It's so tight they would have hard time to find some space for any frame markers.

[DmitryKo]

I've just recalled how I came to use "quincunx" in relation to checkerboard; actually, it comes from MPEG/VCEG draft for H.264/AVC Multiview Video Coding, in particular the "Frame Packing Arrangement SEI message". In the document, the checkerboard pattern is referenced to as "checkerboard (quincunx)", since missing pixels have to be interpolated from 3 (at sides) or 4 neighbouring pixelsm and this method has very long been known as "quincunx sampling".

This made me re-examine the text of the most current draft of the H.264 standard - JVT-AF11.zip - which contains the syntax of the frame packing arrangement message, and now I believe that "frame packing" in HDMI context is indeed a "container" structure which defines every possible "half-resolution" packing method - that could explain why it's not present in some slides, since it's not a separate format.


SO, I think the correct list of supported "3D over HDMI" formats in HDMI 1.4 and CEA-861-E specifications should look like this:

* Frame packing ("frame compatible"): mandatory;
- includes 1080p24 (Blu-ray 3D), 720p50 and 720p60 (stereo gaming) signal formats (all three formats are mandatory for displays/sinks, at least one is mandatory for players/sources);
- includes the following packing arrangements (probably a combination of several predefined patterns and checkerboard sampling, see the bottom paragraph):
-- checkerboard sampling (TI)
-- line-interleaved (VREX etc.)
-- side-by-side half
-- side-by-side half with checkerboard sampling (RealD)
-- top/bottom half (in a future revision)
-- top/bottom half with checkerboard (?!!)
-- column-interleaved (?!!)
-- (?!!)

* Field alternative full (frame sequential) - 720p50/100/120 and 1080p48/50 (100/120?!!), optional
* Field alternative half (interlaced signal) - 1080i50/60, optional
* Line alternative full - 1920x2160p24/25/30, optional
* Side-by-side full - 3840x1080p24/25/30, optional
* Top/bottom full - 1920x2160p24/25/30, optional (in a future revision)
* 2D + depth (WOWvx) - 1080p60, optional
* 2D + depth + graphics + depth (WOWvx Declipse) - 1080p60, optional
* Additional manufacturer-defined combinations of 3D structures an video signal formats (720p24/25 etc)

Note: 24, 30 and 60 formats include NTSC 1000/1001 equivalents 23.975, 29.97 an 59.94 Hz; 25i/30i formats may be presented with 50i/60i singal in some hardware
I'm not really sure about any items marked with (?!!).


For reference:
Frame Packing Arrangement SEI message can be used to define the following common arrangements (illustrated by figures D-1 to D-7 in the text of the draft): D-1. Checkerboard interleaving (frame_packing_arrangement_type = 0, quincunx_sampling_flag = 1); D-2. Column interleaving (type = 1); D-3. Row interleaving (type = 2); D-4. Side-by-side packing (type = 3); D-5. Top-bottom packing (type = 4); D-6. Side-by-side packing with quincunx sampling (frame_packing_arrangement_type = 3, quincunx_sampling_flag = 1); D-7. temporal interleaving (type = 5)
I believe this message only matters for legacy content and all those recently announced "half-resolution" broadcast 3D channels, which would have to work with legacy set-top-boxes that do not use Stereo High profile - and HDMI Licensing LLC currently allows the use of HDMI 3D formats for legacy HDMI devices, so they will still be able to talk with these newly announced 3DTV displays. 3D Blu-ray is said to use full resolution stereoscopic encoding offered by Stereo High profile, so it would not use any frame packing.

[ssiu]

The interesting discussions are over my head and a bit too theoretical for me, but I have practical questions like:

-- would my PS3 work on all future 3D HDTVs (i.e. would the 3D signal it outputs be a mandatory type), or I need to check each brand

-- if all the real "full quality highend" signals are optional, could we have a situation like: Sony chooses to support side-by-side full only, and Panasonic chooses to support top/bottom full only, so we get full quality when using "Sony 3D Blu-Ray player --> Sony 3D HDTV" and "Panasonic 3D Blu-Ray player --> Panasonic 3D HDTV", but not when we mix brands, which defeat the whole purpose of a standard

[taz291819]

The interesting discussions are over my head and a bit too theoretical for me, but I have practical questions like:

-- would my PS3 work on all future 3D HDTVs (i.e. would the 3D signal it outputs be a mandatory type), or I need to check each brand

-- if all the real "full quality highend" signals are optional, could we have a situation like: Sony chooses to support side-by-side full only, and Panasonic chooses to support top/bottom full only, so we get full quality when using "Sony 3D Blu-Ray player --> Sony 3D HDTV" and "Panasonic 3D Blu-Ray player --> Panasonic 3D HDTV", but not when we mix brands, which defeat the whole purpose of a standard

For Full 1080p, yes, it defeats the purpose. But the displays may also incorporate frame packing, thus being compatible with any 3D player (half resolution though).

[ssiu]

For Full 1080p, yes, it defeats the purpose.

I can't believe they allow it to happen and call it a standard, if that is really the case ... They could mandate that "if your device support any full 1080p signal, then you must support at least this one particular full 1080p format (and leave all other formats optional)". Then we won't have interoperability issue like that.

[taz291819]

For Full 1080p, yes, it defeats the purpose.

I can't believe they allow it to happen and call it a standard, if that is really the case ... They could mandate that "if your device support any full 1080p signal, then you must support at least this one particular full 1080p format (and leave all other formats optional)". Then we won't have interoperability issue like that.

Well, just look at the ATSC spec, the primary channel has to be MPEG2, but subchannels can be MPEG4, which wouldn't be compatible with most (if not all) current STBs. Not to mention, I don't believe there is a single broadcast station in the U.S. that is using MPEG4 on a subchannel, though they should.

And look at standard BD players. None of them have to decode DTS HD MA either, but they do have to be able to decode the regular DTS core.

[DmitryKo]

would my PS3 work on all future 3D HDTVs

Since Sony chose to upgrade PS3 to HDMI 1.4, yes it should; however it would probably use "half-resolution" frame packing only.

if all the real "full quality highend" signals are optional, could we have a situation like: Sony chooses to support side-by-side full only, and Panasonic chooses to support top/bottom full only ... which defeat the whole purpose of a standard

Yes, exactly, this is the way these folks chose it to be. This sh.t happens with industry-backed consortiums all the time, vendors are just backing their proprietary formats and interfaces and are unwilling to make any radical departure for something better. DisplayPort 1.1a/1.2 looks so much better in this regard... I'd predict that no-one will ever implement anything beyond "frame packing" as a result. The quality benefit of "full-resolutions" formats is somewhat to be seen though.

just look at the ATSC spec, the primary channel has to be MPEG2, but subchannels can be MPEG4, which wouldn't be compatible with most (if not all) current STBs.

AFAIK the transport stream must be MPEG-2, but the encoding can be H.264/MPEG-4 AVC; this is a 2008 amendment. You're right that this update came too late and H.264 probably won't take off in the USA until the launch Full HD stereo channels which will require entirely new STBs, if ever, since H.265 is already nearing completion.

[BlackShark]

From the hdmi pdf i read, both side by side half and full resolutions were in the optionnal format list.

I still believe that frame packing is a container for one unique clearly identified way of transmitting left and right images at full resolution. which is different from any of the current ransmission methods used on non hdmi 1.4 displays. And that the PS3 will support it as it is mandatory.
The game rendering may happen at full res or upscaled but the transmission should follow the mandatory format (which is stereo 720p 50/60Hz for games)

just look at the ATSC spec, the primary channel has to be MPEG2, but subchannels can be MPEG4, which wouldn't be compatible with most (if not all) current STBs.

AFAIK the transport stream must be MPEG-2, but the encoding can be H.264/MPEG-4 AVC; this is a 2008 amendment. You're right that this update came too late and H.264 probably won't take off in the USA until the launch Full HD stereo channels which will require entirely new STBs, if ever, since H.265 is already nearing completion.

Yes the ASTC specs arrived too early in the US and allowed the use of mpeg2 for HD content.
In europe the DVB specs arrived later and preferred mpeg4 AVC for HD content. In France by law all new HDTVs larger than a certain size (56 centimeters i think, not sure) must be equipped with a mpeg4 HD DVB tuner.

[DmitryKo]

I still believe that frame packing is a container for one unique clearly identified way of transmitting left and right images at full resolution

That would be quite non-standard use of the term that is clearly associated with half-resolution methods.

BTW, did you watch the interview with President of HDMI Licensing LLC? These folks can't even properly implement standard features such as L/R frame marking in sequential stereo, and yet you expect them to introduce some brilliant yet unknown full-resolution format that uses twice less the bandwidth and features an extensible container format with metadata to support all possible future applications. I don't buy it if they can't even copy someone else's ideas without screwing them up too much.

the ASTC specs arrived too early in the US

At least the USA launched its digital HDTV services 5 years earlier than Japan and Europe did. But stil ATSC should have introduced MPEG-4 AVC back in 2005-2006 to match the projected analog switch-off date, 2008 was clearly too late.

[BlackShark]

you expect them to introduce some brilliant yet unknown full-resolution format that uses twice less the bandwidth

Nope : that's only your interpretation of the expression "frame packing"
I expect an hdmi packed stereo frame to use twice the bandwidth of a standard frame for full resolution transmission.

Bandwidth isn't an issue : there is enough bandwidth for full resolution stereo 720p @50/60Hz and stereo 1080p @24Hz, which are what the hdmi pdf revision clearly states as the mandatory formats.

[AlexHD]

[wrong assumptions deleted]

[DmitryKo]

The choice of 720p/60 and 1080p/24 as the only mandatory 3D formats is obviously made to be able to pump 3D through "standard" (not "high speed") cables, as they are only capable to pass 720p/60 and 1080p/30, but not 1080p/60

Yep, HDMI Standard Speed cables are indeed rated for 74.5 Mhz or 1.78 Gbps only, and that's only enough for 720p60, 1080p30 and 1080i60.

What's more, there is really 1080p60 support, except for gaming consoles and the PC. I wonder how many surround sound receivers would accept and decode standard 1080p60 signal, which could be used to carry full-resolution 1080p48 sequential stereo movie content? I guess not many, except for some very high-end models - which means most people won't get any sound from their brand new Blu-ray 3D certified player. Talk again about lack of any foresight on part of HDMI, and how it punches you in the back when you don't really expect it.

[AlexHD]

Now the question is, what is the packing algorithm. Probably something like those RealD game screenshots.

How are checkerboard-multiplexed stereo pairs usually supposed to be decoded: via a standardized straightforward algorithm that is the same in every implementation, or it is like deinterlacing, where every implementation is different and no particular algorithm is perfect?

[DmitryKo]

Now the question is, what is the packing algorithm

I think it does at least include RealD side-by-side checkerboard (see above) - at least 3 TV companies announced they signed licensing deals for it, they wouldn't do it for some entirely optional format...

standardized straightforward algorithm that is the same in every implementation, or it is like deinterlacing, where every implementation is different

Quincunx sampling is inherently progressive, it has samples on every row so there should be no pitfalls, not even to close to the problems with the interlaced signal.

If you check the latest draft for H.264 (VCEG-AM11.zip ISO/IEC or JVT-AF11.zip ITU-T ) for "Frame packing arrangement SEI message" which describes legacy "frame packed" stereo (non the latest MVC or Stereo High), it says the missing pixels should be averaged among the neighbours - which practically means linear or bilinear interpolation - but does not specify the exact algorythm; I don't think HDMI 1.4 specifies it as well.

[taz291819]

The choice of 720p/60 and 1080p/24 as the only mandatory 3D formats is obviously made to be able to pump 3D through "standard" (not "high speed") cables, as they are only capable to pass 720p/60 and 1080p/30, but not 1080p/60

Yep, HDMI Standard Speed cables are indeed rated for 74.5 Mhz or 1.78 Gbps only, and that's only enough for 720p60, 1080p30 and 1080i60.

What's more, there is really 1080p60 support, except for gaming consoles and the PC. I wonder how many surround sound receivers would accept and decode standard 1080p60 signal, which could be used to carry full-resolution 1080p48 sequential stereo movie content? I guess not many, except for some very high-end models - which means most people won't get any sound from their brand new Blu-ray 3D certified player. Talk again about lack of any foresight on part of HDMI, and how it punches you in the back when you don't really expect it.

Actually, quite a few AV receivers handle 1080p60 just fine. I don't hear any complaints over at AVSForum about it, as many folks use the newer ATI HD 5000 series to bitstream the newer audio codecs to their AV receivers, and most run at 1920x1080@60Hz.

[Jadentheman]

I rhought it would have happen. But once again companies manage to screw up a standard for blu-ray. WTF!! Isn't this exactly y the reason why they had these meetings and proposals to the BDA?

HDMI also should have known better than to half ass another standard. People are getting sick and tired of this. One of the factors why 3D at home won't be as good as just going to the theatres

[DmitryKo]

quite a few AV receivers handle 1080p60 just fine. I don't hear any complaints over at AVSForum about it

Well, then I can't come with any valid reason why the speed of a $15 cable came to be the limitation for the standard level of stereo 3D features.

It does look like the vendors just wanted to restrict their players and displays to only use a full-resolutuion transmission format with devices from the same brand, as discussed above.

[DmitryKo]

I rhought it would have happen. But once again companies manage to screw up a standard for blu-ray. WTF!!

This is not a standard for Blu-ray, this is a standard for general device connectivity. Blu-ray will use Multiview Video Coding extension and Stereo High profile which is based on intellectual property largely owned by TDVision, according to TDVCEO, and uses 2D+delta compression to deliver a separate full-resolution view, in a way compatible with standard mono encoding. So this is a major screw-up on part of HDMI founders, that is Hitachi, Panasonic, Philips, Silicon Image, Sony, Thompson, and Toshiba, not the BDA association.

[DmitryKo]

From the publicly released 3D portion of HDMI 1.4 spec, it looks like the 1080p24 and 720p50/60 mandatory frame packing formats are indeed full-resolution (for progressive video) top-bottom picture formats (with aditional guard band interval in between the frames) and so they have twice the bandwidth of the similar mono signal. Frame packing at 1080p50/60 and 1080i50/60 are defined as optional formats. Interlaced video is supported, so it's basically a format that can support both full-resolution frame sequential (active shutter glasses) and half-resolution passive polarized line-interleaved formats. (Hats off to Blackshark, seems like you nailed it down pal).

Standard definition resolutions such as 720x480p are supported for stereo as well.

Quincunx sampling and horizonal interleaving are a part of the optional side-by-side half format; this includes Real D format. TI's checkerboard format is NOT included; curiously, line (vertical) interleaved half is not included as well. I guess the latter will be added in the forthcoming revision 1.4a.

Other formats include side-by-side full, field alternative (interlaced stereo) and line alternative full. 100/120 Hz frame alternative format is NOT included, however sequential frame devices will be effectively handled by the frame packing format.


I have updated my bandwidth and resolutions table with some common frame packing and side-by-side formats.

[taz291819]

It's strange that they didn't include the checkerboard method, but it certainly brings up a nice conspiracy theory.

Possibly for the extra revenue Mits and/or RealD will get for the converters?

[DmitryKo]

It's strange that they didn't include the checkerboard method

They did include checkerboard, it's just not TI's checkerboard but RealD checkerboard But even if HDMI did include TI's variant (or will include it in a future revision), a converter box or a firmware update would still be required for current DLP TVs, since the supported stereo modes are communicated with HDMI control messages which are not defined in HDMI 1.3. So the decision was really up to Samsung and Mitsubishi, and they opted not to implement support for HDMI 3D formats in a firmware update, even though HDMI LLC does allow legacy HDMI 1.3 devices to support any of HDMI 1.4 3D formats and do so without supporting all the mandatory formats.

[AlexHD]

From the publicly released 3D portion of HDMI 1.4 spec, it looks like

For viewing convenience (not to have to fill the form), I've uploaded it to http://www.sendspace.com/file/j0m2k8" onclick="window.open(this.href);return false;

[taz291819]

What's the difference between RealD's checkerboard format and TI's?

[DmitryKo]

It's really only the grouping of pixels - TI's checkerboard is fixed at odd sampling for the Left frame and even sampling for the Right frame, while the side-by-side half format supports diferent sampling methods.

I think RealD samples the Left frame at the same positions as the Right frame (either odd/odd or even/even), which means objects at screen depth (zero convergence) will look identical in both eyes, giving a better stero picture since the interpolation to the full resolution would introduce less artefacts.
Here is the screenshot from this thread; see 3D Portion Of HDMI Version 1.4 Available For Download thread for a diagram which explains possible sample positions in horizontal and quincunx sampling.

[taz291819]

That makes sense.

Basically, Mits' converter needs to incorporate both of these in a single device:

http://www.reald.com/Content/POD-Seq-Page-Flip.aspx" onclick="window.open(this.href);return false;

and

http://www.reald.com/Content/POD-Side-B ... rlace.aspx" onclick="window.open(this.href);return false;

If so, the Mits converter would be compatible with 3D Blu-Ray and Directv's 3D (which is using RealD's SbS format).

Is this a safe assumption? (Not saying that's what it's going to do, but plausibly)

[DmitryKo]

Mits' converter needs to incorporate both of these in a single device ... Is this a safe assumption?

The broadcasting encoding format is independent from video link format. You can use some legacy half-resolution pattern in broadcasting, since your set-top box won't support TDV-codec for MPEG-2 or H.264 MVC "Stereo High" profile, but the decoded picture would be upsampled to full-scale 3D in the set-top box and transmitted as full-resolution frame-packing to a 120 Hz frame-sequential LCD TV.

I guess Mitsubishi converter will certainly accept standard frame packing formats, since these would be required for Blu-ray 3D movies and PS3 gaming; anything else is a big unknown now. Their TVs will still continue to accept standard checkerboard from the PC, so there's no real need to implement each and every stereo format, including the ones which require a 3rd party license.

[NASAGuy]

Wow.. in the dozen or so various Forums (computer, rocketry, film, etc) that I'm a member of, this has to be the most technically advanced group I've come upon (meaning that as a compliment)

I've learned more in this single thread about the technology (or possible technology) than any other thread I've ever looked at..

Thanks to everyone for being as knowledgeable as you all are and I can only hope to learn as much as I possible can from you all..

[BlackShark]

Nice to see my intuition was right, but it was mainly guessing from the numerous declarations of Panasonic then joined by Sony consistently requesting the BluRay3D format to be fullHD in both eyes.
I didn't make any sense to me that Full3D would not be mandatory since the display technology is already available and widely backed by almost every single display manufacturer (Full resolution shutter based displays).

So hdmi1.4 frame packing sends the two full pictures (which i used to call sub-frames) with each individual sync signals (the non visible data pixels around the image) and calls the whole package as one frame is that right ?
I haven't read the published documents yet but in the additional sync data around the frames, do you know if there is specific metadata to distinguish the left and right eye frames or is the frame order already imposed from the hdmi spec ?