There is too much 4K inertia at trade shows and the effort is concentrated on the concept of displays with four times the pixel count, and less on better methods of improving image quality.
Other than the professional trade meetings I regularly attend, too little is said about the concept of better quality for 4K pixels, or better quality for even current HD pixels. The industry has jumped into 4K without considering the possibility of improving the pixels of HD during 16 years of HDTV implementation since 1998. Is it too late for that now that we have 4K?
At CEDIA, and at the yearly Society of Motion Pictures and Television Engineers (SMPTE) Technical Conference in Hollywood this past October, we discussed again how 4K pixels can be improved, the "better pixels" concept.
Ideas such as using more bits than the current 8-bit color depth, and pursue High Dynamic Range, less color compression than the current 4:2:0 chroma sub-sampling that discards 75% of the color data of the 4:4:4 uncompressed original, faster frame rates than the current 24/30fps (film and video sources respectively) to smooth out the temporal presentation of the content, and using a larger color space than the current HDTV Rec. 709 to a better UHDTV Rec. 2020 to show colors the human eye can see, have been proposed and discussed in the engineering meetings for several years, but, what is the catch?
Any combination of the above improvements, such as to 10 or 12 bit color depth, 4:2:2 chroma sub-sampling, and 120/240 frames per second, have an impact:
a) More bandwidth would be required to transport the higher-data signal even when using a more efficient HEVC compression algorithm,
b) More space would be required for storage and discs (the 4K Blu-ray specification is expected for 2015 and it was announced that it will be 10-bits at 60fps on larger capacity discs), and
c) More data means also longer download times for 4K content, possibly exceeding ISP data caps and incurring in higher cost for the Internet service, as I cover in these two recent articles ("total cost of ownership", and "pricing" sections, respectively).
A question that is often asked at the meetings is:
"Which of those improvements will bring the best bang for the buck?" And so far there is a consensus of High Dynamic Range to show very bright images in contrast with good blacks, and faster frame rate (at least 120fps).
While most attendees at those meetings discussed how and when those improvements may eventually be applied to 4K, I wonder how current Blu-Ray and HDTVs could benefit from the "better pixels" idea if that improvement could be done with a simple and cost conscious approach, not benefitting only 4K.
Perhaps the idea may be worthwhile if some improvements could be done to the image quality of current HDTVs to a level that would be easily appreciated by consumers, and 4K can still run its due course but not as the only beneficiary of the "better pixel" effort.
That brings to surface the hard question to respond:
"Was jumping to a 4K format with just 4 times of the same HD quality pixels for the first years of UHDTVs to later add the quality improvements mentioned above actually the best way to improve image quality to consumers? Or it was just the best way to sell more (and expensive) TVs?"
If in 16 years the industry did not make those image improvements on HDTV what are the chances that the industry would apply all of those improvements at once to 4K UHDTVs/content when implementing gradual improvements has the potential to sell more TVs over the next decades to the same consumers again? The audio and video industry loves that concept of upgradeability.
On the consumer side, most are not going to change their viewing distance and rearrange sofas to sit half-way closer to the set because 4K can be better appreciated at that closer distance, but they may certainly notice a difference if their HDTV has an improved dynamic contrast without having to move one inch from where they were viewing the HD image.
Some products were demoed that claimed to provide similar improvement to any resolution not just 4K, like the improvement Darbee does to HD content (a product I reviewed here).
At SMPTE I met again Joe Kane while attending similar tracks related to our common interest of image quality. We exchanged several ideas about this subject, not only technical, we discussed how feasible the concept of improving HD may be embraced by a TV industry that jumped to 4K in full force to sell more TVs, and how it could be welcomed by a consumer that could actually see an improvement in image quality without investing in a 4K UHDTV.
As Joe says "UHD TV should be a system where at least some of its benefits would be available to everyone who has already invested in a 720p or 1080p TV with an HDMI connection. UHD should replace HD, but include HD in a way where everyone would potentially see better picture quality. It's the modern version of set top boxes for DTV when it first came out. Your old CRT set displayed a better picture when tuned to a DTV station through the set top box."
After several days of technical sessions, engineering paper presentations, and mental challenges, Joe cordially invited me to his "viewing lab" at his home, I use that name because, as it happens with my "viewing lab", the equipment, screens and home theater are primarily used to calibrate, test equipment, and review content rather than for personal entertainment.
They are available day and night at the mercy of one more test, review, component change, content evaluation, or demonstration. In my case I put more lamp hours on my 4K projector for testing and reviewing equipment and content than for enjoying movies, although I still do when I need a break from "appreciating" video artifacts most of the day.
We sat on his "viewing lab" while viewing content and exchanging ideas about screen materials, calibration settings, past and future of image quality, what can we do about it, and to evaluate if the inertia of the TV industry would be interested on the idea of improving "old HDTV".
We continued our exchanges over dinner and agreed that it should not be too late to offer a "better pixel" HDTV image to millions of people with good working condition HDTVs that do not need to be replaced by still evolving 4K UHDTVs and 4K content, which eventually may implement "better pixel" improvements, typically gradually on new sets, that consumers are expected to buy again.
I told Joe that I wanted to write an article about this subject. Since he wanted to also share his ideas with the readership I suggested for him to do it in his own words in an interview style, as follows:
Joe, you are a known figure in the TV industry for pursuing the best in video image quality since the early days of NTSC and CRTs, and for introducing the concept of ISF, please introduce yourself:
I got my start in this industry at Eastman Kodak in 1973. It was the year electronic news gathering got started and EK was worried about the life of film. It was no longer "News at 6, film at 11". In our research we were always trying to make the best pictures possible from the existing system, and doing it by the rules of the system. It was an early lesson in not being able to do any better than the system specifications at the same time realizing few people ever understood how much was possible under those specifications. I found my calling in helping consumers see more of what could be produced. I've played my part in many of the innovations that expanded the capability of what we could deliver in image quality.
What possible image quality improvements can be applied to today's video formats?
As much as we seem to be inching forward in what can be delivered to consumers, the higher resolution of UHD as an example, we have yet, with just a few exceptions, to build display devices that can take full advantage of the current system. From one point of view it may take a significantly better system to help consumers step up to the current capability of HDTV. Another point of view surrounding a future UHD system may be the educational step needed to get consumers up to speed with what can be done in image quality.
A good example of where UHD could bring better image quality to consumers with current TV technology in their home comes from most HD content being produced at a higher level than is currently conveyed to the consumer. Who would have guessed we built an electronic highway to the consumer that wasn't capable of transporting the quality we created?
If you understand what I just said you'll realize one of the most important parameters to be defined for UHD is the highway to the consumer. All of the potential improvements we talk about for UHD are worth nothing if we can't get it to you!
The mantra in Hollywood is More, Better, Faster. More pixels, meaning higher resolution, better pixels, providing the bit depth to support a large color space(s) and better dynamic range, and faster, more frames per second, smoother motion. None of it is possible without the highway. So I ask: why isn't the highway the main topic of UHD conversations?
How could the current HD infrastructure and consumer equipment be reused to capitalize from such quality improvements?
Current technology in DisplayPort, DVI and HDMI connected sets allow the sets to communicate with the source device. That alone suggests UHD coming down our ideal highway could contain versions of programs specifically designed to look good on particular sets. The consumer would have to be responsible for picking the right display options in the user menu system of the set to get the desired picture quality because as of yet we don't include a capability of the incoming signal setting up the TV.
I feel that if we were to deliver program content to consumers at the quality level at which it is produced we'd have a potential of consumers seeing a better image quality in currently available TV sets. It's unfortunate that we'd have to send along a separate set of instructions telling consumers which user selectable options they need to select in the set's menu system. In a future system we could do that automatically, giving consumers the option of not accepting instructions from the program producer.
What hardware or software has to be built to complement existing HDTVs so consumers can appreciate the image improvements?
Making existing 720p and 1080p TV sets compatible with an ideal UHD TV system would likely require a set top box. The box would be capable of understanding what the TV would need from the UHD signal to produce a good picture and then format the UHD content to provide the signal the TV set would like to see.
The capabilities of the set top box, the UHD signal interface, would be built into future TV sets of varying capability. In other words, as much as we expect 2160p to be a common pixel count for new TV sets you will likely have lots of other options in what you buy. The set may only have the higher pixel count capability. Other sets might include a larger color option or a larger dynamic range. The set you own would tell the signal interface what it can do and then pick the right option out of the content to work with your set.
Staying with what is currently available in UHD content, 8-bit, 4:2:0, 24/30fps, Rec. 709, but adding the idea of a set top box as an interface between the signal and a 1080p TV set, is it possible a current HDTV set will display a better picture quality using UHD content?
The set top box isn't needed or even useful until we get beyond delivering UHD content in a better way than we are now doing. One could argue if a 2160p 8 bit 4:2:0 signal were down converted to 1080p for display on a 1080p set the picture quality would be better than if the content originated at 1080p. I don't see this situation being important. Few people will go out of their way to get 2160p content only to convert it to 1080p. In fact most 2160p content will already be available as a 1080p signal. The 1080p version of the content might look better just because it was produced at 2160 instead of 1080. Beyond the idea of over sampling, creating at 2160p and watching at 1080p, there is nothing better to be had for the consumer if the content is delivered as 8 bit 4:2:0.
You mentioned the Rec 709 color space. I want to talk about it in a different context. Every display, including those to come in the next few years, must have Rec 709 color capability if the display is to comply with existing HD and UHD methods of delivering video. There are currently UHD sets on the market that don't have Rec 709 capability. Those particular set manufacturers will tell you the sets have 'better' color. The problem is their 'better' color is different from what is required to properly display the picture. Therefore their 'better' color is wrong.
The set top box will come into play when we start delivering a minimum of 10 bit 4:2:2 content, which is the format in which most 1080p content is archived. Even if the set is only capable of an 8 bit input, the down conversion from 10 bit to 8 bit could greatly reduce the visibility of contouring in the image, therefore improving the picture quality on an existing set.
The HDMI connection will allow a 10 bit 4:4:4 signal to be passed to the set in 1080p, but there is little 4:4:4 source video material.
How such equipment is to be connected to HDTVs, AVRs, and Home-Theater systems to minimize consumer expense when having multiple video sources that can be improved?
In being compatible with existing TV sets most everything would happen from the set top box to the existing A/V system through an HDMI connection. I mentioned two other options DisplayPort and DVI. If you need to use either one there may be more connections involved. DVI, as an example, doesn't usually carry audio so you'd need a separate audio connection.
I don't yet know what the set top box will have for an input. One would suspect an HDMI connection might be a starting point, but USB or DisplayPort or WiFi sources might also be available as the UHD signals will likely come from internet connected devices.
Could such equipment improve SD as well? HD 720p? HD 1080i? and HD 1080p Blu-ray? And what type of image improvement a consumer may expect over each of those resolutions? Is the primary beneficiary actually HD?
SD?, what's that? Just kidding. I do have a rather large collection of laserdiscs and DVD's. I suspect up conversion of SD content will continue to be handled as it always has been, with the exception of the processor having a far better bit depth. There won't be a lot of benefit to SD in UHD. Depending on circumstances some will see an improvement in SD sources converted to 1080p or 2160p, but up converted SD will always be limited by SD.
In a way all existing content will suffer from what it is, it being less than could be done in an ideal UHD system. Of course all UHD systems will have to accept existing content and there will always be those who will find ways of 'improving' the image quality of older source material.
Does the improvement require a different type of original HD content? Or existing HD content can be reused and be pre/post-processed with the "better pixel" ingredient to obtain the expected image improvement?
I see UHD as being multiple things, a system to replace HD, dealing with all of our existing content, and a new system freed up from the confines of our current system. The new system of More, Better, Faster needs to be set out in a way where we can grow within the system.
I've had a chance to see your UHD display. Would you share something about it with our readers?
I'm using the Samsung 85 inch S9 full backlit LED LCD set. I never imagined having such a device in my home so when it became necessary to have it for research purposes it went in the living room. The lighting and color of the walls is not right for such a display but we're making the most of it. The seating distance is at 1.4h, (where 'h' is the height dimension of the image) a bit closer than the 1.5h recommended in the ITU 1769 document. I've often treated their numbers as a maximum seating distance. In my home theater I use 2.5h instead of the 3h found in the document for a 1080p image. The rest you can see in the picture.
Do you have any final comments on this subject?
In reality UHD shouldn't be anything about the display. It should free us up from the narrowly defined SD and HD systems, allowing for all sorts of innovations in display technology and or program production capability to come. UHD needs to define the package or container in which the signal comes to us, allowing for all those options to come. It also needs to define the way in which it comes to us, the highway I discussed earlier. We hear about HEVC as a compression system, part of the current definition of the highway, but that is essentially already out of date if you consider More, Better, Faster. Part of defining the highway is coming up with a compression system that will deal with all of the options we'd like to put into the container.
Another part is defining how we deliver instructions. Some of you know it as metadata. We can do a lot with a single source and lots of different sets of instructions on how to use it. Come up with a new display a few years from now and send a new set of instructions on how to use the signal in the container to work with the new set.
Think about what I just wrote and you'll understand where I believe the set has no part to play in defining the new system. The ideal UHD system will work with future displays, things we haven't thought of yet. It will work with existing 720p and 1080p sets with HDMI connections and all of the current, so called, UHD sets.
By the way, there are those who think that "More" also means 4320p, four times the resolution of 2160p. We're starting to create the test signals for it.
Thank you Joe for advising and supporting TV manufacturers and consumers in the search for the best image quality.
Posted by Rodolfo La Maestra, December 16, 2014 7:30 PM
About Rodolfo La Maestra
Rodolfo La Maestra is the Senior Technical Director of UHDTV Magazine and HDTV Magazine and participated in the HDTV vision since the late 1980's. In the late 1990's, he began tracking and reviewing HDTV consumer equipment, and authored the annual HDTV Technology Review report, tutorials, and educative articles for HDTV Magazine, DVDetc and HDTVetc magazines, Veritas et Visus Newsletter, Display Search, and served as technical consultant/editor for the "Reference Guide" and the "HDTV Glossary of Terms" for HDTVetc and HDTV Magazines. In 2004, he began recording a weekly HDTV technology program for MD Cable television, which by 2006 reached the rating of second most viewed.
Rodolfo's background encompasses Electronic Engineering, Computer Science, and Audio and Video Electronics, with over 4,700 hours of professional training, a BS in Computer and Information Systems, and thirty+ professional and post-graduate certifications, some from MIT, American, and George Washington Universities. Rodolfo was also Computer Science professor in five institutions between 1966-1973 in Argentina, regarding IBM, Burroughs, and Honeywell mainframe computers. After 38 years of computer systems career, Rodolfo retired in 2003 as Chief of Systems Development from the Inter-American Development Bank directing sixty+ software-development computer professionals, supporting member countries in north/central/south America.