Overview of 8K AI Upscaling Technology

Experts have not yet finished arguing about the prospects for 4K TVs, but companies are already actively offering 8K models. Basically, skeptics of this trend use two fairly convincing arguments.

Introduction

First, many have not forgotten the relatively recent history of 3D TVs. The rapid growth of its popularity began after a demonstration at CES and IFA in 2010. Many experts quite logically predicted its excellent prospects. Unfortunately, the high cost of producing 3D content has become an insurmountable obstacle to its further development. As a result, almost all companies abandoned 3D support in their TVs.

The second argument is based on the eye capabilities. Unfortunately, its maximum sharpness is significantly inferior, for example, to eagle vision. Of course, it would never occur to anyone to watch a 75-inch TV from a distance of, for example, up to 1 meter. But at a more adequate distance, the sharpness of our vision does not allow us to distinguish individual pixels smaller than a certain size. That is, vision does not perceive the difference between, for example, Full HD and 4K TVs with the same screen size. Perhaps the owners of a falcon or an eagle will be able to please their pets with a high-quality image, but they themselves will not see the difference.

But these arguments have counterarguments. For example, companies are actively improving 8K AI upscaling technology.

At a minimum, 8K TVs with AI upscaling will provide better quality playback of lower resolution content.

In addition, the results of experiments open up some new aspects. In particular, South Korean scientists have studied the effect of vernier acuity (hyperacuity) to image formation by our brain. Their results allowed scientists to hypothesize that the brain is less tired when perceiving content with high resolution. At the same time, it watching provides richer emotions due to high realism of the perceived content.

Resolution

Of course, any TV provides maximum playback quality for corresponding native content in pixel-to-pixel mode. Unfortunately, the real situation on the market is often different from the ideal. For example, the capabilities of modern TVs often significantly exceed the resolution of the content offered. But sometimes devices, on the contrary, do not support content resolution.

Of course, companies solve this problem. Modern algorithms use several basic technologies. The simplest of them increases the image resolution, replacing each pixel with four copies of it. Bilinear interpolation is more efficient. These algorithms compare the missing pixel with the two nearest ones, forming a linear gradient to sharpen the image.

Even more efficient bicubic interpolation uses 16 closest points in all directions, increasing color accuracy. Unfortunately, it often provokes a halo effect.

Main methods

A list of the most popular modern methods includes supersampling, checkerboard rendering and upscaling.

Supersampling is a spatial anti-aliasing method for removing the aliasing in the rendered images. In fact, it first renders the image in 4K, and then displays it in 1080p resolution. As a result, the detail of the final image increases. Of course, rendering requires sufficient processing power. The Insects screenshot from Microsoft on Full HD TV demonstrates this effect.

Checkerboard Rendering reduces computing power requirements. In particular, it removes every second pixel and forms an image from the rest. The diagrams show the differences between Standard Rendering and Checkerboard Rendering.

Removing pixels slightly reduces the image resolution, but significantly reduces the amount of computation.

In fact, upscaling only resizes while maintaining its proportions. It can be considered the simplest method of converting image resolution. In this case, the TV processor forms additional pixels, providing the required resolution. Accordingly, the final image is a set of new averaged pixels.

Unfortunately, the final image comparison demonstrates a significant decrease in quality compared to native content.

Upscaling

A few years ago, Samsung was the first to use the marketing term UHD Upscaling. Strictly speaking, almost any smartphone or computer has long supported this feature. Of course, its quality directly depends on the content. For example, the quality of SD – UHD Upscaling is significantly lower compared to Full HD – UHD Upscaling due to the insufficient number of pixels in the original image. The picture illustrates the difference in quality between SD / HD / Full HD – 4K Upscaling.

But its visualization is almost disappearing for Full HD.

The first 4K TVs did not have an AI processor and simply scaled the image to the entire screen area using traditional interpolation and increasing the pixels size.

Unfortunately, bilinear interpolation provides a fairly low quality due to loss of information, reduced detail, blurring outlines of objects, noise, etc. Moreover, mutual influence complicates their elimination. For example, sharpening increases noise, and conversely, reducing noise reduces detail.
But companies have improved technology by developing AI Upscaling. Using a powerful high-performance processor radically improved upscaling quality.

Today, almost all leaders use 8K AI Upscaling technology under different names. But, of course, they all use the same principles.

Samsung is one of the leaders in the development of this technology. The company uses the powerful 8K Quantum processor, optimized for neural networks and AI processing. Samsung’s MLSR (Machine Learning Super Resolution) technology uses machine learning, object recognition algorithms, and software filters.

Samsung AI Upscaling

Simplified, this technology uses a constantly updated database of the images pairs in high and low resolution.

It’s formed as follows. The system selects an 8K image and lowers its resolution, adding noise and reducing the clarity of outlines. In addition, it performs several compression/decompression cycles with different quality levels to add characteristic artifacts. Generated images pairs of different quality are divided into classes, processed by algorithms based on neural networks and stored in a database.

Subsequently, the processor analyzes the low-resolution image, finds the corresponding algorithm in the database with filters, and replaces it with an 8K pattern.

In fact, the 8K AI Upscaling technology generates an image by analogy with the creation of Frankenstein from 8K patterns in the database.

During processing, AI analyzes image fragments up to 60 times per second, correcting saturation, detail and white balance. CPU uses code lines of patterns base for localization of processing fragments. Such an algorithm increases the recognition accuracy of displayed objects. As a result, the processor almost accurately distinguishes, for example, the sky from its reflection in the windows.

Moreover, the technology works quite well even with streaming video. But cloud computing for such content is not possible.

Previously, AI machine learning used a formula bank, which is downloaded to the TV and updated periodically. But new Samsung Q950TS supports innovative Deep Learning using a neural network to expand machine learning. In fact, it creates its own formulas and algorithms, using processor-integrated basic training coefficients. Of course, Deep Learning has become a quality leap in AI processing. It is performed on a per-pixel basis with the use of noise reduction, restoration of contours, texture creation and restoration of small details. Deep Learning is primarily intended for small details and complex images.

Sony and LG 8K AI Upscaling

Sony’s 8K AI Upscaling use dual database processing.

The first database recognizes compression noise, compares it with the patterns and eliminates artifacts. The second database increases resolution based on patterns.

All new LG TVs use AI 8K Upscaling with new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. In addition, it supports AI Picture Pro, AI Brightness and AI Sound Pro technologies.

Conclusions

Probably, experts will continue to argue about the reasonable limits of the screen resolution. But in any case, 8K AI upscaling provides improved quality for lower resolution content. In addition, technology has an added bonus. In particular, Deep Learning of neural networks provides continuous quality improvement due to the expansion of patterns base. Theoretically, the quality of 8K AI Upscaling will constantly improve, creating a paradoxical situation when the new TV works worse than the old model.

But of course, the price remains one of the main criteria for the consumer when choosing. Today, this difference reaches several thousand dollars. But many consider it inadequately overpriced compared to the quality gain that provides the modern 8K AI Upscaling. Most likely, companies will continue to improve it performance, while lowering prices for 8K models.

This video shows the capabilities of the Samsung 8K AI Upscaling Technology.