Video conferencing, podcasts, and webinars became popular during the pandemic of 2020 and 2021, when remote work became part of the new normal. With the pandemic now in the rearview mirror, video communication technology has shown no signs of slowing down.
What made me laugh was that, despite the prevalence of video communication, too little attention is paid to how unpleasant we often look on camera using inadequate, low-resolution webcams. Bad lighting, mainly during video calls from home, is definitely a big problem. The sub-HD resolution webcams built into most laptops, even high-end ones, don’t help.
Without the professional resources available in a professional TV studio, politicians, celebrities and industry experts often look terrible when interviewed remotely from home.
Regular video conferencing calls from home are particularly vulnerable to the “hobby hour” look and feel, especially during a formal presentation where a wandering eye (for example, not looking directly at the webcam) can distract the viewer.
The location of the webcam causes this undesirable effect, as the camera is usually built into the upper part of the laptop panel or on a separate stand, which is difficult to place in front of the desktop computer display.
Since conventional video conferencing using a desktop or laptop computer does not have a proper teleprompter function, which is complex, cumbersome and expensive, it is almost impossible to read the speaker’s notes without experiencing the annoying phenomenon of the horrible angle of the webcam looking up and down at your the nose
Are there any quick fixes to the eye problem?
There are several ways to mitigate this problem in a typical home computer or laptop. However, these approaches are purely fancy and do not eliminate the problem.
Several companies offer small external webcams, often without a built-in microphone, to reduce the size of the device and allow it to be placed in the center of the screen, in front of any text material or the viewer window of the video program you are using.
These cameras use a thin wire attached to the top of the display. This way, you’re looking directly into the webcam and can see most, if not all, of the presentation or text you’re presenting.
However, another method is to use a clear piece of acrylic plastic that allows you to mount almost any webcam and hook it to the top of the display so that the webcam hangs in front of the center point of the display.
The advantage of this approach is that it frees you up from using your preferred webcam. The downside is that the webcam’s size and acrylic plastic device often cover much of the screen, making it less useful as a teleprompter alternative.
In the future, we may see laptop and PC displays with built-in webcams behind the LCD panel that are invisible to the user. Although this is an ideal solution to the problem I described above, the disadvantage is that the cost of these special displays will be very high and most manufacturers will not offer it due to the price elasticity implications.
AI can conveniently and cost-effectively correct eye contact problems.
The idea of using artificial intelligence to soften or eliminate eye contact during video conferencing is not new. If done right, AI can eliminate the need to buy the expensive telesuffling equipment that TV studios use, or resort to some of the trickier techniques I’ve described above.
The problem with using AI to correct eye contact on the fly (live) or even in a recorded script is that it requires CPU power to do most of the heavy lifting.
Apple Silicon has had this integrated capability for several years with its iPhone chips. Few users know that Apple’s FaceTime application has an eye contact correction feature (which can be turned off) that ensures that your gaze is focused on the center of the screen, regardless of the orientation of the iPhone.
Setting up eye contact in Apple’s FaceTime app
Microsoft has also joined the artificial intelligence party to solve eye contact problems. Last year, the company announced that it would add the ability to make eye contact in Windows 11, using the power of Qualcomm Arm solutions and the advantages of a neural processing unit (NPU) to improve video and audio in meetings, including framing objects, suppressing background noise. and background blur.
Many of these features were already available on Microsoft’s Surface Pro X device, which uses an Arm chip. However, Microsoft will be rolling out the feature widely this year on more compatible models from major PC manufacturers.
Nvidia streaming with eye contact
Nvidia Broadcast program , which runs on a wide range of external Nvidia graphics cards, is a robust AI tool that improves video calling and communication on x86-based PCs. Last week, Nvidia expanded the utility in version 1.4 to support the implementation of eye contact, giving the impression that the subject in the video is directly looking at the camera.
The new Eye Contact effect adjusts the speaker’s eyes to create eye contact with the camera. This capability is achieved by leveraging the power of artificial intelligence in Nvidia GPUs for accurate gaze estimation and alignment.
The new eye contact effect in Nvidia Broadcast 1.4 moves the speaker’s eyes to simulate eye contact with the camera. | Image credit: Nvidia
The advantage of Nvidia’s approach is that the capabilities are not limited to a single video conferencing platform or application. Apple only supports the ability to correct eye contact using the FaceTime app for iPhone. However, I wouldn’t be surprised if Apple expands this capability to macOS users in conjunction with the Continuity Camera capability later this year.
In addition, Nvidia Broadcast provides a Vignette feature that is comparable to many users of the Instagram app. In this way, Nvidia Broadcast can generate a fuzzy background blur to give you an AI-simulated blurry image on your webcam, instantly improving image quality.
Changing background images during video conferencing is nothing new. However, Nvidia’s approach is likely to deliver better quality as it uses the power of its graphics cards, which are optimized for video content and gaming.
Final thoughts
The eye contact feature in the Nvidia Broadcast app is currently in beta and not ready for deployment yet. As with any beta version, it will suffer from the inevitable glitches, and we have to postpone an official evaluation of its quality until a working version is available.
In addition, Nvidia Broadcast is not just an ordinary application, but an open SDK with features that can be integrated into third-party applications. This opens up an interesting new potential for third-party applications to use Nvidia Broadcast features directly.
Even so, I am amazed at some of the backlash that has emerged over the past few years around the possibility of using AI to correct eye contact. Some tech analysts have used phrases like “creepy factor” to categorize this feature in the most unappealing way possible.
Indeed, this ability will inspire many, perhaps deserved jokes, if the consequences look unnatural and artificial. However, this creepy designation seems excessive and disingenuous. The same insinuation can be made about the use of make-up or the deployment of advanced tools that correct audio imperfections during a video call. Apps like TikTok or Instagram wouldn’t exist without filters, which in my opinion create much scarier images.
Like it or not, video conferencing has survived as one of the positive outcomes of the post-pandemic world. We should welcome, not scorn, the use of technology that facilitates more productive, compelling and impactful video calls.
