Sony unveils 3D head-mounted display for surgeons to peer inside you

Sony's 3D head-mounted displays have been asking for a place in your film-watching and gaming routines since 2011, but now their latest HMD is volunteering to act as the viewing end of medical endoscopes. Hirai and Co. have just unveiled what amounts to a retooled HMZ-T2, dubbed the HMM-3000MT, which helps surgeons peek inside a patient's body in 2D or 3D -- the latter of which is said to improve precision. By eliminating the need to keep tabs on an external screen, the electronics giant thinks doctors won't have to restrict their posture and movement.

Though the hardware carries the same 720p 0.7-inch OLED panels as its sibling, it's been tweaked for use in a standing position with beefed up support for balance and comfort. The hardware also packs a picture-in-picture feature to catch different views of operations, and even the ability to rotate and flip video feeds. While the headgear has been approved for use in Japan, it hasn't been confirmed for a launch in other territories. In other words, don't expect to see it if you go under the knife relatively soon.

Show full PR text

Sony Introduces 'head-mount image processing unit' for endoscopic image display

Images from an endoscope are output in 3D/2D to organic EL panels built in to the wearable monitor (included), enabling more flexible work styles

*Please note that although this product has been approved in Japan, its launch in other countries has not yet been confirmed.

Tokyo, Japan - July 23, 2013 - Sony Corporation ("Sony") today announced the launch of a head-mount image processing unit capable of receiving and outputting endoscope image signals, or controlling video images, which can then be displayed in 3D or 2D on an accompanying head-mounted monitor. The head-mounted monitor can also be purchased separately as an optional extra.

Laparoscopic surgery, whereby an endoscope is inserted through multiple keyhole incisions in a patient's abdomen allowing the surgeon to confirm video images displayed on a monitor in real time, is becoming increasingly widespread as a procedure that minimizes the strain on patients when compared to open surgery

Furthermore, in recent years, realistic 3D images capable of conveying visual depth-related information related in high definition and with extreme precision have been recognized as providing significant advantages in the medical field, and there is great potential demand for this technology.

This year, a number of medical device manufacturers have released 3D-compatible endoscopes on the market and these devices have gained attention for their extremely accurate three-dimensional images of the area being operated on, and thereby contribute to improving surgical precision. Consequently, there has been an increase in demand for high-precision 3D images and monitors.

Accordingly, Sony is launching a head-mount image processor, which includes a 3D head-mounted monitor, and is compatible with 3D surgical laparoscope. It incorporates Sony's advanced 3D and display-related technologies to realize a standard of 3D images that meet the demands of medical professionals, and proposes new workflows.

The new unit maximizes the technological advantages of OLED (organic light-emitting diode) panels to enable extremely detailed image representation of the target area. These characteristics include high resolution, superb reproduction of blacks, excellent video image response times, and precise color reproduction. Two panels are fitted inside the monitor: one each for the left and right eye. Independent HD images are displayed on the left and right panels respectively with no crosstalk (the phenomenon of images appearing in duplicate), in order to display the target area in high definition, with faithful color reproduction and highly-precise information relating to depth.

In conventional laparoscopic procedures, surgeons generally have to check the images on an external monitor as they perform the surgery, restricting their posture and movement. However, Sony's head-mounted display enables the surgeons to position themselves flexibly as they perform their procedures, supporting smooth workflow procedures in addition to the benefit of a 3D image display.

Furthermore, Sony's 'Picture in Picture (PinP)' capability enables two images to be displayed simultaneously. Images can also be flipped to the left, right, up, or down, for different display perspectives. For example, if a team of surgeons are working together in the same operating theater, this feature can be utilized to enable laparoscopic images from the operating surgeon to be viewed by other surgeons or assistants standing in other positions, and each can view the laparoscopic images from their respective viewing angle. Sony aims to contribute to the development of 3D laparoscopic surgery by providing functionality of this nature that meets the latest operating theater needs.

Sony is positioning the medical business as one of its mid- to long-term growth areas, and was quick to focus on the potential of 3D imaging in the medical field. Based on the 3D technologies and know-how accumulated through Sony's research and development in professional broadcasting equipment and other areas, the company has already established a proven track record in peripheral medical devices such as 3D cameras, which are already fitted to optical microscopes and medical 3D recorders. Sony is launching this new medical product with the objective of further contributing to the medical arena, and will aim to continue to provide high-grade, innovative products in the future.

Example of connection configuration
In this system, the head-mount image processing unit is connected to an laparoscope camera. Image control, such as image flipping or dual-screen display and output adjustment are performed by the processor in real time, and the images are then delivered to the head-mounted monitor. Furthermore, up to two head-mounted monitors can be connected to a single head-mount image processing unit. Image signal selection and 'Picture in Picture (PinP)' capability can be controlled independently for each head-mounted monitor.

Key Features

1. HD OLED panels deliver high-grade images
This new monitor is fitted with 0.7-inch (18.0mm diagonal) OLED panels (1280 x 720), the product of Sony's unique OLED and semiconductor silicon drive technologies. In addition to the high contrast, color reproduction qualities and rapid response time of OLED panels, the monitor displays extraordinary depth to provide rich detail and subtle information about the target area for surgery.

High contrast: The panels are self-emitting, which achieves extremely high contrast ratio that exceeds measurement parameters.
Color reproducibility: Enables extremely pure coloring and smooth gradation. Vividly displays subtle color differences in the target area, which needs to be closely observed during surgery.
Rapid response performance: OLED panels emit light the instant electric current is applied, giving incredibly fast response performance. The panels can emulate the fast movements of surgical instruments to vividly display images with minimal residual image.

2. 'Dual Panel 3D method' delivers extremely pure, crosstalk-free 3D images
Displaying 3D images using a single screen generally requires either the Frame Sequential (FS) method, in which the screens for left and right are switched rapidly, or the Line by Line (LBL) method, in which the video for left and right is displayed alternately along the scan line. The FS method tends to generate crosstalk, which occurs when the left and right video frames do not switch completely, resulting in mixed images, while under the LBL method, the number of pixels is halved.
Sony's new monitor adopts the 'Dual Panel 3D method,' which utilizes separate panels for the left and right eyes, each with its own dedicated 3D image, thus preventing any possibility of crosstalk. Sony aims to contribute to 3D laparoscopic operations by delivering non-blurry, reliable 3D image display that allow medical professionals to confirm subtle details about the target area for surgery, such as depth.

3. Ability to switch between 2D and 3D images, depending on the type of endoscope
The new product is compatible with both 2D and 3D signal output from endoscopes. The display can be switched between 2D and 3D images simply by selecting the "Input" button on the image processing unit. Therefore it is compatible with a wide range of 2D endoscopes in addition to the latest 3D models.

4. Achieves comfort and wearability to suit the operating environment
Surgeons can wear the device and continue to move their bodies freely and flexibly. The device fits securely, even if the person wearing it moves their head to the left or right, up or down.

The device has been designed to provide balance when the user wears it in a standing position, while the cushioning at the forehead and on the top of the head provides comfort even when worn for long periods of time.

The adjustable headband mechanism employs the same band technology that Sony uses in its headphones, and the rear band can be adjusted easily using a single switch at the back of the head. Furthermore, a special adjustment window has been fitted to enable persons other than the surgeon wearing the device to make adjustments, for example in the event the device shifting during a lengthy procedure.

A gap has been created at the bottom of the device to enable the wearer to view both the images inside the head-mounted monitor, and the area immediately below them, with the smallest of eye movements. This also enables the assistant to seamlessly pass any required instruments to the surgeon during the operation.

5. Built-in Picture in Picture (PinP) feature
This product is equipped with a 'Picture in Picture (PinP)' feature, which enables a second image to appear as a window while the image from the laparoscopic is kept as the main image. Select the "PinP" button on the image processing unit when two image sources are being input at the same time to display both images simultaneously.*
In addition to images from the endoscope in the operating theater, other image information (such as from ultrasonic laparoscopic）can also be displayed simultaneously.

6. Display image flip feature: left or right, or 180 degree rotation
Images output by the laparoscope can be flipped to the left or right, or rotated 180 degrees. This enables the images to be viewed from each individual's standing position, regardless of the orientation of the endoscopic camera. It is also possible to flip each of the input images horizontally to the left or right, or to vertically rotate the images 180 degrees, and the images can be maintained in this state even when using Picture in Picture (PinP) mode. This feature enables other practitioners observing or assisting the endoscopic surgery from different angles to view the images from their respective viewing position.

7. Equipped with wide range of input-output terminals for connection to various endoscopic cameras
This device is equipped with four different input-output terminals, including DVI and SDI, to ensure compatibility with image signals from various endoscopic cameras. The ability to output the incoming images without modification (the video 'through out' function) enables the surgeon to view the laparoscopic images using the head-mounted monitor, while the same image information can be displayed simultaneously on an external monitor. This application enables multiple practitioners to share the information in real time.