rus
Issue #2/2023
A. V. Medvedev, A. V. Grinkevich, S. N. Knyazeva
Electro-Optical Surveillance and All-Round Camera Systems
Electro-Optical Surveillance and All-Round Camera Systems
DOI: 10.22184/1993-7296.FRos.2023.17.2.146.166
The article considers a special class of optoelectronic devices, namely the television panoramic all-round viewing devices designed for 360° instantaneous viewing. Various design versions of panoramic devices are given, special attention is paid to the all-round television panoramic device with the upper hemisphere that provides an opportunity to observe the upper hemisphere without any dead areas using 4 television channels and solving the problem of UAV detection and counteraction in the anti-aircraft area.
The article considers a special class of optoelectronic devices, namely the television panoramic all-round viewing devices designed for 360° instantaneous viewing. Various design versions of panoramic devices are given, special attention is paid to the all-round television panoramic device with the upper hemisphere that provides an opportunity to observe the upper hemisphere without any dead areas using 4 television channels and solving the problem of UAV detection and counteraction in the anti-aircraft area.
Теги: all-round view anti-aircraft area instantaneous view area panoramic device uav detection зенитная область круговой обзор область мгновенного обзора обнаружение бпла панорамный прибор
Electro-Optical Surveillance and All-Round Camera Systems
A. V. Medvedev1, A. V. Grinkevich2, S. N. Knyazeva1
Rostov Optical and Mechanical Plant PJSC, Rostov, Yaroslavl region, Russia
EVS CJSC, Moscow, Russia
The article considers a special class of optoelectronic devices, namely the television panoramic all-round viewing devices designed for 360° instantaneous viewing. Various design versions of panoramic devices are given, special attention is paid to the all-round television panoramic device with the upper hemisphere that provides an opportunity to observe the upper hemisphere without any dead areas using 4 television channels and solving the problem of UAV detection and counteraction in the anti-aircraft area.
Keywords: panoramic device, all-round view, anti-aircraft area, UAV detection, instantaneous view area
Article received: 28. 08.2022
Article accepted: 23.11. 2022
Almost all optical and optoelectronic systems of specialized transport vehicles have a common drawback, namely the limited visibility. When operating such specialized machines in the urban conditions, there is also a psychological aspect, since in the absence of immediate information about events occurring outside the operators working in a confined space with the closed doors and hatches have not the most comfortable feelings.
One of the ways to solve this issue is to install a television panoramic device, a successful example of which is given on the Ukrainian armored personnel carrier BTR‑4 “Ladia”. In addition to the all-around view, the applied device “Panorama‑2P” has the opportunity to the television camera to a height of up to 0.5 m above the turret (Fig. 1).
The television camera of the panoramic device is mounted on the weapon station turret on the plate of the elevating and traversing mechanism that is located inside the turret. The ability to have an all-around view, among other things, is used by the commander for target designation to the gunner.
The military conflicts of recent years demonstrate that the mere availability of armored panoramas does not improve the visibility conditions, since for this class of devices all-round visibility is achieved by raising and turning the head, and the instantaneous field of view is rather limited. The design of such panoramic devices is comprehensive, and the labor intensity is high. In addition, they lack the ability to make observations in the upper hemisphere that in modern conditions is necessary and mandatory for countering unmanned aerial vehicles (UAVs).
According to the application analysis results of the Panorama‑2P type devices, performance of works in this area by the design bureau of Rostov Optical and Mechanical Plant PJSC was initially recognized as inappropriate.
The enterprise purposefully searched for an acceptable construction principle of a television-type device for all-round viewing with an instantaneous 360° horizontal field of view. The works were performed in several design areas: the panoramic option with a spherical attachment, the option with ultra-wide-angle optics, and the option with television channels on a specialized photodetector.
Panoramic option with a spherical attachment
At the initial stage of the work, a television option of the device with a megapixel TV camera and a panoramic optical head was considered, where the circular image transformation into a familiar observation format was performed by the software-based methods.
The well-known designs [1] implied the use of a special panoramic catadioptric optical system forming a circular image and being a single optical component with four working surfaces and an aperture diaphragm (Fig. 2). The first working surface is a convex spherical refractive surface, the second one is a concave spherical reflecting surface, the third one is a convex spherical reflecting surface, and the fourth one is a flat refractive surface combined with the aperture diaphragm.
Since such an optical system is quite comprehensive for serial optical production, as a result of design study, a simple option of the panoramic device optical part most suitable for the available serial production was found [2]. The system consisted of a TV lens installed with the entrance pupil vertically upwards, and a spherical attachment placed above the entrance pupil and representing a convex spherical surface with a mirror coating that reflects the surrounding 360° panorama in a certain vertical sector “αВН”.
An option has been developed according to which any lens with a convex polished surface and a radius (Rmir) could be used as a spherical attachment, on which it is required to apply a mirror reflective coating (Fig. 3a).
The option was tested on a breadboard model, where the air gap between the mirror sphere and the television lens was protected against external weather impact by a circular protective glass (Fig. 3b). The panoramic device adjustment in this embodiment is a simple operation and consists in determining the necessary shift of the TV lens to adjust the panoramic television channel to a sharp image of an external environment, with due regard to the circular reflection from the mirror sphere site.
The required shift value “x” [3] that compensates for the optical beam divergence with an optical power Фmir, is determined by the following formula:
,
where Flens is the focal length of the TV lens; Фmir is the optical power of a convex mirror reflecting surface in diopters, determined by its focal length and ultimately depending on the mirror surface radius Rmir [4]:
.
Thus, the final formula for the shift calculation is as follows:
.
The vertical field of view sector of a panoramic device is determined by the beam path calculation with the optimal selection of Rmir and Flens.
After such generation of the circular panoramic image, it is necessary to make a geometric evolution, namely conversion of the circular panoramic image into a conventional rectangular image applying any of the methods used [5] to watch the panoramic image on a widescreen display. A 3‑megapixel 1/2" TV camera of the VEI‑335-USB type with a resolution of 2048 × 1536 of CMOS elements developed and manufactured by EVS LLC (Saint-Petersburg) was used as a photodetector in the breadboard model.
The light-sensitive element of the camera is an OV3620 CMOS sensor manufactured by Omni Vision. Voltage required for the matrix operation is provided by a clock generator built into the photodetector crystal. It also provides the required voltage for the amplifier. The built-in amplifier processes the signals coming from the photosensitive matrix array.
The use of an automatic gain control system (AGC) in conjunction with an automatic storage time control system (ASTC) allows to work in a wide illumination range of the surveillance objects. The TV camera outputs a USB signal according to the RAW RGB Data (24‑bit) standard. An example of a circular panoramic image obtained using the described breadboard model is shown in Fig. 4.
In addition to the geometric scanning, a horizontal panorama can be obtained by projecting a hemispherical image onto a cylindrical surface surrounding the mirror, i. e. by converting a hemispherical image from a polar coordinate system to a rectangular coordinate system by the so-called fast scanning method [6].
A feature of the panoramic device option with a spherical attachment is its single-channel design that determines its simplicity and producibility, does not require new technologies and is available for implementation at almost any production site. The panoramic device can be mounted on any type of vehicle, it is easy to install and assembly and solves most of the all-round visibility tasks.
The present-day circumstances have set new tasks for the all-round viewing systems, and the most urgent is to detect and counteract UAVs that requires the possible observation in the anti-aircraft area.
Since the central part of the circular image does not transmit images in the upper hemisphere, the works have been continued towards the development of a multi-purpose all-around viewing device capable of simultaneously surveying the ground situation and airspace without any dead areas during the observation process.
Panoramic option
with ultra-wide-angle optics
The specialists are aware of the development of UAVs for various purposes. However, there is a class of military UAVs that pose high danger, namely the individual killing drones: each human target is aimed by its own UAV carrying the required 200 g of explosive. Such a drone can be used for reconnaissance and surveillance, but its ultimate purpose is to make flights as a loitering munition.
The development of this UAV class is a serious threat to humans and any military equipment. The need to counter the UAVs has become the most important goal, the main parameter of which is detection of a flying small target. This problem can be solved by using a panoramic device with a view not only of the area adjacent to the observer, but also of the entire upper hemisphere.
To make such a panoramic viewing device that provides a circular view not only of the ground environment, but also of the upper hemisphere with a radius of (1–1.5) km, a combination of an upwardly directed ultra-wide-angle lens, a megapixel photodetector and specialized software that expands the resulting circular field of 360° × 180° into a familiar rectangular picture on a widescreen display [7] was used.
A prototype TV channel contained a VEC‑555-IP 5‑megapixel TV camera developed and manufactured by EVS LLC and a fisheye lens that was an ultra-wide-angle lens Fujinon FE185C046HA‑1 providing the required image quality for 5‑megapixel matrices. The channel is oriented vertically, can be installed in any convenient place and closed with an optically transparent cap.
An example of such a design is shown in Fig. 5. The 5‑megapixel VEC‑555-IP color television camera allows to get images with maximum resolution in the mode of 2592 × 1920 elements with a minimum noise level. When combined with a fisheye lens, a TV channel can provide a 185° view of the upper hemisphere while capturing the terrestrial part of the image as well.
In addition to changing the format and compression, the applied television camera allows to control other parameters, such as brightness, contrast, sharpness, saturation, white balance, switching on the backlight compensation (BLC) mode and operating modes: automatic gain control, automatic exposition, noise suppressor, night mode, binning, skipping, windowed mode, flicker reduction, etc.
The parameters of the Fujinon FE185C046HA‑1 ultra-wide-angle lens are fully compatible with those of the VEC‑555-IP TV camera. The overall dimensions and specifications of the Fujinon FE185C046HA‑1 lens are shown in Fig.6.
The fisheye lenses usually contain two negative concave-convex lenses that allow oblique light beams to be captured and directed into the lens, and the lens components with an aperture diaphragm [2]. The absence of moving parts and the high aperture ratio of the Fujinon FE185С046HA‑1 lens (1:1.8) guarantee the high reliability of the panoramic device in the field and help to solve the tasks of a circular view of large areas in the best possible way.
An option of the possible field of view observed through a panoramic television channel with an all-around viewing and an upper hemisphere using an ultra-wide-angle fisheye lens is shown in Fig. 7.
The Sun can be observed in the field of view of the channel that eloquently demonstrates the capabilities and dynamic range of the television channel, as well as a certain UAV-type object disguised as a bird, various types of which are being developed today by our military “partners”.
The channel resolution can be estimated along the long frame side (~2500 pixels) that has an angle of ~180°. One pixel of a TV camera corresponds to an angle of 180 / 2 500 ≈ 0.072° ≈ 4.32 arc minutes or ~1.2 thousandths in small divisions of the goniometer being equal to 1.2 m at a distance of 1 km.
This fact suggests that one photodetector pixel of a television camera can detect an UAV with the dimensions of (1.2–1.5) m at a distance of 1 km. The application range of the devices with such dimensions in military activities is quite wide. One version of the American-made UAV, stylized as a bird, can be assessed based on a sample shot down in Pakistan in 2011 (Fig. 8).
The computing system of the all-around panoramic device with the upper hemisphere, as a part of the software and algorithmic support and standard computing facilities, allow to solve the following issues: detection and target indication of the drone; recognition and identification of the drone class; auto-tracking and coordinate measurement of the detected “target”.
The only stable sign for detecting a “target” (attacking UAV) can only be the kinematic UAV motion cue (velocity vector) towards the protection facility of the anti-UAV complex. Other measurable and physically distinguishable features of small-sized camouflaged UAVs are as follows: spectral-energy brightness contrast of the “target” and the background; topological and geometric features (shape, size of an object, etc.) in the conditions of natural and jamming interference with the ultra-small sizes of micro- and mini-UAVs and limited sensitivity and resolution of a television channel. Such features can be unambiguously indistinguishable by the hardware and algorithms due to the availability of other (false) moving objects (clouds, birds, etc.).
Accordingly, the correct solution to the “target” identification problem at the stage of “detection” by its only indicator (“movement”) cannot be higher than the probability Ро ≤ 0.5, i. e. the selected object may or may not be a “target”. Thus, the real identification range of a small-sized object by the considered option of the all-around panoramic device with the upper hemisphere will be about ~ 100–200 m that, in principle, is sufficient to ensure the operation of modern countermeasure systems.
Similar to the previous option of the panoramic device considered, the TV panoramic device with ultra-wide-angle optics is a single-channel system, therefore it has the same simplicity and producibility, while solving the issues of all-around visibility with the upper hemisphere and identification ranges up to 1 km when observing the large UAVs with the dimensions of about ~ 10 m.
However, the rapid development of the miniature UAV class poses new challenges for the panoramic surveillance systems while maintaining the identification ranges of small targets. The need to increase the spatial resolution capability of the circular viewing panoramic device with the upper hemisphere, providing identification ranges required for timely response and activation of countermeasure systems has come to the forefront.
Panoramic option with TV channels on a specialized photodetector
The modern design solutions for panoramic surveillance systems are based on a combination of several television channels placed around the vehicle or around the chassis if the vehicle has a turret. Such a multichannel construction principle of a panoramic system makes it possible to simultaneously implement a circular field of view and high angular resolution over the entire field [8].
EVS LLC has developed a specialized television module for use in the special television surveillance systems. This type 742 module can be applied in the operating illumination range from 0.003 to 30,000 lux. On the basis of this module, Lytkarino Optical Glass Plant OJSC has developed a specialized multi-purpose TV camera TVKT‑95N for television system of field all-around surveillance. An option of the all-around viewing system using the TVKT‑95N cameras is practically implemented as a device for the commander of the T‑90M/MS armored fighting vehicle [7]. For the first time, the vehicle designers revised the approach to arranging the armored vehicle’s viewing system, while implementing a completely television version of the system. The parameters of television cameras are given in Table 1.
The field surveillance system provides the commander and gunner with an all-around view of the area adjacent to the armored vehicle. An antenna mast with the weather sensors is installed on the roof of the rotating fighting cab. This mast has three TV cameras that provide an almost all-around view, while transmitting images to the monitors of the commander and gunner. The fourth camera is located on the right side of the turret, since if it is placed on the wind sensor, the remote machine gun mount on the turret roof blocks the view (Fig. 9, top image).
To monitor the battlefield in the fighting cab, there is a control panel and a 10‑inch display connected to the all-around video cameras that shows the images from 4 video cameras (Fig. 9, bottom image). Additionally, to monitor the situation over the rear vehicle part, as well as for the convenience of reverse movement, the driver uses a separate rear-view television camera.
The system resolution can be estimated along the long frame side (752 pixels) with an angle of ~95º. One pixel of a TV camera corresponds to an angle of 95/752 ≈ 0.126° ≈ 7.5 arc minutes or ~2.1 thousandths in small divisions of the goniometer (that is 2.1 m at a distance of 1 km). These parameters are more than sufficient for driving a vehicle, navigating the ground and assessing threats at the near and medium distances. For the long-range systems, there is a separate device on the vehicle.
To raise the range in a number of developments, there is a tendency to increase the number of television channels. One of such novelties is the Panorama video surveillance and target designation system developed by Design Bureau “Display” OJSC (Belarus) (Fig. 10).
The Panorama set includes a module on an arm (Fig. 8, left photo), equipped with six video cameras installed at 60° increments in the horizontal plane, a computing module, and a 15‑inch LCD touchscreen monitor.
On the screen of the commander’s workplace in the Panorama mode, there are three rows and three columns. The top row contains images from three foreground cameras, the bottom row contains mirror images from the background cameras. The screen center (second row, second column) shows an image from a separate long-range video camera. The second row of first column is used to display the Screen Control panel. The second row of the third column is applied to display the ADUNOK Status panel.
The field angle of video cameras in the horizontal plane is 62°. The camera resolution that has been assessed according to the above method along the long frame side (752 pixels) having an angle of ~62º, is about 1.3 thousandths in small divisions of the goniometer or 1.3 m at a distance of 1 km. This indicator allows not only the ground navigation and threat assessment in the middle range, but also ensures the acceptable target designation accuracy. The Panorama system has been developed for the Adunok complex, but can also be used as an independent tool on the light unarmored or slightly armored vehicles and other moving and stationary facilities.
The multichannel all-around viewing systems can solve almost all problems, including the issue of possible observation in the upper hemisphere. However, this problem requires an additional number of channels that increases the labor intensity of the panoramic system production, difficulty of its installation at the facility and operation in a live situation.
An attempt to radically improve the panoramic image quality by increasing the number of channels was made by the Ministry of Defense of Taiwan that presented the external surveillance camera system designed for the CM‑32 8×8 Cloud Leopard armored personnel carrier. The all-around viewing system developed by the 209th munition factory is one of the latest achievements of Taiwanese technology demonstrated at the TADTE exhibition (Taipei Aerospace & Defense Technology Exhibition 2015) held in Taipei (Fig. 11).
The panoramic system consists of 16 high-resolution cameras that provide surveillance even in the low illumination conditions. Three cameras are installed in the front part, three cameras are installed on the sides, two cameras – in the rear and one camera in each rear-view mirror. Such construction provides a 360° view around the vehicle. Three additional cameras are installed on the driver’s hatch to monitor the situation when driving with the closed hatches. The images from 16 cameras are combined into a composite image that demonstrates the surrounding landscape. If desired, it is possible to display images from each individual camera.
The signal processing system automatically detects pixel movements in the output images using the sophisticated algorithms. Having recognized the potential for false alarms in a noisy environment, the engineers have programmed the system to ignore moving objects of a certain size, such as birds and small animals. According to some manufacturers, the design challenge of such systems is focused on the precise placement of a set of cameras, as well as on fusion of individual image streams in order to provide understandable and accurate information relating to the circumstances around the vehicle.
It should be noted that the complexity of data perception and analysis is raised with increase in the number of channels. When using 16 video cameras, such fact requires additional study and a comprehensive assessment, especially for various situations in the combat conditions (Fig. 12).
The specialists of Rostov Optical and Mechanical Plant PJSC have considered another way to increase the parameters by implementing a panoramic option with the non-standard photodetectors. The purpose of development has been a drastic (by several times) resolution enhancement during the all-around observation of the surrounding area with visibility of the entire upper hemisphere without any dead areas using only 4 television channels.
The ability to develop an all-angle panoramic system with high resolution parameters is based on the use of the GMAX2518 photodetector manufactured by Gpixel being an 18‑megapixel non-standard sensor with an aspect ratio of ~11:10. The 2.5 µm pixel provides a maximum dynamic range of 66 dB. Due to the technology combining the use of micro-lenses and light guides, the quantum efficiency (QE) of the sensor reaches 65%. The matrix uses 32 pairs of sub-LVDS channels, each of which operates at a maximum velocity of 960 MHz and supports the image acquisition at the frame rate of up to 64 Hz in 12‑bit mode and 150 Hz in 10‑bit mode at full resolution.
The main technical parameters of the GMAX2518 photodetector are given in Table 2.
Fig.13 shows a structural diagram of a television channel based on an 18‑megapixel CMOS sensor. The main video processing core is Artix 7 UltraScale+ chip by Xilinx being a field-programmable gate array that receives data from a video sensor at a frequency of 960 MHz. This array performs primary processing of the incoming signal and prepares data for the video output for subsequent intelligent image processing. Moreover, it determines the zones of interest of the video frame.
The video processing core controls the video sensor, while adjusting the matrix parameters for the maximum received signal quality.
The power supply unit generates all the necessary supply voltages for the TV channel microcircuits using an input voltage of 12–30 V.
For the all-around observation, the photodetector in each of the channels is oriented horizontally by the short side, on which the lens provides a field of view angle of 92°, i. e. four channels cover 360° horizontal view. Accordingly, the vertical angle of the field of view will be 101° that allows each channel to observe in the ascendant and in the ground part of the field of view, covering the entire upper hemisphere without any dead areas and without large distortions. In this case, the diagonal field of view is ~136°.
The developed design solution for a television panoramic all-around viewing device with an upper hemisphere is based on the placement of four 16‑megapixel channels stationary installed in a single housing at an angle of 90° to each other and with the title of each channel by 45° to the horizontal plane (Fig. 14).
It should be noted that the up-to-date optical solutions can also provide a single-channel option, similar to the panoramic option with an ultra-wide-angle lens. For example, in 2014 Theia Technologies introduced an option of the Theia SY185F lens with a field of view of up to 190°, designed for the use with single-matrix photodetectors up to 20 megapixels [9]. The lens is capable of using a pixel size of 1.2 microns and applies the aspherical optical elements made of glass with an extra low light dispersion degree (ELD-glass). The non-uniformity of optical properties typical for the ultra-wide-angle lenses is demonstrated by the new product in the fact that its optical resolution in the central zone is 400 lines/mm, and in the case of deviation by 85 ° – 200 lines/mm.
However, the 4‑channel option provides much greater resolution of the panoramic device, and, consequently, the greater detection ranges.
In the new panoramic device design, the TV panorama lens should have a field of view of ~136° that allows to increase the focal length of the lens and minimize distortion by using fairly typical circuit optical solutions when making the lens design. The relations between the parameters of such lenses can be determined not by the classical formula [3]:
, (4)
by, for example, by the following formula:
,
where: y′ is a half of the linear field of view along the diagonal;
Flens is the focal length of the lens;
β is the angular diagonal field of view of the lens.
With a linear diagonal field of view equal to 15.2 mm (see the details in Table 2), and in the case of an angular diagonal field of view equal to β = 136°, the lens focal length value can be about Flens = 8 mm.
The field angle of video cameras in the horizontal plane is 92°. The channel resolution that has also been assessed along the short frame side (4096 pixels) having an angle of ~92°, is ~ 0.37 thousandths in small divisions of the goniometer or 0.37 m at a distance of 1 km. To ensure such an angular resolution, it is sufficient to use a lens with a focal length of at least Flens = 6.5 mm.
The resolution of 4 television channels provides the highest performance of a panoramic device that allows not only the ground navigation and threat assessment at the middle and far distances, but also the use of electronic magnification methods with the software image processing, providing operating ranges of more than 1 km for the small UAVs and up to several kilometers for the ground targets. The data output can be performed by the proven solutions to the display connected to the all-around video cameras that demonstrates the view of 4 video cameras (similar to Fig. 9, bottom photo).
Moreover, the design solution of the panoramic device provides for the installation of laser detectors in the upper part of the housing (Fig. 15).
The laser detectors must be coordinated with the TV channels in direction, and an additional detector is installed for the anti-aircraft region. When irradiated from any direction by the enemy guidance devices with the quantum ranging or illumination tools, the panoramic device records the fact of object irradiation, determines the type of emitter (distance gauge or illuminator) and the radiation source coordinates, and provides information on the basis of which one of the counteraction system control modes is launched as an option.
A feature of the 4‑channel panoramic design option is the optimal combination of vertical and horizontal fields of view that makes it possible to implement a high-resolution all-around viewing with an upper hemisphere without any dead areas during the observation process. If necessary, the panoramic device can be installed on a telescopic or fixed arm, the height of which is determined by the specific vehicle type. The panoramic option is simple and technologically advanced, designed for implementation in the batch production conditions, characterized by the ease of installation on any type of vehicle, ease of use, and allows to solve issues of instant review with the data provision in a format well-known to the operator.
Thus, it is shown how various options of the panoramic devices make it possible to observe the upper hemisphere without any dead areas when using 4 television channels and solve the problem of UAV detection and countering in the anti-aircraft area.
AUTHORS
Medvedev Alexander Vladimirovich, General Designer, Rostov Optical and Mechanical Plant OJSC (ROMZ OJSC), Rostov Veliky, Yaroslavl Region, Russia.
Grinkevich Alexander Vasilievich, ZAO “EVS”, Moscow, Russia.
Knyazeva Svetlana Nikolaevna, Design Engineer, Design Bureau of OJSC “Rostov Optical and Mechanical Plant, (OJSC ”ROMZ“), Rostov the Great, Yaroslavl Region, Russia.
Conflict of interest
The authors declare no conflicts of interest.
A. V. Medvedev1, A. V. Grinkevich2, S. N. Knyazeva1
Rostov Optical and Mechanical Plant PJSC, Rostov, Yaroslavl region, Russia
EVS CJSC, Moscow, Russia
The article considers a special class of optoelectronic devices, namely the television panoramic all-round viewing devices designed for 360° instantaneous viewing. Various design versions of panoramic devices are given, special attention is paid to the all-round television panoramic device with the upper hemisphere that provides an opportunity to observe the upper hemisphere without any dead areas using 4 television channels and solving the problem of UAV detection and counteraction in the anti-aircraft area.
Keywords: panoramic device, all-round view, anti-aircraft area, UAV detection, instantaneous view area
Article received: 28. 08.2022
Article accepted: 23.11. 2022
Almost all optical and optoelectronic systems of specialized transport vehicles have a common drawback, namely the limited visibility. When operating such specialized machines in the urban conditions, there is also a psychological aspect, since in the absence of immediate information about events occurring outside the operators working in a confined space with the closed doors and hatches have not the most comfortable feelings.
One of the ways to solve this issue is to install a television panoramic device, a successful example of which is given on the Ukrainian armored personnel carrier BTR‑4 “Ladia”. In addition to the all-around view, the applied device “Panorama‑2P” has the opportunity to the television camera to a height of up to 0.5 m above the turret (Fig. 1).
The television camera of the panoramic device is mounted on the weapon station turret on the plate of the elevating and traversing mechanism that is located inside the turret. The ability to have an all-around view, among other things, is used by the commander for target designation to the gunner.
The military conflicts of recent years demonstrate that the mere availability of armored panoramas does not improve the visibility conditions, since for this class of devices all-round visibility is achieved by raising and turning the head, and the instantaneous field of view is rather limited. The design of such panoramic devices is comprehensive, and the labor intensity is high. In addition, they lack the ability to make observations in the upper hemisphere that in modern conditions is necessary and mandatory for countering unmanned aerial vehicles (UAVs).
According to the application analysis results of the Panorama‑2P type devices, performance of works in this area by the design bureau of Rostov Optical and Mechanical Plant PJSC was initially recognized as inappropriate.
The enterprise purposefully searched for an acceptable construction principle of a television-type device for all-round viewing with an instantaneous 360° horizontal field of view. The works were performed in several design areas: the panoramic option with a spherical attachment, the option with ultra-wide-angle optics, and the option with television channels on a specialized photodetector.
Panoramic option with a spherical attachment
At the initial stage of the work, a television option of the device with a megapixel TV camera and a panoramic optical head was considered, where the circular image transformation into a familiar observation format was performed by the software-based methods.
The well-known designs [1] implied the use of a special panoramic catadioptric optical system forming a circular image and being a single optical component with four working surfaces and an aperture diaphragm (Fig. 2). The first working surface is a convex spherical refractive surface, the second one is a concave spherical reflecting surface, the third one is a convex spherical reflecting surface, and the fourth one is a flat refractive surface combined with the aperture diaphragm.
Since such an optical system is quite comprehensive for serial optical production, as a result of design study, a simple option of the panoramic device optical part most suitable for the available serial production was found [2]. The system consisted of a TV lens installed with the entrance pupil vertically upwards, and a spherical attachment placed above the entrance pupil and representing a convex spherical surface with a mirror coating that reflects the surrounding 360° panorama in a certain vertical sector “αВН”.
An option has been developed according to which any lens with a convex polished surface and a radius (Rmir) could be used as a spherical attachment, on which it is required to apply a mirror reflective coating (Fig. 3a).
The option was tested on a breadboard model, where the air gap between the mirror sphere and the television lens was protected against external weather impact by a circular protective glass (Fig. 3b). The panoramic device adjustment in this embodiment is a simple operation and consists in determining the necessary shift of the TV lens to adjust the panoramic television channel to a sharp image of an external environment, with due regard to the circular reflection from the mirror sphere site.
The required shift value “x” [3] that compensates for the optical beam divergence with an optical power Фmir, is determined by the following formula:
,
where Flens is the focal length of the TV lens; Фmir is the optical power of a convex mirror reflecting surface in diopters, determined by its focal length and ultimately depending on the mirror surface radius Rmir [4]:
.
Thus, the final formula for the shift calculation is as follows:
.
The vertical field of view sector of a panoramic device is determined by the beam path calculation with the optimal selection of Rmir and Flens.
After such generation of the circular panoramic image, it is necessary to make a geometric evolution, namely conversion of the circular panoramic image into a conventional rectangular image applying any of the methods used [5] to watch the panoramic image on a widescreen display. A 3‑megapixel 1/2" TV camera of the VEI‑335-USB type with a resolution of 2048 × 1536 of CMOS elements developed and manufactured by EVS LLC (Saint-Petersburg) was used as a photodetector in the breadboard model.
The light-sensitive element of the camera is an OV3620 CMOS sensor manufactured by Omni Vision. Voltage required for the matrix operation is provided by a clock generator built into the photodetector crystal. It also provides the required voltage for the amplifier. The built-in amplifier processes the signals coming from the photosensitive matrix array.
The use of an automatic gain control system (AGC) in conjunction with an automatic storage time control system (ASTC) allows to work in a wide illumination range of the surveillance objects. The TV camera outputs a USB signal according to the RAW RGB Data (24‑bit) standard. An example of a circular panoramic image obtained using the described breadboard model is shown in Fig. 4.
In addition to the geometric scanning, a horizontal panorama can be obtained by projecting a hemispherical image onto a cylindrical surface surrounding the mirror, i. e. by converting a hemispherical image from a polar coordinate system to a rectangular coordinate system by the so-called fast scanning method [6].
A feature of the panoramic device option with a spherical attachment is its single-channel design that determines its simplicity and producibility, does not require new technologies and is available for implementation at almost any production site. The panoramic device can be mounted on any type of vehicle, it is easy to install and assembly and solves most of the all-round visibility tasks.
The present-day circumstances have set new tasks for the all-round viewing systems, and the most urgent is to detect and counteract UAVs that requires the possible observation in the anti-aircraft area.
Since the central part of the circular image does not transmit images in the upper hemisphere, the works have been continued towards the development of a multi-purpose all-around viewing device capable of simultaneously surveying the ground situation and airspace without any dead areas during the observation process.
Panoramic option
with ultra-wide-angle optics
The specialists are aware of the development of UAVs for various purposes. However, there is a class of military UAVs that pose high danger, namely the individual killing drones: each human target is aimed by its own UAV carrying the required 200 g of explosive. Such a drone can be used for reconnaissance and surveillance, but its ultimate purpose is to make flights as a loitering munition.
The development of this UAV class is a serious threat to humans and any military equipment. The need to counter the UAVs has become the most important goal, the main parameter of which is detection of a flying small target. This problem can be solved by using a panoramic device with a view not only of the area adjacent to the observer, but also of the entire upper hemisphere.
To make such a panoramic viewing device that provides a circular view not only of the ground environment, but also of the upper hemisphere with a radius of (1–1.5) km, a combination of an upwardly directed ultra-wide-angle lens, a megapixel photodetector and specialized software that expands the resulting circular field of 360° × 180° into a familiar rectangular picture on a widescreen display [7] was used.
A prototype TV channel contained a VEC‑555-IP 5‑megapixel TV camera developed and manufactured by EVS LLC and a fisheye lens that was an ultra-wide-angle lens Fujinon FE185C046HA‑1 providing the required image quality for 5‑megapixel matrices. The channel is oriented vertically, can be installed in any convenient place and closed with an optically transparent cap.
An example of such a design is shown in Fig. 5. The 5‑megapixel VEC‑555-IP color television camera allows to get images with maximum resolution in the mode of 2592 × 1920 elements with a minimum noise level. When combined with a fisheye lens, a TV channel can provide a 185° view of the upper hemisphere while capturing the terrestrial part of the image as well.
In addition to changing the format and compression, the applied television camera allows to control other parameters, such as brightness, contrast, sharpness, saturation, white balance, switching on the backlight compensation (BLC) mode and operating modes: automatic gain control, automatic exposition, noise suppressor, night mode, binning, skipping, windowed mode, flicker reduction, etc.
The parameters of the Fujinon FE185C046HA‑1 ultra-wide-angle lens are fully compatible with those of the VEC‑555-IP TV camera. The overall dimensions and specifications of the Fujinon FE185C046HA‑1 lens are shown in Fig.6.
The fisheye lenses usually contain two negative concave-convex lenses that allow oblique light beams to be captured and directed into the lens, and the lens components with an aperture diaphragm [2]. The absence of moving parts and the high aperture ratio of the Fujinon FE185С046HA‑1 lens (1:1.8) guarantee the high reliability of the panoramic device in the field and help to solve the tasks of a circular view of large areas in the best possible way.
An option of the possible field of view observed through a panoramic television channel with an all-around viewing and an upper hemisphere using an ultra-wide-angle fisheye lens is shown in Fig. 7.
The Sun can be observed in the field of view of the channel that eloquently demonstrates the capabilities and dynamic range of the television channel, as well as a certain UAV-type object disguised as a bird, various types of which are being developed today by our military “partners”.
The channel resolution can be estimated along the long frame side (~2500 pixels) that has an angle of ~180°. One pixel of a TV camera corresponds to an angle of 180 / 2 500 ≈ 0.072° ≈ 4.32 arc minutes or ~1.2 thousandths in small divisions of the goniometer being equal to 1.2 m at a distance of 1 km.
This fact suggests that one photodetector pixel of a television camera can detect an UAV with the dimensions of (1.2–1.5) m at a distance of 1 km. The application range of the devices with such dimensions in military activities is quite wide. One version of the American-made UAV, stylized as a bird, can be assessed based on a sample shot down in Pakistan in 2011 (Fig. 8).
The computing system of the all-around panoramic device with the upper hemisphere, as a part of the software and algorithmic support and standard computing facilities, allow to solve the following issues: detection and target indication of the drone; recognition and identification of the drone class; auto-tracking and coordinate measurement of the detected “target”.
The only stable sign for detecting a “target” (attacking UAV) can only be the kinematic UAV motion cue (velocity vector) towards the protection facility of the anti-UAV complex. Other measurable and physically distinguishable features of small-sized camouflaged UAVs are as follows: spectral-energy brightness contrast of the “target” and the background; topological and geometric features (shape, size of an object, etc.) in the conditions of natural and jamming interference with the ultra-small sizes of micro- and mini-UAVs and limited sensitivity and resolution of a television channel. Such features can be unambiguously indistinguishable by the hardware and algorithms due to the availability of other (false) moving objects (clouds, birds, etc.).
Accordingly, the correct solution to the “target” identification problem at the stage of “detection” by its only indicator (“movement”) cannot be higher than the probability Ро ≤ 0.5, i. e. the selected object may or may not be a “target”. Thus, the real identification range of a small-sized object by the considered option of the all-around panoramic device with the upper hemisphere will be about ~ 100–200 m that, in principle, is sufficient to ensure the operation of modern countermeasure systems.
Similar to the previous option of the panoramic device considered, the TV panoramic device with ultra-wide-angle optics is a single-channel system, therefore it has the same simplicity and producibility, while solving the issues of all-around visibility with the upper hemisphere and identification ranges up to 1 km when observing the large UAVs with the dimensions of about ~ 10 m.
However, the rapid development of the miniature UAV class poses new challenges for the panoramic surveillance systems while maintaining the identification ranges of small targets. The need to increase the spatial resolution capability of the circular viewing panoramic device with the upper hemisphere, providing identification ranges required for timely response and activation of countermeasure systems has come to the forefront.
Panoramic option with TV channels on a specialized photodetector
The modern design solutions for panoramic surveillance systems are based on a combination of several television channels placed around the vehicle or around the chassis if the vehicle has a turret. Such a multichannel construction principle of a panoramic system makes it possible to simultaneously implement a circular field of view and high angular resolution over the entire field [8].
EVS LLC has developed a specialized television module for use in the special television surveillance systems. This type 742 module can be applied in the operating illumination range from 0.003 to 30,000 lux. On the basis of this module, Lytkarino Optical Glass Plant OJSC has developed a specialized multi-purpose TV camera TVKT‑95N for television system of field all-around surveillance. An option of the all-around viewing system using the TVKT‑95N cameras is practically implemented as a device for the commander of the T‑90M/MS armored fighting vehicle [7]. For the first time, the vehicle designers revised the approach to arranging the armored vehicle’s viewing system, while implementing a completely television version of the system. The parameters of television cameras are given in Table 1.
The field surveillance system provides the commander and gunner with an all-around view of the area adjacent to the armored vehicle. An antenna mast with the weather sensors is installed on the roof of the rotating fighting cab. This mast has three TV cameras that provide an almost all-around view, while transmitting images to the monitors of the commander and gunner. The fourth camera is located on the right side of the turret, since if it is placed on the wind sensor, the remote machine gun mount on the turret roof blocks the view (Fig. 9, top image).
To monitor the battlefield in the fighting cab, there is a control panel and a 10‑inch display connected to the all-around video cameras that shows the images from 4 video cameras (Fig. 9, bottom image). Additionally, to monitor the situation over the rear vehicle part, as well as for the convenience of reverse movement, the driver uses a separate rear-view television camera.
The system resolution can be estimated along the long frame side (752 pixels) with an angle of ~95º. One pixel of a TV camera corresponds to an angle of 95/752 ≈ 0.126° ≈ 7.5 arc minutes or ~2.1 thousandths in small divisions of the goniometer (that is 2.1 m at a distance of 1 km). These parameters are more than sufficient for driving a vehicle, navigating the ground and assessing threats at the near and medium distances. For the long-range systems, there is a separate device on the vehicle.
To raise the range in a number of developments, there is a tendency to increase the number of television channels. One of such novelties is the Panorama video surveillance and target designation system developed by Design Bureau “Display” OJSC (Belarus) (Fig. 10).
The Panorama set includes a module on an arm (Fig. 8, left photo), equipped with six video cameras installed at 60° increments in the horizontal plane, a computing module, and a 15‑inch LCD touchscreen monitor.
On the screen of the commander’s workplace in the Panorama mode, there are three rows and three columns. The top row contains images from three foreground cameras, the bottom row contains mirror images from the background cameras. The screen center (second row, second column) shows an image from a separate long-range video camera. The second row of first column is used to display the Screen Control panel. The second row of the third column is applied to display the ADUNOK Status panel.
The field angle of video cameras in the horizontal plane is 62°. The camera resolution that has been assessed according to the above method along the long frame side (752 pixels) having an angle of ~62º, is about 1.3 thousandths in small divisions of the goniometer or 1.3 m at a distance of 1 km. This indicator allows not only the ground navigation and threat assessment in the middle range, but also ensures the acceptable target designation accuracy. The Panorama system has been developed for the Adunok complex, but can also be used as an independent tool on the light unarmored or slightly armored vehicles and other moving and stationary facilities.
The multichannel all-around viewing systems can solve almost all problems, including the issue of possible observation in the upper hemisphere. However, this problem requires an additional number of channels that increases the labor intensity of the panoramic system production, difficulty of its installation at the facility and operation in a live situation.
An attempt to radically improve the panoramic image quality by increasing the number of channels was made by the Ministry of Defense of Taiwan that presented the external surveillance camera system designed for the CM‑32 8×8 Cloud Leopard armored personnel carrier. The all-around viewing system developed by the 209th munition factory is one of the latest achievements of Taiwanese technology demonstrated at the TADTE exhibition (Taipei Aerospace & Defense Technology Exhibition 2015) held in Taipei (Fig. 11).
The panoramic system consists of 16 high-resolution cameras that provide surveillance even in the low illumination conditions. Three cameras are installed in the front part, three cameras are installed on the sides, two cameras – in the rear and one camera in each rear-view mirror. Such construction provides a 360° view around the vehicle. Three additional cameras are installed on the driver’s hatch to monitor the situation when driving with the closed hatches. The images from 16 cameras are combined into a composite image that demonstrates the surrounding landscape. If desired, it is possible to display images from each individual camera.
The signal processing system automatically detects pixel movements in the output images using the sophisticated algorithms. Having recognized the potential for false alarms in a noisy environment, the engineers have programmed the system to ignore moving objects of a certain size, such as birds and small animals. According to some manufacturers, the design challenge of such systems is focused on the precise placement of a set of cameras, as well as on fusion of individual image streams in order to provide understandable and accurate information relating to the circumstances around the vehicle.
It should be noted that the complexity of data perception and analysis is raised with increase in the number of channels. When using 16 video cameras, such fact requires additional study and a comprehensive assessment, especially for various situations in the combat conditions (Fig. 12).
The specialists of Rostov Optical and Mechanical Plant PJSC have considered another way to increase the parameters by implementing a panoramic option with the non-standard photodetectors. The purpose of development has been a drastic (by several times) resolution enhancement during the all-around observation of the surrounding area with visibility of the entire upper hemisphere without any dead areas using only 4 television channels.
The ability to develop an all-angle panoramic system with high resolution parameters is based on the use of the GMAX2518 photodetector manufactured by Gpixel being an 18‑megapixel non-standard sensor with an aspect ratio of ~11:10. The 2.5 µm pixel provides a maximum dynamic range of 66 dB. Due to the technology combining the use of micro-lenses and light guides, the quantum efficiency (QE) of the sensor reaches 65%. The matrix uses 32 pairs of sub-LVDS channels, each of which operates at a maximum velocity of 960 MHz and supports the image acquisition at the frame rate of up to 64 Hz in 12‑bit mode and 150 Hz in 10‑bit mode at full resolution.
The main technical parameters of the GMAX2518 photodetector are given in Table 2.
Fig.13 shows a structural diagram of a television channel based on an 18‑megapixel CMOS sensor. The main video processing core is Artix 7 UltraScale+ chip by Xilinx being a field-programmable gate array that receives data from a video sensor at a frequency of 960 MHz. This array performs primary processing of the incoming signal and prepares data for the video output for subsequent intelligent image processing. Moreover, it determines the zones of interest of the video frame.
The video processing core controls the video sensor, while adjusting the matrix parameters for the maximum received signal quality.
The power supply unit generates all the necessary supply voltages for the TV channel microcircuits using an input voltage of 12–30 V.
For the all-around observation, the photodetector in each of the channels is oriented horizontally by the short side, on which the lens provides a field of view angle of 92°, i. e. four channels cover 360° horizontal view. Accordingly, the vertical angle of the field of view will be 101° that allows each channel to observe in the ascendant and in the ground part of the field of view, covering the entire upper hemisphere without any dead areas and without large distortions. In this case, the diagonal field of view is ~136°.
The developed design solution for a television panoramic all-around viewing device with an upper hemisphere is based on the placement of four 16‑megapixel channels stationary installed in a single housing at an angle of 90° to each other and with the title of each channel by 45° to the horizontal plane (Fig. 14).
It should be noted that the up-to-date optical solutions can also provide a single-channel option, similar to the panoramic option with an ultra-wide-angle lens. For example, in 2014 Theia Technologies introduced an option of the Theia SY185F lens with a field of view of up to 190°, designed for the use with single-matrix photodetectors up to 20 megapixels [9]. The lens is capable of using a pixel size of 1.2 microns and applies the aspherical optical elements made of glass with an extra low light dispersion degree (ELD-glass). The non-uniformity of optical properties typical for the ultra-wide-angle lenses is demonstrated by the new product in the fact that its optical resolution in the central zone is 400 lines/mm, and in the case of deviation by 85 ° – 200 lines/mm.
However, the 4‑channel option provides much greater resolution of the panoramic device, and, consequently, the greater detection ranges.
In the new panoramic device design, the TV panorama lens should have a field of view of ~136° that allows to increase the focal length of the lens and minimize distortion by using fairly typical circuit optical solutions when making the lens design. The relations between the parameters of such lenses can be determined not by the classical formula [3]:
, (4)
by, for example, by the following formula:
,
where: y′ is a half of the linear field of view along the diagonal;
Flens is the focal length of the lens;
β is the angular diagonal field of view of the lens.
With a linear diagonal field of view equal to 15.2 mm (see the details in Table 2), and in the case of an angular diagonal field of view equal to β = 136°, the lens focal length value can be about Flens = 8 mm.
The field angle of video cameras in the horizontal plane is 92°. The channel resolution that has also been assessed along the short frame side (4096 pixels) having an angle of ~92°, is ~ 0.37 thousandths in small divisions of the goniometer or 0.37 m at a distance of 1 km. To ensure such an angular resolution, it is sufficient to use a lens with a focal length of at least Flens = 6.5 mm.
The resolution of 4 television channels provides the highest performance of a panoramic device that allows not only the ground navigation and threat assessment at the middle and far distances, but also the use of electronic magnification methods with the software image processing, providing operating ranges of more than 1 km for the small UAVs and up to several kilometers for the ground targets. The data output can be performed by the proven solutions to the display connected to the all-around video cameras that demonstrates the view of 4 video cameras (similar to Fig. 9, bottom photo).
Moreover, the design solution of the panoramic device provides for the installation of laser detectors in the upper part of the housing (Fig. 15).
The laser detectors must be coordinated with the TV channels in direction, and an additional detector is installed for the anti-aircraft region. When irradiated from any direction by the enemy guidance devices with the quantum ranging or illumination tools, the panoramic device records the fact of object irradiation, determines the type of emitter (distance gauge or illuminator) and the radiation source coordinates, and provides information on the basis of which one of the counteraction system control modes is launched as an option.
A feature of the 4‑channel panoramic design option is the optimal combination of vertical and horizontal fields of view that makes it possible to implement a high-resolution all-around viewing with an upper hemisphere without any dead areas during the observation process. If necessary, the panoramic device can be installed on a telescopic or fixed arm, the height of which is determined by the specific vehicle type. The panoramic option is simple and technologically advanced, designed for implementation in the batch production conditions, characterized by the ease of installation on any type of vehicle, ease of use, and allows to solve issues of instant review with the data provision in a format well-known to the operator.
Thus, it is shown how various options of the panoramic devices make it possible to observe the upper hemisphere without any dead areas when using 4 television channels and solve the problem of UAV detection and countering in the anti-aircraft area.
AUTHORS
Medvedev Alexander Vladimirovich, General Designer, Rostov Optical and Mechanical Plant OJSC (ROMZ OJSC), Rostov Veliky, Yaroslavl Region, Russia.
Grinkevich Alexander Vasilievich, ZAO “EVS”, Moscow, Russia.
Knyazeva Svetlana Nikolaevna, Design Engineer, Design Bureau of OJSC “Rostov Optical and Mechanical Plant, (OJSC ”ROMZ“), Rostov the Great, Yaroslavl Region, Russia.
Conflict of interest
The authors declare no conflicts of interest.
Readers feedback
