rus
Issue #1/2022
M. A. Starasotnikau, R. V. Feodortsau
Accuracy of Determining the Elements of Interior Orientation of Optoelectronic Devices by Different Methods in Order to Form a Reference Vector Link
Accuracy of Determining the Elements of Interior Orientation of Optoelectronic Devices by Different Methods in Order to Form a Reference Vector Link
DOI: 10.22184/1993-7296.FRos.2022.16.1.60.74
The article describes methods of geometric calibration of optoelectronic devices (OED) and ways to improve its accuracy. The results are used for the geometric calibration of the Earth remote sensing OED, the optical system of which is focused to infinity, and the image is formed by “stitching” images formed by several photodetectors. To expand the technological capabilities of calibration equipment, it is proposed to use a digital micromirror device (DMD) to form a test object, which allows you to create almost any shape and size of a test object pattern for the tasks of OED calibration. The geometric calibration of the layout of the multi-matrix OED using different methods of forming a reference bunch of vectors: sequential projection of the grid intersection with the theodolite onto the focal plane of the OED layout, test objects in the form of an array of points made by lithography on a glass substrate, and a test object formed by a DMD device. The features of mathematical processing for each method of forming a reference bunch of vectors are described. The estimation of the error of the geometric calibration of the OED model, performed with various methods of setting the reference bunch of vectors, was carried out: when using theodolite – 0.48"; array of points on a glass substrate – 0.21" and formed DMD – 0.09".
The article describes methods of geometric calibration of optoelectronic devices (OED) and ways to improve its accuracy. The results are used for the geometric calibration of the Earth remote sensing OED, the optical system of which is focused to infinity, and the image is formed by “stitching” images formed by several photodetectors. To expand the technological capabilities of calibration equipment, it is proposed to use a digital micromirror device (DMD) to form a test object, which allows you to create almost any shape and size of a test object pattern for the tasks of OED calibration. The geometric calibration of the layout of the multi-matrix OED using different methods of forming a reference bunch of vectors: sequential projection of the grid intersection with the theodolite onto the focal plane of the OED layout, test objects in the form of an array of points made by lithography on a glass substrate, and a test object formed by a DMD device. The features of mathematical processing for each method of forming a reference bunch of vectors are described. The estimation of the error of the geometric calibration of the OED model, performed with various methods of setting the reference bunch of vectors, was carried out: when using theodolite – 0.48"; array of points on a glass substrate – 0.21" and formed DMD – 0.09".
Теги: calibration digital micromirror device earth remote sensing elements of interior orientation optoelectronic device photodetector дистанционное зондирование земли калибровка оптико-электронный аппарат фотоприемник цифровое микрозеркальное устройство элементы внутреннего ориентирования
Accuracy of Determining the Elements of Interior Orientation of Optoelectronic Devices by Different Methods in Order to Form a Reference Vector Link
M. A. Starasotnikau , R. V. Feodortsau
Peleng JSC, Minsk, Republic of Belarus
Belarusian National Technical University, Minsk,
Republic of Belarus
The article describes methods of geometric calibration of optoelectronic devices (OED) and ways to improve its accuracy. The results are used for the geometric calibration of the Earth remote sensing OED, the optical system of which is focused to infinity, and the image is formed by “stitching” images formed by several photodetectors. To expand the technological capabilities of calibration equipment, it is proposed to use a digital micromirror device (DMD) to form a test object, which allows you to create almost any shape and size of a test object pattern for the tasks of OED calibration. The geometric calibration of the layout of the multi-matrix OED using different methods of forming a reference bunch of vectors: sequential projection of the grid intersection with the theodolite onto the focal plane of the OED layout, test objects in the form of an array of points made by lithography on a glass substrate, and a test object formed by a DMD device. The features of mathematical processing for each method of forming a reference bunch of vectors are described. The estimation of the error of the geometric calibration of the OED model, performed with various methods of setting the reference bunch of vectors, was carried out: when using theodolite – 0.48"; array of points on a glass substrate – 0.21" and formed DMD – 0.09".
Keywords: Earth remote sensing, calibration, digital micromirror device, optoelectronic device, elements of interior orientation, photodetector
Received on: 02.12.2021
Accepted on: 28.01.2021
Introduction
Calibration consists in establishing the relationship between the readings of the measuring equipment (instrument) and the size of the measured (input) quantity. Geometric calibration involves the measurement of elements of interior orientation (EIO) of optoelectronic devices (OEDs). Geometric calibration makes it possible to measure the actual mutual arrangement of photodetectors installed in the focal plane of the OED, as well as the distortions introduced by the OED lens, primarily distortion [1]. The resulting calibration results are used in subsequent image processing. EIO of the OED are determined by the following parameters: photogrammetric (effective) focal length; the location of photodetectors in the focal plane of the OED; an array of coefficients describing lens distortion for each pixel.
EIO of the OED are used to build a model of the relationship between the pixels of the image and the corresponding point of the object, for example, the earth’s surface taken by the OED (to create topographic maps). This procedure is also necessary when “stitching” several images formed by different OED photodetectors. According to the method proposed in the source [2], the camera, from the measuring point of view, is a goniometer. Therefore, the model of the connection between the image pixels and the corresponding point on the earth’s surface is produced through an angular representation, and the angles should act as a reference. The accuracy of determining the EIO of the OED is characterized by the reliability of the angular representation during geometric calibration and, as a result, determines the accuracy of topographic maps. In the limit, this is the totality of all angular measurements, i. e. “bundle of rays” [2]. Thus, the reference bundle of vectors is several vectors with a known mutual spatial arrangement relative to each other.
The expansion of remote sensing tasks entails the production of a wide range of OED nomenclature. There are many options for the existence of EIO in them due to various designs of equipment, the format and relative position of photodetectors in the focal plane of the OED, the pixel size of the matrices and the focal length. For the purposes of geometric calibration of OED with different characteristics, an optimal test object is made in the form of a glass plate with the required pattern (mutual arrangement, size and shape of the elements of the test object), applied to its surface by lithography. However, the production of test objects with different patterns each time entails time and technological costs.
For geometric calibration, you can use a goniometric device, for example, a theodolite [3], which is installed in front of the OED being calibrated, and the spotting scope of the theodolite sequentially projects the crosshairs of the spotting scope grid, illuminated by the illuminator installed instead of the eyepiece, onto different parts of the OED photodetector. A scheme can be implemented in which the directly calibrated OED rotates, and its rotation is measured by a goniometric device. The goniometric device in this case determines the reference bunch of vectors, and the calibration error will primarily depend on the accuracy of the goniometric device.
Collimation schemes [3] are often used for geometric calibration, in which an array of points acts as a test object, and they are simultaneously projected onto the entire focal plane. The calibration error primarily depends on the accuracy of determining the distance between points.
The root-mean-square calibration error in such cases can reach 0.4–0.6" (σ) [3]. In this case, the coordinates of the array of points, recalculated into an angular measure, taking into account the focal length of the collimator, set the reference bunch of vectors. The field of view of the collimator and the size of the test object will limit the size of the calibrated field of view of the OED. In this case, it is possible to project the image of the test object with overlap onto the focal plane of the OED at different angles by turning either the collimator or the OED being calibrated, which slightly increases the calibration time, but complicates the process of processing the results. The calibration error by this method in [4] was 0.5" (σ).
The authors proposed a digital micromirror device (Digital Micromirror Device – DMD) for use as a test object. DMD has become widespread in various fields of optoelectronics, in projectors, spectroscopy, lithography, machine vision systems, etc. [5]. In the process of calibration, DMD makes it possible to form a pattern of a test object of almost any shape and size, limited only by its physical platform, and also provides the possibility of simultaneously projecting an image onto all OED photodetectors, taking into account their relative position. DMD device allows you to experimentally select the parameters of the pattern of the test object, taking into account the relative position of the photodetectors, both for single unique OED, requiring the manufacture of a separate special test object, and for the calibration of OED, using photodetectors in the form of “rulers”.
An example of using DMD for geometric calibration is presented in [6]. The procedure consists in projecting a DMD onto a point array screen and capturing this screen with a digital camera [6]. However, OED for the Earth remote sensing (ERS) have long focal lengths (up to tens of meters) and are focused to infinity. Therefore, the geometric calibration scheme [6] is not suitable for them.
The authors suggest installing the DMD in the focal plane of the collimator and using an array of DMD micromirrors with an accurate uniform spatial structure as a measuring scale. This option will provide an imitation of an infinitely distant object and will allow you to place the collimator and the OED being calibrated as close as possible to each other. DMD devices are fabricated by lithography, which makes it possible to create micromirrors as small as 5.5 µm [7]. Therefore, we believe that the error in manufacturing the spatial structure of DMD micromirrors will lead to calibration results comparable to the results of calibration using a test object on a glass substrate.
To verify this, the authors studied the influence of various methods of forming a reference bunch of vectors on the results of the geometric calibration of the OED with a long-focus lens. For this purpose, calibration methods have been developed that minimize the influence of the considered factors on its accuracy.
Object of study
As an object of study, a calibrated model of the OED was taken, the main technical characteristics of which are presented in Table 1. The OED is equipped with three photodetectors staggered in the focal plane. Such an arrangement is typical for OED remote sensing, carrying out route scanning surveys, which consists in surveying the Earth’s surface along the flight path of the spacecraft. The photodetectors are installed with an overlap used for “stitching” the images formed by several photodetectors. In such systems, the number of photodetectors can reach several tens.
Description of the compared methods of forming a reference bunch of vectors
The calibration model (Fig. 1) shows the reference vectors, which are defined in terms of the linear coordinates of the test object Xki, Yki and the focal length of the collimator lens f'K; projections of reference vectors (theoretical) Xki', Yki'; and projections of reference vectors (actual) Xi', Yi'. To simplify the construction of the ray path, the collimator lens and the OED layout lens are shown as a single optical system. From the point of view of mathematical transformations, the optical system of the collimator lens and the OED lens can be represented as a whole by recalculating the coordinates of the points of the test object Xki, Yki into the focal plane of the OED layout Xki', Yki' using a scale factor defined as the ratio of the foci of the collimator lens and the OED layout. Due to distortions of the lens of the OED layout, caused by distortion and manufacturing error, the coordinates of the points of the test object projected in some other positions. The number of vectors is set in such a way as to uniformly project them over the entire area of the photodetectors of the calibrated OED layout, and at the same time to leave a sufficient area between the images on the photodetectors for automatic calculation of the coordinates of the points centers. For all three cases of setting the reference bunch of vectors, the same number of reference vectors was used – 90 (45 in the direct position of the collimator and 45 in the inverted position).
The geometric calibration stand (Fig. 2) consists of a collimator 1, an OED layout 2 and an illuminator 3 [8]. The collimator 1 includes a lens 1a and a test object 1b located in the focal plane of the lens 1a. The test object is a glass plate with a pattern made by lithography. The glass plate is made of Zerodur (expansion class 0) to maintain dimensional stability when subjected to heat. The manufacturing technology ensures high accuracy of the mutual spatial arrangement of the elements of the test object pattern through the use of imaging units, e. g., Heidelberg DWL 66+ [9] or KBTEM-OMO EM‑5189-02 [10]. The scheme allows you to form a reference bunch of vectors with an error of no more than 0.01″. To measure the movement of the coordinate table in these installations, a high-resolution differential interferometric system is used, for example Renishaw RLD10, providing a measurement error of no more than 1 nm [11].
The pattern of the test object is an array of transparent points on an opaque background. The illuminator includes an array of LEDs and milky glass to provide uniform illumination of the test object pattern. On the back side of the board, on which the LEDs are installed, air cooling (fan) is provided for efficient heat dissipation during LED operation.
A DMD is used to form a reference bunch of vectors, then in the geometric calibration scheme, instead of the test object 1b and illuminator 3, a DLP LightCrafter 4500 is installed without lens 1 (Fig. 3). This device provides spatio-temporal control of the light flux from illuminator 3 (Fig. 4), modulated DMD 1 through software [12].
Another method of forming a reference bunch of vectors was considered – using a theodolite. A Leica TS30 total station 1 was used as a theodolite (angle measurement error – 0.5"). The device is installed in the circuit (Fig. 4) instead of the collimator 1 and illuminator 3. Instead of a standard eyepiece, a specially designed illuminator 3 is placed in the spotting scope 2 of the Leica TS30 total station 1 for an adjustable level of illumination of the spotting scope reticle.
In the presented in Fig. 2, 4 and 5 geometric calibration schemes have heating elements: illuminators, photodetectors. During the calibration process, illuminators and photodetectors heat up their environment, which leads to the appearance of a refractive index gradient. In addition, air flows from cooling systems may appear in the room. These factors lead to the fact that in the optical path of the geometric calibration scheme, the projected beams can deviate by some amount. To reduce this effect, the optical path is closed by a casing made of a porous material of the spunbond type with a density of not more than 17 g / m2. The casing is installed in such a way that the parts of the illuminator and the OED model that are heated and not included in the optical path of the calibration circuit are open. This is necessary for the free removal of the heat generated by them during calibration. The casing must be porous to ensure heat removal from the optical path, as well as to eliminate the stagnation of air masses. To reduce the effect of vibrations, the equipment included in the geometric calibration scheme, placed on a vibration isolation table.
Method for obtaining images of test objects
The collimator with a test object in the form of a glass plate or DMD, the OED to be calibrated, and the illuminator are visually aligned. A test object drawing with the required period and point size is loaded into the DMD. Illumination is selected taking into account the integrated sensitivity of the photodetector and the charge accumulation time of the photodetector is selected so that the signal in ADC units is at the level of 80–90% of the digitization maximum (to increase the signal-to-noise ratio). This is necessary for accurate calculation of the centers of point images. It also excludes the region (more than 90% of the maximum signal level), which is characterized by the greatest non-linearity of the photodetector.
For DMD, if the adjustment range of the illuminator current is insufficient, several layers of the Folarex HS 1x matt 140 matte film are installed in the lighting system 3 (Fig. 4), which scatters light like milk glass, reduces the luminous flux by ~50% and makes it uniform. Picked up operating frequency of the DMD micromirrors together with the value of the illuminator supply current in such a way that no flickering is observed due to the error of the non-synchronization of the operation of individual micromirrors or blocks of DMD micromirrors and due to the inconsistency in the frequency of operation of the DMD and the OED photodetector.
According to the image on the OED photodetectors, the pattern of the test object is located in the center of the focal plane of the collimator (Fig. 5). A matrix of 15 points is projected onto each OED photodetector (5 horizontal and 3 vertical) drawing of the test object of the collimator.
Filming is in progress the operating time mode of the OED, so that the heating of the photodetectors and, accordingly, the change in their mutual spatial position and the periodic structure of the pixels during the calibration were close to the working one.
The collimator rotates 180° around its optical axis to eliminate the systematic error associated with the collimator, test object on a glass substrate or DMD. Similar measurements are repeated for the position of the collimator rotated by 180°.
Description of the Theodolite Crosshair Imaging Technique
The spotting scope 2 (Fig. 5) of the theodolite 1 and the calibrated OED are visually aligned coaxially. The illumination of the OED photodetector is provided by 80–90% of the value of its saturation capacitance. By linear movements, the theodolite is set coaxially with the OED, so that the difference in illumination over the entire focal plane of the OED is no more than 10–20%. The theodolite spotting scope is sequentially rotated in such a way as to project and shoot the theodolite grid (Fig. 7) onto each OED photodetector in 15 different evenly spaced sections, similar to the arrangement of points in Fig. 6 for the collimator test object.
The spotting scope of the theodolite is translated through the zenith, the theodolite is rotated 180° in the horizon plane to eliminate the systematic error of the theodolite. The measurements are similarly repeated for the new position of the theodolite.
Peculiarities of Mathematical Image Processing for a Test Object
Initially, the coordinates of the projections of the reference bunch of vectors on the photodetectors are determined by calculating the energy centers of gravity of the image points of the test object, that is, by the distribution of brightness in the image. To reduce the effect of noise, the signal values in each pixel are squared [13]. To reduce the influence on the accuracy of calculating the coordinates of the energy center of gravity of the points of the background signal, a threshold was introduced, the signal values below which were taken as “0” [14]. Then the residuals of the projection of the reference bunch of vectors (theoretical) and the projection of the reference bunch of vectors (actual) on the OED photodetectors are determined. Residuals are included in the system of linear algebraic equations, the solution of which by the least squares method will refine the EIO of the OED [8]: mutual arrangement of photodetectors, photogrammetric focal length and coefficients describing lens distortion. The extent to which the corrected EIO of the OED reduce the residuals determines the accuracy of the calibration, root-mean-square deviation (RMSD) calculated from the uncompensated residuals.
A feature of the OED calibration using a test object formed by DMD, is the preliminary preparation of the test object pattern template. To do this, before calibration, an image is formed in any mathematical package with the required period and point size. The pixel values in the generated image, which will act to form the reference bunch of vectors, are set equal to “1”, the rest is “0”. It takes into account the counting period for rows 2 times smaller than for columns, as well as the fact that the DLP software LightCrafter 4500 converts an orthogonal image to a diagonal one, corresponding to the micromirror structure shown in fig. eight.
Peculiarities of Mathematical Processing of Images of Theodolite Crosshairs
Since the image of the theodolite grid is not projected onto all photodetectors simultaneously, and there are significant differences in illumination on the image that are not associated with the image of the grid crosshair, an approximate search for the crosshair is carried out on the entire image by calculating the cross-correlation [15] between the crosshair template of the theodolite grid and the image from the OED photodetector (Fig. 7). A pre-cut image area from the entire OED photodetector is used as a template. Next, the exact calculation of the coordinates of the energy center of gravity of the crosshairs and EIO is carried out in the same way as for the test object installed in the focal plane of the collimator.
Comparison of Results in Terms of Accuracy
Calibration results in the coordinate system of the focal plane of the OED layout: projections of the reference bunch of vectors and uncompensated residuals (Fig. 9). For clarity, the uncompensated residuals are magnified by a factor of 2000. The absence of a simultaneous shift of all residuals for any photodetector characterizes a reliable determination of its position, the absence of a pincushion arrangement of residuals – a reliable determination of the coefficients describing the lens distortion. It can be seen that the residuals are random in nature, for the case with a total station to a greater extent.
The error of the geometric calibration of the OED layout, depending on the method of forming the reference bunch of vectors, is presented in Table 2.
The error in determining the photogrammetric focal length is ~4 times greater when using a Leica TS 30 total station compared to a test object deposited on a glass substrate. This is due to the need to manually change the position of the spotting scope to project the reticle crosshairs over the entire focal plane of the OED layout, which leads to an increase in the calibration time, which was more than 1.5–2 hours, which is more than for a test object on a glass substrate and DLP LightCrafter 4500 4 times. During this time, the photodetectors of the OED layout heat up and, accordingly, change their mutual spatial position and the periodic structure of pixels. The error in determining the photogrammetric focal length is also greater when using DLP LightCrafter 4500, due to DMD heating, which is not present when using a test object on a glass substrate.
Geometric calibration error is greater when using a Leica TS 30 total station. This is due to the error of a single measurement of the angles of the Leica TS 30 total station in the range of 0–360° was 0.5" (σ), discreteness was 0.1". It is assumed that this value includes both systematic and random errors. The overall error was reduced by 3 times. Due to the large number of measurements (90 in total), it was possible to reduce the random error. Measurements of small angles, in the range of several degrees, transfer through the zenith of the spotting scope and rotation of the total station by 180° in the horizon plane managed to reduce the systematic. The main contribution to the geometric calibration error when using a test object on a glass substrate and DLP LightCrafter 4500 introduces an error in their manufacturing technology, for DLP LightCrafter 4500 error is 2 times less.
Conclusions
The proposed method of geometric calibration of OED with a long-focus lens and a universal method for setting a reference bunch of vectors based on DMD devices installed in the focal plane of the collimator, in which an array of DMD micromirrors with an accurate uniform spatial structure is used as a measuring scale, which makes it possible to electronically form a test object of any shape and size with an imitation of an infinitely distant object in real conditions. The proposed calibration method demonstrates results that are no worse in accuracy than the calibration method using a theodolite and a test object in the form of an array of points on a glass substrate.
About authors
Starasotnikau Mikalai Alegavich – Research Engineer (Republic of Belarus, Minsk, Peleng JSC, Kosmos NCU; e-mail: starasotnikau@peleng.by. Technical University (Republic of Belarus, Minsk e-mail: starasotnikau@gmail.com.
Feodortsau Rastsislau Valerievich – Cand. of Sciences (Engineer), Associate Professor, Instrumentation Engineering Faculty, Laser Devices and Technology Department, Belarusian National Technical University, Republic of Belarus, Minsk, e-mail: feodrw@gmail.com.
CAUTHORS' CONTRIBUTION
Starasotnikau M. A. – concept, organization and conduct of experiments, processing of results; Feodortsau R. V. – discussions, suggestions and comments. The experiments were carried out at Peleng JSC.
Conflict of interest
The authors declare that they have no conflict of interest. All authors took part in writing the article and supplemented the manuscript in part of their work.
M. A. Starasotnikau , R. V. Feodortsau
Peleng JSC, Minsk, Republic of Belarus
Belarusian National Technical University, Minsk,
Republic of Belarus
The article describes methods of geometric calibration of optoelectronic devices (OED) and ways to improve its accuracy. The results are used for the geometric calibration of the Earth remote sensing OED, the optical system of which is focused to infinity, and the image is formed by “stitching” images formed by several photodetectors. To expand the technological capabilities of calibration equipment, it is proposed to use a digital micromirror device (DMD) to form a test object, which allows you to create almost any shape and size of a test object pattern for the tasks of OED calibration. The geometric calibration of the layout of the multi-matrix OED using different methods of forming a reference bunch of vectors: sequential projection of the grid intersection with the theodolite onto the focal plane of the OED layout, test objects in the form of an array of points made by lithography on a glass substrate, and a test object formed by a DMD device. The features of mathematical processing for each method of forming a reference bunch of vectors are described. The estimation of the error of the geometric calibration of the OED model, performed with various methods of setting the reference bunch of vectors, was carried out: when using theodolite – 0.48"; array of points on a glass substrate – 0.21" and formed DMD – 0.09".
Keywords: Earth remote sensing, calibration, digital micromirror device, optoelectronic device, elements of interior orientation, photodetector
Received on: 02.12.2021
Accepted on: 28.01.2021
Introduction
Calibration consists in establishing the relationship between the readings of the measuring equipment (instrument) and the size of the measured (input) quantity. Geometric calibration involves the measurement of elements of interior orientation (EIO) of optoelectronic devices (OEDs). Geometric calibration makes it possible to measure the actual mutual arrangement of photodetectors installed in the focal plane of the OED, as well as the distortions introduced by the OED lens, primarily distortion [1]. The resulting calibration results are used in subsequent image processing. EIO of the OED are determined by the following parameters: photogrammetric (effective) focal length; the location of photodetectors in the focal plane of the OED; an array of coefficients describing lens distortion for each pixel.
EIO of the OED are used to build a model of the relationship between the pixels of the image and the corresponding point of the object, for example, the earth’s surface taken by the OED (to create topographic maps). This procedure is also necessary when “stitching” several images formed by different OED photodetectors. According to the method proposed in the source [2], the camera, from the measuring point of view, is a goniometer. Therefore, the model of the connection between the image pixels and the corresponding point on the earth’s surface is produced through an angular representation, and the angles should act as a reference. The accuracy of determining the EIO of the OED is characterized by the reliability of the angular representation during geometric calibration and, as a result, determines the accuracy of topographic maps. In the limit, this is the totality of all angular measurements, i. e. “bundle of rays” [2]. Thus, the reference bundle of vectors is several vectors with a known mutual spatial arrangement relative to each other.
The expansion of remote sensing tasks entails the production of a wide range of OED nomenclature. There are many options for the existence of EIO in them due to various designs of equipment, the format and relative position of photodetectors in the focal plane of the OED, the pixel size of the matrices and the focal length. For the purposes of geometric calibration of OED with different characteristics, an optimal test object is made in the form of a glass plate with the required pattern (mutual arrangement, size and shape of the elements of the test object), applied to its surface by lithography. However, the production of test objects with different patterns each time entails time and technological costs.
For geometric calibration, you can use a goniometric device, for example, a theodolite [3], which is installed in front of the OED being calibrated, and the spotting scope of the theodolite sequentially projects the crosshairs of the spotting scope grid, illuminated by the illuminator installed instead of the eyepiece, onto different parts of the OED photodetector. A scheme can be implemented in which the directly calibrated OED rotates, and its rotation is measured by a goniometric device. The goniometric device in this case determines the reference bunch of vectors, and the calibration error will primarily depend on the accuracy of the goniometric device.
Collimation schemes [3] are often used for geometric calibration, in which an array of points acts as a test object, and they are simultaneously projected onto the entire focal plane. The calibration error primarily depends on the accuracy of determining the distance between points.
The root-mean-square calibration error in such cases can reach 0.4–0.6" (σ) [3]. In this case, the coordinates of the array of points, recalculated into an angular measure, taking into account the focal length of the collimator, set the reference bunch of vectors. The field of view of the collimator and the size of the test object will limit the size of the calibrated field of view of the OED. In this case, it is possible to project the image of the test object with overlap onto the focal plane of the OED at different angles by turning either the collimator or the OED being calibrated, which slightly increases the calibration time, but complicates the process of processing the results. The calibration error by this method in [4] was 0.5" (σ).
The authors proposed a digital micromirror device (Digital Micromirror Device – DMD) for use as a test object. DMD has become widespread in various fields of optoelectronics, in projectors, spectroscopy, lithography, machine vision systems, etc. [5]. In the process of calibration, DMD makes it possible to form a pattern of a test object of almost any shape and size, limited only by its physical platform, and also provides the possibility of simultaneously projecting an image onto all OED photodetectors, taking into account their relative position. DMD device allows you to experimentally select the parameters of the pattern of the test object, taking into account the relative position of the photodetectors, both for single unique OED, requiring the manufacture of a separate special test object, and for the calibration of OED, using photodetectors in the form of “rulers”.
An example of using DMD for geometric calibration is presented in [6]. The procedure consists in projecting a DMD onto a point array screen and capturing this screen with a digital camera [6]. However, OED for the Earth remote sensing (ERS) have long focal lengths (up to tens of meters) and are focused to infinity. Therefore, the geometric calibration scheme [6] is not suitable for them.
The authors suggest installing the DMD in the focal plane of the collimator and using an array of DMD micromirrors with an accurate uniform spatial structure as a measuring scale. This option will provide an imitation of an infinitely distant object and will allow you to place the collimator and the OED being calibrated as close as possible to each other. DMD devices are fabricated by lithography, which makes it possible to create micromirrors as small as 5.5 µm [7]. Therefore, we believe that the error in manufacturing the spatial structure of DMD micromirrors will lead to calibration results comparable to the results of calibration using a test object on a glass substrate.
To verify this, the authors studied the influence of various methods of forming a reference bunch of vectors on the results of the geometric calibration of the OED with a long-focus lens. For this purpose, calibration methods have been developed that minimize the influence of the considered factors on its accuracy.
Object of study
As an object of study, a calibrated model of the OED was taken, the main technical characteristics of which are presented in Table 1. The OED is equipped with three photodetectors staggered in the focal plane. Such an arrangement is typical for OED remote sensing, carrying out route scanning surveys, which consists in surveying the Earth’s surface along the flight path of the spacecraft. The photodetectors are installed with an overlap used for “stitching” the images formed by several photodetectors. In such systems, the number of photodetectors can reach several tens.
Description of the compared methods of forming a reference bunch of vectors
The calibration model (Fig. 1) shows the reference vectors, which are defined in terms of the linear coordinates of the test object Xki, Yki and the focal length of the collimator lens f'K; projections of reference vectors (theoretical) Xki', Yki'; and projections of reference vectors (actual) Xi', Yi'. To simplify the construction of the ray path, the collimator lens and the OED layout lens are shown as a single optical system. From the point of view of mathematical transformations, the optical system of the collimator lens and the OED lens can be represented as a whole by recalculating the coordinates of the points of the test object Xki, Yki into the focal plane of the OED layout Xki', Yki' using a scale factor defined as the ratio of the foci of the collimator lens and the OED layout. Due to distortions of the lens of the OED layout, caused by distortion and manufacturing error, the coordinates of the points of the test object projected in some other positions. The number of vectors is set in such a way as to uniformly project them over the entire area of the photodetectors of the calibrated OED layout, and at the same time to leave a sufficient area between the images on the photodetectors for automatic calculation of the coordinates of the points centers. For all three cases of setting the reference bunch of vectors, the same number of reference vectors was used – 90 (45 in the direct position of the collimator and 45 in the inverted position).
The geometric calibration stand (Fig. 2) consists of a collimator 1, an OED layout 2 and an illuminator 3 [8]. The collimator 1 includes a lens 1a and a test object 1b located in the focal plane of the lens 1a. The test object is a glass plate with a pattern made by lithography. The glass plate is made of Zerodur (expansion class 0) to maintain dimensional stability when subjected to heat. The manufacturing technology ensures high accuracy of the mutual spatial arrangement of the elements of the test object pattern through the use of imaging units, e. g., Heidelberg DWL 66+ [9] or KBTEM-OMO EM‑5189-02 [10]. The scheme allows you to form a reference bunch of vectors with an error of no more than 0.01″. To measure the movement of the coordinate table in these installations, a high-resolution differential interferometric system is used, for example Renishaw RLD10, providing a measurement error of no more than 1 nm [11].
The pattern of the test object is an array of transparent points on an opaque background. The illuminator includes an array of LEDs and milky glass to provide uniform illumination of the test object pattern. On the back side of the board, on which the LEDs are installed, air cooling (fan) is provided for efficient heat dissipation during LED operation.
A DMD is used to form a reference bunch of vectors, then in the geometric calibration scheme, instead of the test object 1b and illuminator 3, a DLP LightCrafter 4500 is installed without lens 1 (Fig. 3). This device provides spatio-temporal control of the light flux from illuminator 3 (Fig. 4), modulated DMD 1 through software [12].
Another method of forming a reference bunch of vectors was considered – using a theodolite. A Leica TS30 total station 1 was used as a theodolite (angle measurement error – 0.5"). The device is installed in the circuit (Fig. 4) instead of the collimator 1 and illuminator 3. Instead of a standard eyepiece, a specially designed illuminator 3 is placed in the spotting scope 2 of the Leica TS30 total station 1 for an adjustable level of illumination of the spotting scope reticle.
In the presented in Fig. 2, 4 and 5 geometric calibration schemes have heating elements: illuminators, photodetectors. During the calibration process, illuminators and photodetectors heat up their environment, which leads to the appearance of a refractive index gradient. In addition, air flows from cooling systems may appear in the room. These factors lead to the fact that in the optical path of the geometric calibration scheme, the projected beams can deviate by some amount. To reduce this effect, the optical path is closed by a casing made of a porous material of the spunbond type with a density of not more than 17 g / m2. The casing is installed in such a way that the parts of the illuminator and the OED model that are heated and not included in the optical path of the calibration circuit are open. This is necessary for the free removal of the heat generated by them during calibration. The casing must be porous to ensure heat removal from the optical path, as well as to eliminate the stagnation of air masses. To reduce the effect of vibrations, the equipment included in the geometric calibration scheme, placed on a vibration isolation table.
Method for obtaining images of test objects
The collimator with a test object in the form of a glass plate or DMD, the OED to be calibrated, and the illuminator are visually aligned. A test object drawing with the required period and point size is loaded into the DMD. Illumination is selected taking into account the integrated sensitivity of the photodetector and the charge accumulation time of the photodetector is selected so that the signal in ADC units is at the level of 80–90% of the digitization maximum (to increase the signal-to-noise ratio). This is necessary for accurate calculation of the centers of point images. It also excludes the region (more than 90% of the maximum signal level), which is characterized by the greatest non-linearity of the photodetector.
For DMD, if the adjustment range of the illuminator current is insufficient, several layers of the Folarex HS 1x matt 140 matte film are installed in the lighting system 3 (Fig. 4), which scatters light like milk glass, reduces the luminous flux by ~50% and makes it uniform. Picked up operating frequency of the DMD micromirrors together with the value of the illuminator supply current in such a way that no flickering is observed due to the error of the non-synchronization of the operation of individual micromirrors or blocks of DMD micromirrors and due to the inconsistency in the frequency of operation of the DMD and the OED photodetector.
According to the image on the OED photodetectors, the pattern of the test object is located in the center of the focal plane of the collimator (Fig. 5). A matrix of 15 points is projected onto each OED photodetector (5 horizontal and 3 vertical) drawing of the test object of the collimator.
Filming is in progress the operating time mode of the OED, so that the heating of the photodetectors and, accordingly, the change in their mutual spatial position and the periodic structure of the pixels during the calibration were close to the working one.
The collimator rotates 180° around its optical axis to eliminate the systematic error associated with the collimator, test object on a glass substrate or DMD. Similar measurements are repeated for the position of the collimator rotated by 180°.
Description of the Theodolite Crosshair Imaging Technique
The spotting scope 2 (Fig. 5) of the theodolite 1 and the calibrated OED are visually aligned coaxially. The illumination of the OED photodetector is provided by 80–90% of the value of its saturation capacitance. By linear movements, the theodolite is set coaxially with the OED, so that the difference in illumination over the entire focal plane of the OED is no more than 10–20%. The theodolite spotting scope is sequentially rotated in such a way as to project and shoot the theodolite grid (Fig. 7) onto each OED photodetector in 15 different evenly spaced sections, similar to the arrangement of points in Fig. 6 for the collimator test object.
The spotting scope of the theodolite is translated through the zenith, the theodolite is rotated 180° in the horizon plane to eliminate the systematic error of the theodolite. The measurements are similarly repeated for the new position of the theodolite.
Peculiarities of Mathematical Image Processing for a Test Object
Initially, the coordinates of the projections of the reference bunch of vectors on the photodetectors are determined by calculating the energy centers of gravity of the image points of the test object, that is, by the distribution of brightness in the image. To reduce the effect of noise, the signal values in each pixel are squared [13]. To reduce the influence on the accuracy of calculating the coordinates of the energy center of gravity of the points of the background signal, a threshold was introduced, the signal values below which were taken as “0” [14]. Then the residuals of the projection of the reference bunch of vectors (theoretical) and the projection of the reference bunch of vectors (actual) on the OED photodetectors are determined. Residuals are included in the system of linear algebraic equations, the solution of which by the least squares method will refine the EIO of the OED [8]: mutual arrangement of photodetectors, photogrammetric focal length and coefficients describing lens distortion. The extent to which the corrected EIO of the OED reduce the residuals determines the accuracy of the calibration, root-mean-square deviation (RMSD) calculated from the uncompensated residuals.
A feature of the OED calibration using a test object formed by DMD, is the preliminary preparation of the test object pattern template. To do this, before calibration, an image is formed in any mathematical package with the required period and point size. The pixel values in the generated image, which will act to form the reference bunch of vectors, are set equal to “1”, the rest is “0”. It takes into account the counting period for rows 2 times smaller than for columns, as well as the fact that the DLP software LightCrafter 4500 converts an orthogonal image to a diagonal one, corresponding to the micromirror structure shown in fig. eight.
Peculiarities of Mathematical Processing of Images of Theodolite Crosshairs
Since the image of the theodolite grid is not projected onto all photodetectors simultaneously, and there are significant differences in illumination on the image that are not associated with the image of the grid crosshair, an approximate search for the crosshair is carried out on the entire image by calculating the cross-correlation [15] between the crosshair template of the theodolite grid and the image from the OED photodetector (Fig. 7). A pre-cut image area from the entire OED photodetector is used as a template. Next, the exact calculation of the coordinates of the energy center of gravity of the crosshairs and EIO is carried out in the same way as for the test object installed in the focal plane of the collimator.
Comparison of Results in Terms of Accuracy
Calibration results in the coordinate system of the focal plane of the OED layout: projections of the reference bunch of vectors and uncompensated residuals (Fig. 9). For clarity, the uncompensated residuals are magnified by a factor of 2000. The absence of a simultaneous shift of all residuals for any photodetector characterizes a reliable determination of its position, the absence of a pincushion arrangement of residuals – a reliable determination of the coefficients describing the lens distortion. It can be seen that the residuals are random in nature, for the case with a total station to a greater extent.
The error of the geometric calibration of the OED layout, depending on the method of forming the reference bunch of vectors, is presented in Table 2.
The error in determining the photogrammetric focal length is ~4 times greater when using a Leica TS 30 total station compared to a test object deposited on a glass substrate. This is due to the need to manually change the position of the spotting scope to project the reticle crosshairs over the entire focal plane of the OED layout, which leads to an increase in the calibration time, which was more than 1.5–2 hours, which is more than for a test object on a glass substrate and DLP LightCrafter 4500 4 times. During this time, the photodetectors of the OED layout heat up and, accordingly, change their mutual spatial position and the periodic structure of pixels. The error in determining the photogrammetric focal length is also greater when using DLP LightCrafter 4500, due to DMD heating, which is not present when using a test object on a glass substrate.
Geometric calibration error is greater when using a Leica TS 30 total station. This is due to the error of a single measurement of the angles of the Leica TS 30 total station in the range of 0–360° was 0.5" (σ), discreteness was 0.1". It is assumed that this value includes both systematic and random errors. The overall error was reduced by 3 times. Due to the large number of measurements (90 in total), it was possible to reduce the random error. Measurements of small angles, in the range of several degrees, transfer through the zenith of the spotting scope and rotation of the total station by 180° in the horizon plane managed to reduce the systematic. The main contribution to the geometric calibration error when using a test object on a glass substrate and DLP LightCrafter 4500 introduces an error in their manufacturing technology, for DLP LightCrafter 4500 error is 2 times less.
Conclusions
The proposed method of geometric calibration of OED with a long-focus lens and a universal method for setting a reference bunch of vectors based on DMD devices installed in the focal plane of the collimator, in which an array of DMD micromirrors with an accurate uniform spatial structure is used as a measuring scale, which makes it possible to electronically form a test object of any shape and size with an imitation of an infinitely distant object in real conditions. The proposed calibration method demonstrates results that are no worse in accuracy than the calibration method using a theodolite and a test object in the form of an array of points on a glass substrate.
About authors
Starasotnikau Mikalai Alegavich – Research Engineer (Republic of Belarus, Minsk, Peleng JSC, Kosmos NCU; e-mail: starasotnikau@peleng.by. Technical University (Republic of Belarus, Minsk e-mail: starasotnikau@gmail.com.
Feodortsau Rastsislau Valerievich – Cand. of Sciences (Engineer), Associate Professor, Instrumentation Engineering Faculty, Laser Devices and Technology Department, Belarusian National Technical University, Republic of Belarus, Minsk, e-mail: feodrw@gmail.com.
CAUTHORS' CONTRIBUTION
Starasotnikau M. A. – concept, organization and conduct of experiments, processing of results; Feodortsau R. V. – discussions, suggestions and comments. The experiments were carried out at Peleng JSC.
Conflict of interest
The authors declare that they have no conflict of interest. All authors took part in writing the article and supplemented the manuscript in part of their work.
Readers feedback
