METHOD OF INVESTIGATION OF STABILITY AND INFORMATIVENESS OF BASIC AND DERIVATIVE FEATURES OF ANALYSIS OF MICROSCOPIC AND DEFECTOSCOPIC IMAGES OF CAST IRON MICROSTRUCTURE

ABSTRACT

This article presents a method for studying the stability of area, linear, and moment features of objects used for analysis and recognition. The method is based on analyzing microstructural images obtained through microscopy and flaw detection, which allows the assessment of the stability of these features under various imaging conditions and changes in image processing parameters. Experimental results demonstrate that the proposed method effectively identifies differences in the microstructure of cast iron and other materials, which is important for quality control and material classification tasks in metallurgy. Special attention is given to analyzing feature stability in the presence of noise and other distortions encountered during image acquisition.

АННОТАЦИЯ

В данной статье представлена методика исследования устойчивости площадных, линейных и моментных признаков объектов, используемых для их анализа и распознавания. Метод основан на анализе микроструктурных изображений, полученных с помощью микроскопии и дефектоскопии, что позволяет оценить стабильность данных признаков в различных условиях съёмки и при изменении параметров обработки изображений. Экспериментальные результаты показывают, что предложенная методика эффективно выявляет различия в микроструктуре чугуна и других материалов, что имеет значительное значение для задач контроля качества и классификации материалов в металлургии. Особое внимание уделено анализу устойчивости признаков при наличии шума и других искажений, возникающих при получении изображений.

Keywords: Machine vision system (VS), digital image processing, stability of object features, microstructure of cast iron.

Ключевые слова: Система технического зрения, цифровая обработка изображений, стабильность признаков объекта, микроструктура чугуна.

1. Introduction

1.1 It has been established that the majority of VSs use mainly simple geometric features that are unstable when rotating and scaling the image of objects. Shape coefficients are invariant to such transformations, but their number is often insufficient to recognize objects of complex shape, for example, particles of metal microstructure. Therefore, it is necessary to calculate additional characteristics of objects on the basis of which invariant features should be formed [1-2].

1.2 The study of the features of nondestructive and destructive quality control methods shows that the results of most of these methods are snapshots (images) of objects and their defective areas. Therefore, VS, modern methods and algorithms of digital image processing can be successfully used to automate the analysis of images obtained at the output of various quality control methods [6-8].

2. Materials and methods

2.1 A set of test objects was created, consisting of 30 images, each of which contains 20 objects. All test objects are divided into 6 following classes: convex objects of known geometric shape without holes; convex objects of known geometric shape with holes; convex objects of arbitrary shape without holes; convex objects of arbitrary shape with holes; non-convex objects of arbitrary shape without holes; non-convex objects of arbitrary shape with holes.

Algorithms for calculating basic features of objects were developed. In this case, we used separately flat and linear (contour) representation of objects in the image.

The algorithms for calculating the following basic area features of objects are described: the area (number of points) of the object S, the area of the hole of the object S_{d i}, the total area of the holes of the object S_д^сум, the total area of the object (including holes) S_o [2]:

                                                                (1)

We propose algorithms for calculating new features not previously used in VS: the area of the convex shape of the object S_вып, the difference between the areas of the convex shape and the object ΔS_вып, the total area of bays S_з^сум, the area of the minimum circumscribed rectangle S_пр and the difference between the areas of the minimum circumscribed rectangle and the object ΔS_пр [9]:

,                                                         (2)

,                                                        (3)

,                                                         (4)

Figure 1. Area features of a flat object

The minimum circumscribed rectangle is understood as a rectangle of minimum area that completely includes the object. The method of its construction is proposed, which consists in the following: the analyzed object is rotated around its center of gravity (X_ц;Y_ц) by angles from 0 to 180° with step k and for each such rotation the projections of the object on X and Y axes are determined:

,                                                           (5)

,                                                          (6)

i.e., if on the X axis the boundary coordinates of the object are equal to 1 and 10, then the length of its projection on this axis will be equal to 10.

Thus, for a planar object a set is formed

,

where the values l_ix and l_iy represent the lengths of the sides of the rectangle described around the object when it is rotated by the angle i×k°. An example of defining the angle of rotation and the circumscribed rectangle is shown in Figure 2. For clarity, the figure shows an elongated object.

Figure 2. Rectangles described around the object

The product of the projection lengths l_ix and l_iy gives the area of the circumscribed rectangle for the rotation angle i×k°. Determining the minimum such area

it is possible to obtain a rectangle of minimum area described around the object, represented as the lengths of its sides.

To improve the performance of the described algorithm, not all points of the object are processed, but only points of maximum curvature. In a flat object, a point is considered to have maximum curvature if there are more light neighbors than dark neighbors in its 3×3 neighborhood, i.e.

                                       (7)

We developed algorithms for computing linear features: the length of the object perimeter P, the length of the perimeter of each hole P_di, the length of the inner perimeter P_вн and the length of the total perimeter P_o, the length L and the width W of the object:

,                                                                       (8)

.                                                                  (9)

The lengths of the larger and smaller sides of a rectangle of minimum area described around the object are used as the length L and width W of the object.

Construction of convex shape and minimal circumscribed rectangle allows to obtain additional linear characteristics: length of perimeter of convex shape P_вып, difference of perimeter lengths of object and its convex shape ΔP_вып, perimeter of minimal circumscribed rectangle P_пр, difference of perimeters of object and circumscribed rectangle ΔP_пр:

,                                                           (10)

,                                                        (11)

.                                                       (12)

Figure 3. Some linear features of a planar object

Algorithms for calculating linear features based on the object signature, which is a set of distances from each point of the object to its center of gravity r_i, are developed. Using such distances, the features are calculated: maximum, minimum and average distances from the object center to its boundary R_max, R_min and R_ср. The same values are determined for the convex shape: ,  , and .

Algorithms for calculating the moment characteristics of the object have been developed: step moments m_pq (pq is the order of the object), normalized and central normalized moments m_pq and h_pq, moments of inertia with respect to the X and Y axes - m_x and m_y, mixed moment of inertia m_xy, principal moments of inertia M1 and M2. The same features are calculated for the convex shape of the object: , , , , , , , .

The technique of forming sets of derived features of objects invariant to RTCh (rotation, transfer and change of scale) of objects is proposed. The essence of the technique is as follows:

1. The basic features of the object listed above are calculated.

2. The basic features are divided into groups of characteristics of the same dimension: area features, linear features, moments of inertia, stepped, central and normalized central moments, the total orders of which are 2, 3, 4 and 5 - a total of 15 groups.

3. For each subset of basic features, derived characteristics are calculated in accordance with formulas (13) - (15).

                                             (13)

                       (14)

                         (15)

where R is one of the subsets of features of the same dimensionality.

For example:

, , , , , , , , , ,  , , , , ,  etc.

Algorithms of preliminary processing of halftone images of objects are developed: noise removal on the basis of median filtering; image binarization using a global threshold, the value of which is determined automatically on the basis of calculation of intergroup dispersion of background and objects; removal of low-dimensional elements and filling the voids of the binary image using morphological transformations, including conditional ones; marking of connected components by wave algorithm; selection of one-point contour of the object [3-5].

2.2. Since not all generated features have the same stability invariant to rotation, transfer and change of scale (RTCh) of an object, a method for investigating their stability has been developed. This method is schematically presented in Figure 4.

Figure 4. Scheme of the method of trait stability research

Experimental determination of the stability of the features on 600 test objects divided into 6 groups was carried out. For each object the changes and calculations of their features were repeated until all possible combinations of rotation angle j and scaling factor m were used. For this purpose, the step of rotation angle change Dj, as well as the minimum m_min and maximum m_max values of the scaling factor and its step of change Dm was set.

3.Results and discussion

The following parameters were used to investigate the stability of the traits: Dj=10º, m_min=0.8, m_max=1.2, Dm=0.1. That is, each object was changed 85 times. Thus, the characteristics of 51000 objects were calculated. Figure 5 shows the graphs of change of two characteristics depending on the rotation angle and scale of the object.

Figure 5. Graph of feature deviations depending on the rotation angle and scale of the object

The study has shown that there is quite a large number of stable features for objects with holes - about 20 features have an average deviation not exceeding 3%. In the absence of holes in the object the number of such features is significantly less; in addition, some features appear to be duplicated and have the same geometric meaning. It is established that the most stable of the features formed on the basis of area features are the following: S_о/S, S_пр/S_вып,  S_вып/S_о, S_вып/S, S_пр/S_о, S_пр/S.

The features formed on the basis of linear characteristics demonstrate good resistance to RTCh both in the presence of holes in objects and in their absence. A total of 16 features can be identified, the average deviations of which lie within the range of 0 - 3%. About 10 more features show average deviations up to 5%. It was found that among the characteristics based on linear ones the best stability was shown by: P_вып/P_пр, P_пр/L, P_вып/L, R_max/P_вып, P_пр/W, R_max/P_пр, P_вып/W, R_max/W, R_max/L, W/L.

It was found that linear features have larger values of maximum deviations than area features (up to 20% for area features and from 5 to 120% for linear features). This is due to the dependence of linear features values on the quality of the object contour: linear features can noticeably change their values even with a slight change in the contour, while for area features such errors are leveled by a large number of internal points of the object.

Moment characteristics and derived features based on them showed mainly unsatisfactory stability and this is explained by the large order of their values and sensitivity to distortions of the modified object. Only characteristics based on the main moments of inertia show acceptable stability. Relatively small deviations (up to 10%) have relations between any moment feature of the object and the same feature of its convex shell, as well as features based on the principal moments of inertia: M₂/M₁, M_2вып/M_1вып, M_1вып/M₁, M_1вып/M₂, M_2вып/M₂, M_2вып/M₁, /m₀₂, /η₂₂, etc.

The proposed methodology allows us to determine the quality of recognition for each characteristic separately, or for a system of characteristics. In this case, the informativeness of the system is understood as the proportion of correctly recognized objects:

where K_п is the number of correctly recognized objects for the selected set of features;

K_t - total number of objects involved in recognition.

The developed methodology consists in the following. Let there is a set of features C_i, where iÎ(1, …, q), from which it is necessary to form an informative system consisting of n features.

First, for each feature C_i, its informativeness I_Сi is calculated, and then the characteristic with maximum informativeness max{I_Сi} is selected. It is added to the formed system of attributes. At the next step, the remaining features are added to the existing system from one characteristic in turn. As a result, (q-1) systems of two features each are built. From them the most informative one is selected, which is passed to the next step. And so on until a system of n features is formed (Figure 6).

Using the developed methodology, experiments were conducted to determine the systems of informative features in recognizing test and real objects.

Objects with known geometric shapes were chosen as test objects: triangles, rectangles, trapezoids, circles, ellipses, parallelograms. The application of the developed methodology showed that for confident recognition of similar objects (classification reliability - 100%) it is enough to use a system of four features: R_min/L,  DS_вып/S_вып, DP/P_o, R_ср/P_o.

Figure 6. Methodology for studying the informativeness of features

Studies have been carried out on the informativeness of features in recognizing real objects of complex shape-different forms of graphite. The following types of graphite in cast irons are distinguished: lamellar, vermicular, spherical, and flake (Figure 7).

a)

b)

c)

d)

Figure 7. Types of graphite inclusions in cast iron: a)-lamellar; b)-vermicular; c)-spherical; d)-flaky

More than 400 objects like those shown in Figure 7. The objects were selected from GOST 3443-87, where they are given as standards of various graphite forms. The results of the study of real objects for the system of 10 features are shown in Table 1.

The study of the informativeness of invariant features for recognizing real objects has shown that the complexity of recognition in comparison with test objects has increased significantly. A system of 10 features gives a classification accuracy of 96%. The accuracy can be increased to 98.5% using a system of 43 features (Figure 8), but the processing time increases by more than 5 times. The recognition quality of 100% was not achieved in this experiment due to the specifics of the used objects: graphite inclusions of small sizes often have a rounded shape regardless of the type of graphite, so they are recognized by the system as spherical graphite[10-13].

Table 1.

The system of features formed when recognizing real objects

Figure 8. Recognition quality depending on the number of features used

4. Conclusion

Based on the conducted research, an experimental system for processing, analysis and recognition of metallographic and flaw detection images was developed, containing[14-17]:

• a digital video sensor forming a halftone or binary image of a microstructure or flaw detection image;
• an image pre-processing block (noise filtering, segmentation, removal of small-sized objects and filling of voids);
• an object selection block and calculation of their basic features;
• a derived features calculation block based on the basic features;
• database of the system, storing the features of reference objects, as well as a list of characteristics that must be calculated to recognize objects (the system of features used);
• recognition block, which takes as input the calculated features of objects in the image and characteristics of reference objects from the database and attributes the object to a class;
• block of output of the obtained data and their visualization;
• control unit.

References:

1. Стародубов, Д.Н. Комплекс программ обработки и анализа изображений объектов в системах технического зрения / Д.Н. Стародубов, Н.Н. Стулов // Программные продукты и системы. 2006;3:17-20.
2. Садыков, С.С. Алгоритмы определения длины и ширины дискретных площадных объектов / С.С. Садыков, Д.Н. Стародубов // Автоматизация и современные технологии. – М.: Машиностроение, 2007;10:8 – 12.
3. Садыков, С. С., Савичева, С. В., & Самандаров, И. Р. Распознавание отдельных и наложенных реальных плоских объектов по кривизне точек контуров их бинарных изображений. Gospodarka i Innowacje. 2022;22:383-398.
4. Bulanova Y.A., Sadykov S.S., Samandarov I.R., Dushatov N.T., Miratoyev Z.M. Исследования методов повышения контраста маммографических снимков. Oriental Renaissance: Innovative, Educational, Natural, and Social Sciences. 2022;2(10):304-315.
5. Bulanova Yu.A, Sadykov S.S., Samandarov I.R., Dushatov N.T., Miratoyev Z.M. Исследование методов фильтрации шума на маммографических снимках Eastern Renaissance: Innovative, educational, natural and social sciences. 2022;2(10):177-191.
6. Samandarov I.R., Manshurov Sh.T., Dushatov N.T., Miratoyev Z.M. Обработка изображений в c++ с помощью библиотеки opencv. Universum: технические науки.2023;5(110):19-28.
7. Bulanova Yu.A., Sadykov S.S., Samandarov I.R., Malikov M.N., Manshurov Sh.T. Research on the choice of algorithms for recognizing neoplasms on mammograms. Zibaldone. Estudios Italianos. 2023;10(2): 502-511.
8. Илхомжон Расулович Самандаров, Шерзод Туйчибоевич Маншуров, Зохиджон Мирвалиевич Миратоев, Жахонгир Тиркашевич Жонкобилов, Журабек Улугбекович Абдухашимов. Алгоритмы распознавания отдельных и наложенных реальных плоских объектов. Universum: технические науки. 2023;5-2 (110):11-18. May,.
9. ЯЮ Кульков, СС Садыков, ИР Самандаров, АД Орлов. Алгоритм определения стороны плоского объекта симметричной формы. Оптико-электронные приборы и устройства в системах распознавания образов и обработки изображений. 2021: 281-283.
10. Муминов, Ф. М., Душатов, Н. Т., Миратоев, З. М., & Ибодуллаева, М. Ш. (2022). Об одной краевой задаче для уравнения третьего порядка смешанно-составного типа. Oriental renaissance: Innovative, educational, natural and social sciences, 2(6), 606-612
11. Callister, W. D., & Rethwisch, D. G. (2018). Materials Science and Engineering: An Introduction. John Wiley & Sons.
12. ASM International. (1991). ASM Handbook Volume 1: Properties and Selection: Irons, Steels, and High-Performance Alloys. ASM International.
13. Russ, J. C. (2016). The Image Processing Handbook (7th ed.). CRC Press.
14. Springer, G. (2003). Image Analysis, Classification and Change Detection in Remote Sensing. CRC Press.
15. Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
16. Guyon, I., & Elisseeff, A. (2003). An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 3, 1157-1182.
17. Муминов Ф.М., Каримов С.Я., Утабов У нелокальная краевая задача для линейных уравнений смешанного типа Oriental renaissance: Innovative, educational, natural and social sciences, Volume 2 ISSUE 11  ISSN  2181-1784 SJIF 2022: 5.947 ASI Factor  = 1.7