Altair: Automatic image analyzer to assess retinal vessel caliber Gabino Verde 1 , Luis García-Ortiz 2 , Sara Rodríguez 1 , José I. Recio-Rodríguez 2 , Juan F. De Paz 1 , Manuel A. Gómez-Marcos 2 , Miguel A. Merchán and Juan M. Corchado 1 1 Computers and Automation Department, University of Salamanca, Salamanca, Spain 2 Primary care Research unit La Alamedilla. Sacyl. IBSAL. Salamanca. Spain. {gaby, lgarciao, srg, donrecio, fcofds, magomez, merchan, corchado}@usal.es Abstract. The scope of this work is to develop a technological platform specialized in assessing retinal vessel caliber and describing the relationship of the results obtained to cardiovascular risk. Population studies conducted have found retinal vessel caliber to be related to the risk of hypertension, left ventricular hypertrophy, metabolic syndrome, stroke, and coronary artery disease. The vascular system in the human retina has a unique property: it is easily observed in its natural living state in the human retina by the use of a retinal camera. Retinal circulation is an area of active research by numerous groups, and there is general experimental agreement on the analysis of the patterns of the retinal blood vessels in the normal human retina. The development of automated tools designed to improve performance and decrease interobserver variability, therefore, appears necessary. Keywords: arteriolar–venular ratio; arterial stiffness; cardiovascular disease; AI algorithms; pattern recognition, image analysis; expert knowledge 1. Introduction and background Image processing, analysis and computer vision techniques are increasing in prominence in all fields of medical science, and are especially pertinent to modern ophthalmology, as it is heavily dependent on visually oriented signs. Automatic detection of parameters from retinal images is an important problem since are associated with the risk of hypertension, left ventricular hypertrophy, metabolic syndrome, stroke, and coronary artery disease [21][22][23][24]. The vascular system in the human retina has a unique property: it is easily observed in its natural living state in the human retina by the use of a retinal camera. The retina is the only human location where blood vessels can be directly visualized non-invasively. The identification of landmark features such as the optic disc, fovea and the retinal vessels as reference co-ordinates is a prerequisite before systems can achieve more complex tasks identifying pathological entities. Reliable techniques exist for identification of these 2 structures in retinal photographs. The most studied areas in this field can be classified into three [17]: 1. The location of the optic disc, that is important in retinal image analysis for vessel tracking, as a reference length for measuring distances in retinal images, and for identifying changes within the optic disc region due to disease. Techniques as analysis of intensity pixels with a high grey-scale value [14][6] or principal component analysis (PCA) [15] are used for the location of the disk. Others authors [13] use the Hough transform (a general technique for identifying the locations and orientations of certain types of shapes within a digital image [13] ) to locate the optic disc. A ‘‘fuzzy convergence’’ algorithm is other technique use for this goal [7]. 2. The detection of the fovea, usually chosen as the position of maximum correlation between a model template and the intensity image [15]. 3. The segmentation of the vasculature form retinal images, that is, the representation by segments or similar structures of the blood vessels and their connections. There are a lot of techniques to do this, the most significant are: (i) matched filters, which typically has a Gaussian or a Gaussian derivative profile [2] [10] [16] [11]; (ii) vessel tracking, whereby vessel centre locations are automatically sought over each cross- section of a vessel along the vessels longitudinal axis, having been given a starting and end point [20]; (iii) neural networks, which employ mathematical ‘‘weights’’ to decide the probability of input data belonging to a particular output [1]; (iv) morphological processing, that uses characteristics of the vasculature shape that are known a priori, such as it being piecewise linear and connected [7]. An understanding of the design principles of the human vascular system may have applications in the synthetic design of vascular systems in tissue and organ engineering, i.e., bioartificial organs for both liver and kidney. In the current scientific literature one can find a lot of researches devoted to these areas trying to automate the analysis of retinal images [19] [12] [3][5][7]. In this paper, after several years of studies and tests [5], we propose a novel platform image processing for the study of structural properties of vessels, arteries and veins that are observed with a red-free fundus camera in the normal human eye, and the fractal analysis of the branching trees of the vascular system. The platform, called Altair "Automatic image analyzer to assess retinal vessel caliber", employs analytical methods and AI (Artificial Intelligence) algorithms to detect retinal parameters of interest. The sequence of algorithms represents a new methodology to determine the properties of retinal veins and arteries. The platform does not require user initialization and is robust to the changes in the appearance of retinal fundus images typically encountered in clinical environments and is intended as a unified tool to join in all the methods needed to carry out the automation of all processes of measurement on the retinas. This uses the latest computer techniques both as statistical and medical. The next section introduces Altair platform. Section 3 presents the most important characteristics of the platform, showing some of the relevant techniques and results. Finally, in section 4 some conclusions are presented. 3 2. Platform Overview Altair facilitates the study of structural properties of vessels, arteries and veins that are observed with a red-free fundus camera in the normal human eye, and the fractal analysis of the branching trees of the vascular system. Figure 1 shows an example of images taken directly from the fundus. The retinal vessels appear in different color, with the optic disc and fovea. There are many patterns in nature that show ramified as the retinal vessel, with open branching structures and different lengths. These objects can be described by fractal geometry. Different analytical methods and AI algorithms are used to determine the scaling properties of real objects yielding different measures of the fractal dimension, length and area of retinal veins and arteries. Figure 1: Retinograph usually takes three images of each eye: a) Centered papilla. b) With the disc on one side. c) With the macula and disc each to one side of center The main objective is to relate the level of cardiovascular risk in patients with everything that it is possible to observe in the retinas. In this work we are interested in obtaining as much information as possible from the images obtained, in this case we have focused on:  Index Artery / Vein: represents a relationship between the thickness of arteries and veins.  Branching: branching structures: including fractal analysis of the branching trees of the vascular system. In subjects with cardiovascular diseases usually appear more branches, especially around the papilla; branching index number of times that an artery branches; branching pattern: the way in which branch arteries. Actually the manner in which branching is practically as a fingerprint of the person (each have a different shape), however many samples retinas one could observe certain normal patterns. In that case it could study the relationship of these patterns with the diseases.  Area occupied by the veins and arteries.  Distribution of the capillary: according to the blood distribution, the color distribution of the capillaries varies. Moreover, our intention is to incorporate expert knowledge taken from the measures found in the retinal circulation which specify normal values of various retinal structures in healthy subjects, and to apply this information to the study of patients suffering from a number of diseases. Based on the values for area, length, position and patterns of the branching trees of the vascular system in healthy patients, we expect to determine ranges of normalcy within the population for their subsequent application to subjects affected by various diseases. In the next section is explained the main components of the platform. The original image passes through each one of the modules (preprocessing, detection, segmentation 4 and extraction of knowledge) which use different techniques and algorithms to obtain the desired image information. This sequence of steps is a methodology that is explained in the following section, also showing examples of the results obtained. 3. Methodology and Results The methodology used to obtain the functionality of the platform may be divided into two phases. Firstly, a phase, called "digitization of the retina", in which the different parts of the eye image are identified. Here a data structure is created, allowing us to represent and process of the retina without requiring the original image. In this phase modules of preprocessing, detection and segmentation are included. Secondly, a phase of "measurements" in which we work with retinas that have been previously identified. In this phase is included the extraction of knowledge and the manual correction or expert knowledge if it is necessary. In this paper we have focused in the first phase which is in charge of creating and identifying all the elements of interest of the retina. To carry out these phases, the following steps are necessary. 3. 1. Preprocessing The preprocessing or filtering module reduces noise, improves contrast, sharpens edges or corrects blurriness. Some of these actions can be carried out at the hardware level, which is to say with the features included with the camera. For testing, retinography was performed using a Topcon TRC NW 200 nonmydriatic retinal camera (Topcon Europe B.C., Capelle a/d Ijssel, The Netherlands), obtaining nasal and temporal images centered on the disk (Figure 1). The nasal image with the centered disk is loaded into the platform software through preprocessing module. 3.2. Detection limits This module is in charge of the location of the disk and the identification of the center and edges of the retina. The goal here is to construct a data structure that identifies each part of the retina based on the matrices of colors representing the images obtained (Figure 1). In this step, image processing techniques to detect intensity based on the boundaries of the structures [7][3] have been used. The identification of the papilla is important since it serves as the starting point for the detection and identification of the different blood vessels. In this phase boundaries are identified and the retinal papilla from a RGB image of the retina. The following values are returned: Cr is the center of the retina, which identifies the vector with coordinates x, y of the center of the retina. Cp is the center of the disc, which identifies the vector with the coordinates x, y of the center of the papilla. Rr, is the radius of the retina. Rp, is the radius of the papilla. For example, a sequence of output values in this phase is the following table and figure: 5 Table 1. Sequence of output values in detection modules (pixel) Cr Cp Rr Rp 1012,44 ; 774,13 1035,98 ; 734,11 1104,87 ; 562,52 915,38 ; 736,77 900,27 ; 658,74 692,68 111,76 108,92 122,15 101,95 Figure 2: Identification result in the detection phase. To carry out the identification of the limits, and in particular to the identification of the circumferences, it became necessary to carry out a process of image segmentation. Segmentation is the process that divides an image into regions or objects whose pixels have similar attributes. Each segmented region typically has a physical significance within the image. It is one of the most important processes in an automated vision system because allows to extract the objects from the image for subsequent description and recognition. Segmentation techniques can be grouped in three main groups: techniques based on the detection of edges or borders [13], thresholding techniques [14] and techniques based on clustering of pixels [6]. After analyzing the possibilities we chose one of the techniques of the first group that provided the best results. In this case using an optimization of the Hough transform [13]. This technique is very robust against noise and the existence of gaps in the border of the object. It is used to detect different shapes in digital images. When applying the Hough transform to an image must first obtain a binary image of the pixels that form part of the limits of the object (applying edge detection). The aim of the Hough transform is found aligned points that may exist in the image to form a desired shape. For example, to identify a line points that satisfy the equation of the line: (ρ = x ⋅ cos θ + sen θ, in polar coordinate). In our case, we look for points that verify the equation of the circle: (i) in polar coordinate system: r 2 – 2sr⋅ cos (θ - α) + s2 = c2, where (s, α) is the center and c the radius; (ii) in cartesian coordinate system: (x–a)2 + (y–b)2=r2, where (a,b) is the center and r the radius. For the algorithm is not computationally heavy, does not check all radios, or all possible centers, only the candidate values. The candidates centers are those defined in a near portion of the retina and the radius are in approximately one sixth the radius of the retina. To measure the approximate diameter of the retina, the algorithm calculates the average color of the image column: diameter of the retina is the length that has non-zero value (black). load image detect edges for each candidate point (a, b) in the image for each candidate radius r calculate the points (x, y) which are at an edge and y is in the circumference of center (a, b) and radius r and introduce them into the accumulator find the pairs (center, radius) whose values are the highest in the accumulator Figure 3: Pseudocode of the identification algorithm of the papilla Having identified the papilla (Figure 3) is a necessary step because it provides a starting point for other stages of segmentation and reference point for some typical measurements. Typically the correct result is the circumference of the higher value in the 6 accumulator (over 70% of cases). In almost 100% of the cases, the correct identification is between the first 3 greatest accumulator values having. 3.3. Segmentation of the vasculature from retinal images The ultimate goal in this module is to identify each blood vessel as a series of points that define the path of the vessel. Each of these points will be assigned a certain thickness. Moreover, it must distinguish whether a particular blood vessel is a vein or an artery. AI algorithms responsible for identifying veins and arteries must perform a series of sweeps in search of "key points". Algorithms based on matched filters[2] [10] [16] [11], vessel tracking [19] and PCA [15], among others, are used for obtaining the proximity points between objects (veins, arteries, capillaries), the structures retinal structures or assemblies, branching patterns, etc. These algorithms work with transformations of the original image of the retina obtained of the previous step. Within this module it is necessary three steps: (i) identification of vessels; (ii) definition of the structure of vessel; (iii) cataloging of veins and arteries. 3. 3.1 Identification of vessels In this step the blood vessels are identified in the image by thresholding techniques. Its purpose is to remove pixels where no enters the structuring element, in this case the blood vessels. The image on the retina is blurred to keep an image similar to the background. This image is used as a threshold so that the pixels of the original image will be treated as vessels if its intensity reaches 90% of the background intensity. Figure 4: a) Original image in the green channel and applying a filter media 3 times with a 3x3 window to remove noise . b) Background image (to calculate it is applied on (30 times) a filter media with a 15x15 window). The image below represents the application of these techniques in a row. The blue line represents the values of the pixels in the image; the red line, the background values; and the green line the point where there is a vessel: Figure 5: Thresholding techniques for the identification of vessels In the figure, it is possible to observe that on the left there is a very small vessel artery from below. In the middle is a fat vein and right three tiny vessels. Furthermore the edge of the retina is marked as a vessel although obviously not is. To decide where there is a 7 vessel and where, it applies the following algorithm (Figure 6a). Where Original(x,y) is the pixel (x,y) of the original image and Background(x,y) is the pixel (x,y) of the background image. The result is showed in the Figure 6b. If (Original (x, y)