Automating, Analyzing and Improving Pupillometry with Machine Learning Algorithms

György Kalmár; Alexandra Büki; Gabriella Kékesi; Gyöngyi Horváth; László G. Nyúl

doi:10.14232/actacyb.24.2.2019.3

György Kalmár Department of Image Processing and Computer Graphics, Faculty of Science and Informatics, University of Szeged
Alexandra Büki Department of Physiology, Faculty of Medicine, University of Szeged
Gabriella Kékesi Department of Physiology, Faculty of Medicine, University of Szeged
Gyöngyi Horváth Department of Physiology, Faculty of Medicine, University of Szeged
László G. Nyúl Department of Image Processing and Computer Graphics, Faculty of Science and Informatics, University of Szeged https://orcid.org/0000-0002-3826-543X

DOI: https://doi.org/10.14232/actacyb.24.2.2019.3

Keywords: pupillometry, classification, curve properties, U-Net

Abstract

The investigation of the pupillary light reflex (PLR) is a well-known method to provide information about the functionality of the autonomic nervous system. Pupillometry, a non-invasive technique, was applied to study the PLR alterations in a new, schizophrenia-like rat substrain, named WISKET. The pupil responses to light impulses were recorded with an infrared camera; the videos were automatically processed and features were extracted. Besides the classical statistical analysis (ANOVA), feature selection and classification were applied to reveal the significant differences in the PLR parameters between the control and WISKET animals. Based on these results, the disadvantages of this method were analyzed and the measurement process was redesigned and improved. The automated pupil detection method has also been adapted to the new videos. 2564 images were annotated manually and used to train a fully-convolutional neural network to produce pupil mask images. The method was evaluated on 329 test images and achieved 4% median relative error. With the new setup, the pupil detection became reliable and the new data acquisition offers robustness to the experiments.

Downloads

Download data is not yet available.

References

Bär, Karl-Jürgen, Boettger, Michael Karl, Schulz, Steffen, Harzendorf, Christina, Agelink, Marcus Willy, Yeragani, Vikram K, Chokka, Prtap, and Voss, Andreas. The interaction between pupil function and cardiovascular regulation in patients with acute schizophrenia. Clinical Neurophysiology, 119(10):2209–2213, 2008. DOI: 10.1016/j.clinph.2008.06.012.

Bär, Karl-Jürgen, Koschke, Mandy, Boettger, Michael Karl, Berger, Sandy, Kabisch, Alexander, Sauer, Heinrich, Voss, Andreas, and Yeragani, Vikram K. Acute psychosis leads to increased qt variability in patients suffering from schizophrenia. Schizophrenia research, 95(1):115–123, 2007. DOI: 10.1016/j.schres.2007.05.034.

Büki, Alexandra, Kalmár, György, Kekesi, Gabriella, Benedek, Gyorgy, Nyúl, László G., and Horvath, Gyongyi. Impaired pupillary control in "schizophrenia-like" WISKET rats. Autonomic Neuroscience, 213:34–42, 2018. DOI: 10.1016/j.autneu.2018.05.007.

Fuhl, Wolfgang, Santini, Thiago, Kasneci, Gjergji, and Kasneci, Enkelejda. Pupilnet: convolutional neural networks for robust pupil detection. arXiv preprint arXiv:1601.04902, 2016.

Fuhl, Wolfgang, Santini, Thiago, Kasneci, Gjergji, Rosenstiel, Wolfgang, and Kasneci, Enkelejda. Pupilnet v2. 0: Convolutional neural networks for cpu based real time robust pupil detection. arXiv preprint arXiv:1711.00112, 2017.

Hermesh, Haggai, Shiloh, Roni, Epstein, Yoram, Manaim, Hillel, Weizman, Abraham, and Munitz, Hanan. Heat intolerance in patients with chronic schizophrenia maintained with antipsychotic drugs. American Journal of Psychiatry, 157(8):1327–1329, 2000. DOI: 10.1176/appi.ajp.157.8.1327.

Horváth, Gyöngyi, Liszli, Péter, Kékesi, Gabriella, Büki, Alexandra, and Benedek, György. Characterization of exploratory activity and learning ability of healthy and 'schizophrenia-like' rats in a square corridor system (ambitus). Physiology & behavior, 169:155–164, 2017. DOI: 10.1016/j.physbeh.2016.11.039.

Horvath, Gyongyi, Petrovszki, Zita, Kekesi, Gabriella, Tuboly, Gabor, Bodosi, Balazs, Horvath, Janos, Gombköt{H{o}}, Peter, Benedek, Gyorgy, and Nagy, Attila. Electrophysiological alterations in a complex rat model of schizophrenia. Behavioural brain research, 307:65–72, 2016. DOI: 10.1016/j.bbr.2016.03.051.

Kalmár, György, Büki, Alexandra, Kékesi, Gabriella, Horváth, Gyöngyi, and Nyúl, László G. Pupil segmentation dataset. Available at: https://github.com/Gyoorey/PupilDataset.

Kalmár, György, Büki, Alexandra, Kékesi, Gabriella, Horváth, Gyöngyi, and Nyúl, László G. Image processing-based automatic pupillometry on infrared videos. Acta Cybernetica, 23(2):599–613, 2017. DOI: 10.14232/actacyb.23.2.2017.10.

Kékesi, Gabriella, Petrovszki, Z, Benedek, György, and Horváth, Gyöngyi. Sex-specific alterations in behavioral and cognitive functions in a "three hit" animal model of schizophrenia. Behavioural brain research, 284:85–93, 2015. DOI: 10.1016/j.bbr.2015.02.015.

Kingma, Diederik P. and Ba, Jimmy. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.

Lidsky, Arnold, Hakerem, Gad, and Sutton, Samuel. Pupillary reactions to single light pulses in psychiatric patients and normals. Journal of Nervous and Mental Disease, 1971. DOI: 10.1097/00005053-197110000-00007.

Lu, Wei, Tan, Jinglu, Zhang, Keqing, and Lei, Bo. Computerized mouse pupil size measurement for pupillary light reflex analysis. Computer Methods and Programs in Biomedicine, 90(3):202–209, 2008. DOI: 10.1016/j.cmpb.2008.01.002.

McLaughlin, Robert A. Randomized hough transform: better ellipse detection. In TENCON'96. Proceedings., 1996 IEEE TENCON. Digital Signal Processing Applications, volume 1, pages 409–414. IEEE, 1996. DOI: 10.1109/tencon.1996.608850.

Mohan, Kabhilan, Harper, Matthew M, Kecova, Helga, Ye, Eun-Ah, Lazic, Tatjana, Sakaguchi, Donald S, Kardon, Randy H, and Grozdanic, Sinisa D. Characterization of structure and function of the mouse retina using pattern electroretinography, pupil light reflex, and optical coherence tomography. Veterinary ophthalmology, 15(s2):94–104, 2012. DOI: 10.1111/j.1463-5224.2012.01034.x.

Morimoto, C.H, Koons, D, Amir, A, and Flickner, M. Pupil detection and tracking using multiple light sources. Image and Vision Computing, 18(4):331–335, 2000. DOI: 10.1016/s0262-8856(99)00053-0.

Paszke, Adam, Gross, Sam, Chintala, Soumith, Chanan, Gregory, Yang, Edward, DeVito, Zachary, Lin, Zeming, Desmaison, Alban, Antiga, Luca, and Lerer, Adam. Automatic differentiation in pytorch. In NIPS-W, 2017.

Petrovszki, Zita, Adam, Gabor, Tuboly, Gabor, Kekesi, Gabriella, Benedek, Gyorgy, Keri, Szabolcs, and Horvath, Gyongyi. Characterization of gene–environment interactions by behavioral profiling of selectively bred rats: The effect of NMDA receptor inhibition and social isolation. Behavioural brain research, 240:134–145, 2013. DOI: 10.1016/j.bbr.2012.11.022.

Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), volume 9351 of LNCS, pages 234–241. Springer, 2015. DOI: 10.1007/978-3-319-24574-4_28.

Yuen, HK, Illingworth, John, and Kittler, Josef. Detecting partially occluded ellipses using the Hough transform. Image and Vision Computing, 7(1):31–37, 1989. DOI: 10.1016/0262-8856(89)90017-6.

Zahn, Theodore P and Pickar, David. Autonomic activity in relation to symptom ratings and reaction time in unmedicated patients with schizophrenia. Schizophrenia research, 79(2):257–270, 2005. DOI: 10.1016/j.schres.2005.05.025.

Zhu, Danjie, Moore, Steven T, and Raphan, Theodore. Robust pupil center detection using a curvature algorithm. Computer methods and programs in biomedicine, 59(3):145–157, 1999. DOI: 10.1016/s0169-2607(98)00105-9.