A Modified Vector Recovery Index
DOI:
https://doi.org/10.25271/sjuoz.2020.8.3.755Keywords:
Vector Recovery Index, Performance Evaluation, Raster to Vector ConversionAbstract
In this paper, we show that averaging of the Vector Recovery Index (VRI) score for a test involving many images is not accurate and leads to bias. We demonstrate that the higher the difference in primitive count between the data files in an experiment, the higher the bias in calculating the VRI. Normalizing VRI scores is proposed to remove the bias and to get VRI scores that precisely reflects the performance based on images under scrutiny. Empirical performance evaluation on three datasets from the arc segmentation contests attached to International Workshops on Graphics Recognition 2005, 2009, and 2011 shows that the proposed normalization score provides accurate and realistic performance results than the unweighted average of VRI scores. The results based on the modified VRI score show that the vectorisation methods have lower performance than was usually thought.
References
Al-Khaffaf, H. S. M., Talib, A. Z. (2020). "Three-stage Junction Detection in Document Images," 2020 International Conference on Computer Science and Software Engineering (CSASE), Duhok, Iraq, 2020, pp. 142-145, doi: 10.1109/CSASE48920.2020.9142083.
Al-Khaffaf, H. S. M., Talib, A. Z., and Osman, M. A. (2013). Final report of GREC’11 arc segmentation contest: Performance evaluation on multi-resolution scanned documents. In Kwon, Y.-B. and Ogier, J.-M., editors, Graphics Recognition. New Trends and Challenges, pages 187–197, Berlin, Heidelberg. Springer Berlin Heidelberg.
Al-Khaffaf, H. S. M., Talib A. Z., Osman M. A., Wong P. L. (2010) GREC’09 Arc Segmentation Contest: Performance Evaluation on Old Documents. In: Ogier JM., Liu W., Lladós J. (eds) Graphics Recognition. Achievements, Challenges, and Evolution. GREC 2009. Lecture Notes in Computer Science, vol 6020. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13728-0_23
Alwan, S., Le Caillec, J.-M., and Le Meur, G. (2019). Detection of Primitives in Engineering Drawing Using Genetic Algorithm. In ICPRAM 2019 : 8th International Conference on Pattern Recognition .Applications and Methods, Prague, Czech Republic.
Bonnici, A., Akman, A., Calleja, G., Camilleri, K. P., Fehling, P., Ferreira, A., Hermuth, F., Israel, J. H., Landwehr, T., Liu, J., and et al. (2019). Sketch-based interaction and modeling: where do we stand? Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 33(4):370-388.
Bonnici, A. and Camilleri, K. (2013). A circle-based vectorization algorithm for drawings with shadows. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SBIM:13, page 69:77, New York, NY, USA. Association for Computing Machinery.
Bukhari, S. S., Al-Khaffaf, H. S. M., Shafait, F., Osman, M. A., Talib, A. Z., and Breuel, T. M. (2014). Final report of grec’13 arc and line segmentation contest. In Lamiroy, B. and Ogier, J.-M., editors, Graphics Recognition. Current Trends and Challenges, pages 234–239, Berlin, Heidelberg. Springer Berlin Heidelberg.
Chhabra, A. K. and Phillips, I. T. (1998). The second international graphics recognition contest - raster to vector conversion: A report. Graphics Recognition, 1389:390–410.
Chhabra, A. K. and Phillips, I. T. (2000). Performance evaluation of line drawing recognition systems. In Proceedings of the 15th International Conference on Pattern Recognition, volume 4, pages 864–869, Barcelona, Spain.
Hori, O. and Doermann, D. (1995). Quantitative measurement of the performance of raster-to-vector conversion algorithms. volume 1072 of LNCS, pages 57–68.
Inoue, N. and Yamasaki, T. (2019). "Fast Instance Segmentation for Line Drawing Vectorization," 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore, Singapore, pp. 262-265, doi: 10.1109/BigMM.2019.00-14.
Kasimov, D., Kuchuganov, A., and Kuchuganov, V. (2017). Vectorization of raster mechanical drawings on the base of ternary segmentation and soft computing. Program Comput Soft, 43:337-344.
Liu, W. (2004). Report of the arc segmentation contest. In Graphics Recognition: Lecture Notes in Computer Science: Recent Advances and Perspectives, volume 3088, pages 363–366. Springer.
Liu, W., Zhai, J., Dori, D., and Tang, L. (2001). A system for performance evaluation of arc segmentation algorithms. In Proc. Third CVPR Workshop Empirical Evaluation Methods in Computer Vision.
Liu, W. Y. and Dori, D. (1997). A protocol for performance evaluation of line detection algorithms. Machine Vision and Applications, 9(5-6):240–250.
Liu, W. Y., Zhai, J., and Dori, D. (2002). Extended summary of the arc segmentation contest. In Graphics Recognition: Algorithms and Applications, volume 2390 of Lecture Notes in Computer Science, pages 343–349.
Phillips, I. T. and Chhabra, A. K. (1999). Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9):849–870.
Popov, S., Glazunov, V., Chuvatov, M., and Purii, A. (2020). "Raster to Vector Map Convertion by Irregular Grid of Heights," 2020 26th Conference of Open Innovations Association (FRUCT), Yaroslavl, Russia, pp. 386-391, doi: 10.23919/FRUCT48808.2020.9087552.
Shafait, F., Keysers, D., and Breuel, T. M. (2006). Pixel-accurate representation and evaluation of page segmentation in document images. In Proceedings of the 18th International Conference on Pattern Recognition, volume 1, pages 872–875, Hong Kong, China.
Shafait, F., Keysers, D., and Breuel, T. M. (2008). Performance evaluation and benchmarking of six-page segmentation algorithms. Ieee Transactions on Pattern Analysis and Machine Intelligence, 30(6):941–954. 286UW.
Wang, Y., Song, X., and Wang, S. (2010). Algorithm of arcs recognition based on bar tracking. In 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, volume 6, pages 2535–2540.
Wenyin, L. (2006). The third report of the arc segmentation contest. In Lecture Notes in Computer Science, volume 3926 NCS of Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pages 358–361, Hong Kong, China. Springer Verlag, Heidelberg.
Wu, J., Chen, K., and Gao, X. (2013). Fast and accurate circle detection using gradient-direction-based segmentation. J. Opt. Soc. Am. A, 30(6):1184–1192.
Zhang, Z., Wang, X., Han, K., and Jiang, Z. L. (2015). A novel arc segmentation approach for document image processing. International Journal of Pattern Recognition and Artificial Intelligence, 29(01):1553001.
Al-Khaffaf, H. S. M., Talib, A. Z., and Osman, M. A. (2013). Final report of GREC’11 arc segmentation contest: Performance evaluation on multi-resolution scanned documents. In Kwon, Y.-B. and Ogier, J.-M., editors, Graphics Recognition. New Trends and Challenges, pages 187–197, Berlin, Heidelberg. Springer Berlin Heidelberg.
Al-Khaffaf, H. S. M., Talib A. Z., Osman M. A., Wong P. L. (2010) GREC’09 Arc Segmentation Contest: Performance Evaluation on Old Documents. In: Ogier JM., Liu W., Lladós J. (eds) Graphics Recognition. Achievements, Challenges, and Evolution. GREC 2009. Lecture Notes in Computer Science, vol 6020. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13728-0_23
Alwan, S., Le Caillec, J.-M., and Le Meur, G. (2019). Detection of Primitives in Engineering Drawing Using Genetic Algorithm. In ICPRAM 2019 : 8th International Conference on Pattern Recognition .Applications and Methods, Prague, Czech Republic.
Bonnici, A., Akman, A., Calleja, G., Camilleri, K. P., Fehling, P., Ferreira, A., Hermuth, F., Israel, J. H., Landwehr, T., Liu, J., and et al. (2019). Sketch-based interaction and modeling: where do we stand? Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 33(4):370-388.
Bonnici, A. and Camilleri, K. (2013). A circle-based vectorization algorithm for drawings with shadows. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SBIM:13, page 69:77, New York, NY, USA. Association for Computing Machinery.
Bukhari, S. S., Al-Khaffaf, H. S. M., Shafait, F., Osman, M. A., Talib, A. Z., and Breuel, T. M. (2014). Final report of grec’13 arc and line segmentation contest. In Lamiroy, B. and Ogier, J.-M., editors, Graphics Recognition. Current Trends and Challenges, pages 234–239, Berlin, Heidelberg. Springer Berlin Heidelberg.
Chhabra, A. K. and Phillips, I. T. (1998). The second international graphics recognition contest - raster to vector conversion: A report. Graphics Recognition, 1389:390–410.
Chhabra, A. K. and Phillips, I. T. (2000). Performance evaluation of line drawing recognition systems. In Proceedings of the 15th International Conference on Pattern Recognition, volume 4, pages 864–869, Barcelona, Spain.
Hori, O. and Doermann, D. (1995). Quantitative measurement of the performance of raster-to-vector conversion algorithms. volume 1072 of LNCS, pages 57–68.
Inoue, N. and Yamasaki, T. (2019). "Fast Instance Segmentation for Line Drawing Vectorization," 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore, Singapore, pp. 262-265, doi: 10.1109/BigMM.2019.00-14.
Kasimov, D., Kuchuganov, A., and Kuchuganov, V. (2017). Vectorization of raster mechanical drawings on the base of ternary segmentation and soft computing. Program Comput Soft, 43:337-344.
Liu, W. (2004). Report of the arc segmentation contest. In Graphics Recognition: Lecture Notes in Computer Science: Recent Advances and Perspectives, volume 3088, pages 363–366. Springer.
Liu, W., Zhai, J., Dori, D., and Tang, L. (2001). A system for performance evaluation of arc segmentation algorithms. In Proc. Third CVPR Workshop Empirical Evaluation Methods in Computer Vision.
Liu, W. Y. and Dori, D. (1997). A protocol for performance evaluation of line detection algorithms. Machine Vision and Applications, 9(5-6):240–250.
Liu, W. Y., Zhai, J., and Dori, D. (2002). Extended summary of the arc segmentation contest. In Graphics Recognition: Algorithms and Applications, volume 2390 of Lecture Notes in Computer Science, pages 343–349.
Phillips, I. T. and Chhabra, A. K. (1999). Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9):849–870.
Popov, S., Glazunov, V., Chuvatov, M., and Purii, A. (2020). "Raster to Vector Map Convertion by Irregular Grid of Heights," 2020 26th Conference of Open Innovations Association (FRUCT), Yaroslavl, Russia, pp. 386-391, doi: 10.23919/FRUCT48808.2020.9087552.
Shafait, F., Keysers, D., and Breuel, T. M. (2006). Pixel-accurate representation and evaluation of page segmentation in document images. In Proceedings of the 18th International Conference on Pattern Recognition, volume 1, pages 872–875, Hong Kong, China.
Shafait, F., Keysers, D., and Breuel, T. M. (2008). Performance evaluation and benchmarking of six-page segmentation algorithms. Ieee Transactions on Pattern Analysis and Machine Intelligence, 30(6):941–954. 286UW.
Wang, Y., Song, X., and Wang, S. (2010). Algorithm of arcs recognition based on bar tracking. In 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, volume 6, pages 2535–2540.
Wenyin, L. (2006). The third report of the arc segmentation contest. In Lecture Notes in Computer Science, volume 3926 NCS of Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pages 358–361, Hong Kong, China. Springer Verlag, Heidelberg.
Wu, J., Chen, K., and Gao, X. (2013). Fast and accurate circle detection using gradient-direction-based segmentation. J. Opt. Soc. Am. A, 30(6):1184–1192.
Zhang, Z., Wang, X., Han, K., and Jiang, Z. L. (2015). A novel arc segmentation approach for document image processing. International Journal of Pattern Recognition and Artificial Intelligence, 29(01):1553001.
Downloads
Published
2020-09-30
How to Cite
Al-Khaffaf, H. S. M. (2020). A Modified Vector Recovery Index. Science Journal of University of Zakho, 8(3), 118–122. https://doi.org/10.25271/sjuoz.2020.8.3.755
Issue
Section
Science Journal of University of Zakho
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-SA 4.0] that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work, with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online.
