ENHANCING WEBSITE USABILITY TESTING: CORRELATING EYE-TRACKING, GSR, AND SUS DATA WITH RESPECT TO GENDER PREFERENCES
DOI:
https://doi.org/10.25271/sjuoz.2024.12.1.1215Keywords:
Website Usability Testing, Galvanic Skin Response, Eye-Tracking, System Usability Scale, Gender DifferencesAbstract
Having a professional team of web developers can produce a professional website, but cannot guarantee an expected usable website. This study presents a comprehensive multilayer approach for examining the correlations between different layers of user consciousness in website usability testing. It utilizes visual attention data from eye-tracking, emotional engagement data from galvanic skin response, and self-reporting data from the system usability scale. Testing AUK and UoZ university websites with 18 users using the Gazepoint GP3 system revealed insightful correlations among different layers of user consciousness, such as high emotional engagement is associated with higher fixation counts and shorter time-to-complete and thus lower SUS scores. Whereas low emotional engagement is associated with lower fixation counts, longer time-to-complete, and thus higher SUS scores. Gender preferences verifies the results from the literature on female users generally experiencing higher emotional arousal thus having lower time-to-complete and lower SUS scores. Design problems are presented in the form of improvement recommendations. The findings of the study highlight the importance of considering different layers of user consciousness in website usability testing, as well as the importance of gender preferences. Finally, current limitations and future works are presented.
References
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003, April). Consolidating the ISO usability models. In Proceedings of 11th International Software Quality Management Conference (Vol. 2003, pp. 23-25).
Anders, S., Lotze, M., Erb, M., Grodd, W., & Birbaumer, N. (2004). Brain activity underlying emotional valence and arousal: A response‐related fMRI study. Human brain mapping, 23(4), 200-209.
Aziz, N. S., Sulaiman, N. S., Hassan, W. N. I. T. M., Zakaria, N. L., & Yaacob, A. (2021, May). A Review of Website Measurement for Website Usability Evaluation. In Journal of Physics: Conference Series (Vol. 1874, No. 1, p. 012045). IOP Publishing.
Baig, M. Z., & Kavakli, M. (2019). A survey on psycho-physiological analysis & measurement methods in multimodal systems. Multimodal Technologies and Interaction, 3(2), 37.
Bakker, J., Pechenizkiy, M., & Sidorova, N. (2011, December). What's your current stress level? Detection of stress patterns from GSR sensor data. In 2011 IEEE 11th International Conference on Data Mining Workshops (pp. 573-580). IEEE.
Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), 114-123.
Bari, D. S. (2020). Gender differences in tonic and phasic electrodermal activity components. Science Journal of University of Zakho, 8(1), 29-33.
Bari, D. S., Aldosky, H. Y. Y., Tronstad, C., Kalvøy, H., & Martinsen, Ø. G. (2018). Electrodermal responses to discrete stimuli measured by skin conductance, skin potential, and skin susceptance. Skin Research and Technology, 24(1), 108-116.
Bari, D. S., Rammoo, M. N. S., Aldosky, H. Y., Jaqsi, M. K., & Martinsen, Ø. G. (2023). The Five Basic Human Senses Evoke Electrodermal Activity. Sensors, 23(19), 8181.
Barnum, C. M. (2020). Usability testing essentials: Ready, set... test!. Morgan Kaufmann.
Bergstrom, J. R., & Schall, A. (Eds.). (2014). Eye-tracking in user experience design. Elsevier.
Berkovsky, Shlomo, et al. "Detecting personality traits using eye-tracking data." Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019.
Białowąs, S., Pieranski, B., Szyszka, A., & Reshetkova, A. (2021). Experimental design and biometric research. Toward innovations. Poznań University of Economics and Business Press. Chicago.
Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., & Ertl, T. (2014, June). State-of-the-art of visualization for eye-tracking data. In Eurovis (stars) (p. 29).
Boucsein, W. (2012). Electrodermal activity. Springer Science & Business Media.
Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale. Usability evaluation in industry, 189(3), 189-194.
Buchanan, S., & Salako, A. (2009). Evaluating the usability and usefulness of a digital library. Library Review, 58(9), 638-651.
Carter, B. T., & Luke, S. G. (2020). Best practices in eye-tracking research. International Journal of Psychophysiology, 155, 49-62.
Çınar, M. O. (2009). Eye-tracking method to compare the usability of university websites: A case study. In Human Centered Design: First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009 Proceedings 1 (pp. 671-678). Springer Berlin Heidelberg.
Clay, Viviane, Peter König, and Sabine Koenig. "eye-tracking in virtual reality." Journal of Eye Movement Research 12.1 (2019).
Critchley, H. D. (2002). Electrodermal responses: what happens in the brain. The Neuroscientist: A Review Journal Bringing Neurobiology, Neurology and Psychiatry, 8(2), 132–142.
Dawson, M. E., Schell, A. M., & Filion, D. L. (2007). The electrodermal system. Handbook of psychophysiology, 2, 200-223.
De Carolis, B., Loglisci, C., Giuseppe, M., & Trufanova, K. (2023, September). Analyzing Stress Responses Related to Usability of User Interfaces. In Proceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter (pp. 1-9).
Djamasbi, S. (2014). Eye-tracking and web experience. AIS Transactions on Human-Computer Interaction, 6(2), 37-54.
Djamasbi, S., Siegel, M., & Tullis, T. (2011). Visual hierarchy and viewing behavior: An eye-tracking study. In Human-Computer Interaction. Design and Development Approaches: 14th International Conference, HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part I 14 (pp. 331-340). Springer Berlin Heidelberg.
Foglia, P., Prete, C. A., & Zanda, M. (2008, May). Relating GSR signals to traditional usability metrics: Case study with an anthropomorphic web assistant. In 2008 IEEE Instrumentation and Measurement Technology Conference (pp. 1814-1818). IEEE.
Foglia, P., Zanda, M., & Trading, I. O. N. (2014). Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Transactions on Computers, 13, 624.
Fowles, D. C., Christie, M. J., Edelberg, R., Grings, W. W., Lykken, D. T., & Venables, P. H. (1981). Publication recommendations for electrodermal measurements. Psychophysiology, 18(3), 232-239.
Fraiwan, L., Basmaji, T., & Hassanin, O. (2018, November). A mobile mental health monitoring system: a smart glove. In 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) (pp. 235-240). IEEE.
Goldberg, J. H., & Wichansky, A. M. (2003). Eye-tracking in usability evaluation: A practitioner's guide. In the Mind's Eye(pp. 493-516). North-Holland.
Green, D., & Pearson, J. M. (2006). Development of a web site usability instrument based on ISO 9241-11. Journal of Computer Information Systems, 47(1), 66-72.
Huang, Z., & Mou, J. (2021). Gender differences in user perception of usability and performance of online travel agency websites. Technology in Society, 66, 101671.
Isokoski, P., Kangas, J., & Majaranta, P. (2018, June). Useful approaches to exploratory analysis of gaze data: enhanced heatmaps, cluster maps, and transition maps. In Proceedings of the 2018 ACM Symposium on Eye-tracking Research & Applications (pp. 1-9).
Jiang, Y. (2020). A solution to analyze mobile eye-tracking data for user research in GI Science (Master's thesis, University of Twente).
Klaib, Ahmad F., et al. "eye-tracking algorithms, techniques, tools, and applications with an emphasis on machine learning and Internet of Things technologies." Expert Systems with Applications 166 (2021): 114037.
Lin, T., Omata, M., Hu, W., & Imamiya, A. (2005, November). Do physiological data relate to traditional usability indexes?. In Proceedings of the 17th Australia conference on computer-human interaction: Citizens online: Considerations for today and the future (pp. 1-10).
McNeal, K. S., Zhong, M., Soltis, N. A., Doukopoulos, L., Johnson, E. T., Courtney, S., ... & Porch, M. (2020). Biosensors show promise as a measure of student engagement in a large introductory biology course. CBE—Life Sciences Education, 19(4), ar50.
Nielsen, J. (1999). Designing web usability: The practice of simplicity. New riders publishing.
Nielsen, J., & Pernice, K. (2010). Eyetracking web usability. New Riders.
Pernice, K., & Nielsen, J. (2009). How to conduct eyetracking studies. Nielsen Norman Group.
Rinder, J. (2012). The importance of website usability testing.
Satti, F. A., Hussain, M., Hussain, J., Kim, T. S., Lee, S., & Chung, T. (2021, January). User stress modeling through galvanic skin response. In 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM) (pp. 1-6). IEEE.
Sauro, J., & Lewis, J. R. (2011, May). When designing usability questionnaires, does it hurt to be positive?. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2215-2224).
Sharafi, Zohreh, et al. "A practical guide on conducting eye-tracking studies in software engineering." Empirical Software Engineering 25 (2020): 3128-3174.
Stahlke, Samantha N., et al. "Frontiers of immersive gaming technology: A survey of novel game interaction design and serious games for cognition." Recent Advances in Technologies for Inclusive Well-Being: Virtual Patients, Gamification and Simulation (2021): 523-536.
Tang, Wilson YF. "Application of Eye Tracker to Detect Visual Processing of Children with Autism Spectrum Disorder." Current Developmental Disorders Reports 9.4 (2022): 77-88.
Tyler, W. J., Boasso, A. M., Mortimore, H. M., Silva, R. S., Charlesworth, J. D., Marlin, M. A., ... & Pal, S. K. (2015). Transdermal neuromodulation of noradrenergic activity suppresses psychophysiological and biochemical stress responses in humans. Scientific reports, 5(1), 13865.
Wang, J., Antonenko, P., Celepkolu, M., Jimenez, Y., Fieldman, E., & Fieldman, A. (2019). Exploring relationships between eye-tracking and traditional usability testing data. International Journal of Human–Computer Interaction, 35(6), 483-494.
Wang, Q., Yang, S., Liu, M., Cao, Z., & Ma, Q. (2014). An eye-tracking study of website complexity from cognitive load perspective. Decision support systems, 62, 1-10.
Ward, R. D., Marsden, P. H., Cahill, B., & Johnson, C. (2002, April). Physiological responses to well-designed and poorly designed interfaces. In Proceedings of CHI 2002 workshop on physiological computing.
Zardari, B. A., Hussain, Z., Arain, A. A., Rizvi, W. H., & Vighio, M. S. (2021). QUEST e-learning portal: Applying heuristic evaluation, usability testing and eye tracking. Universal Access in the Information Society, 20, 531-543.
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Ismael A. Ali
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-SA 4.0] that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work, with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online.
