Veille bibliographique sur la mesure de l’engagement
[Cette veille est publiée en état “Corrigé”, pour plus d’information voir le mode d’emploi du carnet]
Cette veille a été réalisée sur Google Scholar ([la raison pour laquelle j’utilise Google Scholar c’est, car il lance la recherche sur plusieurs autres bases de données simultanément. C’est plus efficace que le faire directement sur chacun des catalogues]) et ResearchGate avec le mot clé «engagement». Ici figurent les articles permettant de comprendre le concept d’engagement dans différents domaines. J’ai choisi ceux qui se rapprochent le plus des corpus auxquels je m’intéresse : des dispositifs numériques et narratifs. Ici, j’ai repris une sélection des titres et les résumés des articles pertinents. Pour chaque élément, j’insère en premier les auteurs, la date entre parenthèses et le titre de la publication. À la ligne, on a accès au lien du fichier.
Je m’intéresse aux méthodes existantes pour évaluer l’engagement, les attributs qui décrivent le phénomène et les modèles conceptuels existants pour comprendre ce phénomène. Ainsi, je laisse une trace de ma consultation des résumés en ajoutant des parties en gras pour relever les notions concernant l’engagement et les propositions de mesures évoquées. Finalement, après chaque résumé, je qualifie la pertinence de lire l’article en entier par rapport à mon thème. Je me sers d’une échelle en 3 degrés :
- + intéressant, mais hors sujet
- ++ intéressant et dans un sujet proche
- +++ très intéressant et possiblement applicable à mon sujet
- Le symbole √ est ajouté pour moi-même, il signifie que j’ai déjà téléchargé et commencé cette lecture.
On remarque que :
- La recherche se centre sur les réponses des usagers.
- La méthode d’évaluation la plus courante est le questionnaire avec des échelles Likert. Cependant, il peut être complémenté avec l’observation des usagers, des entretiens semi-dirigés, des mesures de temps et d’accomplissement des tâches des usagers.
- L’engagement est médié, et pour cette raison il est produit par plusieurs éléments dans le dispositif évalué.
- Il y a plusieurs étapes dans un processus de l’engagement
- Plusieurs recherches évoquent la compréhension, le flow, l’intérêt, la motivation, l’immersion et le sentiment de présence comme des notions proches à l’engagement.
- Certaines des études cherchent, comme nous, à déterminer des éléments déclencheurs de l’engagement. Ces éléments vont varier selon les corpus évalués.
- Quelques-uns des corpus évalués sont : le cinéma, la lecture, les jeux éducatifs, les jeux vidéos, les sites web, la technologie.
- De toute évidence, une des principales chercheuses de ce concept est Heather O’Brien. Voici une conférence où elle parle du sujet : https://www.youtube.com/watch?v=uWKEWQjqRSg
Ci-dessous vous pourrez lire les articles ordonnés depuis le plus ancien au plus récent.
Verhoeven, Catherine E. Snow (2001) Literacy and Motivation: Reading Engagement in Individuals and Groups
The central question in this volume is how to create a society of “engaged readers” in today’s world, where reading is increasingly overruled by other media, such as television and personal computers. Engaged readers, as the term is used in this book, means readers who are socially interactive, strategic, and motivated. This state-of-the-art review contains research on integrating cognitive, social, and motivational aspects of reading and reading instruction, the chapter authors argue that coming to grips with the notion of engagement in literacy requires redefining literacy itself to acknowledge the degree to which it is not only a cognitive accomplishment, but a social activity and an affective commitment as well. Promoting literacy acquisition thus requires interventions that address attitudes and beliefs as much as those that assure cognitive changes in learners. Equally important, the authors posit that literacy engagement involves the integration of cognitive strategies and motivational goals during literate activities. This necessary link between literacy and motivation is addressed from a variety of perspectives. Acknowledging the value of cross-national and cross-cultural comparisons, the book features chapters on the promotion of literacy in different regions around the world.
+
Bardzell, Bardzell, Pace, Karnell (2008) Making user engagement visible: a multimodal strategy for interactive media experience research
https://dl.acm.org/citation.cfm?id=1358909
This paper describes an industry-academic collaborative research initiative, focused on determining useful measures of user engagement and experience with social media (e.g., video games, virtual worlds, social networking sites, etc.) and digital devices. Using newly designed hardware and software, the research initiative addresses the relationships among neurological, physiological, behavioral, and cognitive assessments of engagement in ongoing and short duration user experiences. It is a centerpiece of an iterative strategy toward understanding and modeling relationships among different engagement measures. The research will lead to design proposals for model-based assessments of engagement calibrated to individuals’ responses.
+++
Kill (2008) Foundation for Measuring Engagement in Educational Games
https://www.learntechlib.org/p/24197/
This study is part of an ongoing attempt to develop a usable and valid scale for assessing the flow experience in educational games. The purpose of the paper is to operationalize the dimensions of flow experience in an educational game context and to test a constructed GameFlow questionnaire. Furthermore, the study strives to validate flow antecedents, the factors contributing to flow experience, proposed in game design models such as the experiential gaming model. The flow antecedents and the GameFlow questionnaire were studied through a business simulation game, which was used in a university level course (N = 92). Results indicate that the flow antecedents studied contributed to the flow experience and should be considered in educational game design. Furthermore, the results provided some baseline evidence for the assumption that flow experience has a positive impact on learning. Overall, these results indicate that the experiential gaming model and the GameFlow questionnaire are useful tools for studying gaming experience and can be used in educational game design.
+
O’Brien, Toms (2008) What is user engagement? A conceptual framework for defining user engagement with technology
https://doi.org/10.1002/asi.20801
The purpose of this article is to critically deconstruct the term engagement as it applies to peoples’ experiences with technology. Through an extensive, critical multidisciplinary literature review and exploratory study of users of Web searching, online shopping, Webcasting, and gaming applications, we conceptually and operationally defined engagement. Building on past research, we conducted semistructured interviews with the users of four applications to explore their perception of being engaged with the technology. Results indicate that engagement is a process comprised of four distinct stages: point of engagement, period of sustained engagement, disengagement, and reengagement. Furthermore, the process is characterized by attributes of engagement that pertain to the user, the system, and user‐system interaction. We also found evidence of the factors that contribute to nonengagement. Emerging from this research is a definition of engagement—a term not defined consistently in past work—as a quality of user experience characterized by attributes of challenge, positive affect, endurability, aesthetic and sensory appeal, attention, feedback, variety/novelty, interactivity, and perceived user control. This exploratory work provides the foundation for future work to test the conceptual model in various application areas, and to develop methods to measure engaging user experiences.
+++ √
Busselle, Bilandzic (2008) Measuring Narrative Engagement
https://doi.org/10.1080/15213260903287259
Research indicates that the extent to which one becomes engaged, transported, or immersed in a narrative influences the narrative’s potential to affect subsequent story-related attitudes and beliefs. Explaining narrative effects and understanding the mechanisms responsible depends on our ability to measure narrative engagement in a theoretically meaningful way. This article develops a scale for measuring narrative engagement that is based on a mental models approach to narrative processing. It distinguishes among four dimensions of experiential engagement in narratives: narrative understanding, attentional focus, emotional engagement, and narrative presence. The scale is developed and validated through exploratory and confirmatory factor analyses with data from viewers of feature film and television, in different viewing situations, and from two different countries. The scale’s ability to predict enjoyment and story-consistent attitudes across different programs is presented. Implications for conceptualizing engagement with narratives as well as narrative persuasion and media effects are discussed.
+++
O’Brien, Toms (2009) The development and evaluation of a survey to measure user engagement
https://doi.org/10.1002/asi.21229
Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructed and evaluated a multidimensional scale to measure user engagement. In this paper we describe the development of the scale, as well as two large‐scale studies (N=440 and N=802) that were undertaken to assess its reliability and validity in online shopping environments. In the first we used Reliability Analysis and Exploratory Factor Analysis to identify six attributes of engagement: Perceived Usability, Aesthetics, Focused Attention, Felt Involvement, Novelty, and Endurability. In the second we tested the validity of and relationships among those attributes using Structural Equation Modeling. The result of this research is a multidimensional scale that may be used to test the engagement of software applications. In addition, findings indicate that attributes of engagement are highly intertwined, a complex interplay of user‐system interaction variables. Notably, Perceived Usability played a mediating role in the relationship between Endurability and Novelty, Aesthetics, Felt Involvement, and Focused Attention.
++
O’Brien, MacLean (2009) Measuring the User Engagement Process
Engagement is a quality of user experience that facilitates more enriching interactions with computer applications. It is defined by a core set of attributes: aesthetic appeal, novelty, involvement, focused attention, perceived usability, and endurability. The ability to engage users influences the products they purchase (e.g. cell phones), the websites they use, and the decisions they make regarding what they will use in future and what they will recommend to others. Engagement is clearly an important component of the user experience, but like other components, it is somewhat intangible, and therefore difficult to measure and evaluate. This workshop paper outlines previous research that has focused on the evaluation of engagement as an outcome of experience. We propose that focusing on measuring the process of engagement is a crucial direction for future research. In order to assess whether or not users are engaged while using an application and what aspects of the system engage them, we must employ mixed methodologies to capture the cognitive, affective, and behavioural components of the experience. But which methods are most appropriate, and how can they be used in concert? Addressing these questions will allow us to understand the nature of engagement and inform design.
++
Peter Daboll, Hongpei Zhang (2010) Literacy and Motivation: Reading Engagement in Individuals and Groups
https://patents.google.com/patent/US20110208585A1/en
Exemplary systems and methods for measurement of engagement are provided. In various embodiments, a method coprises receiving business objectives of a web site or online publisher on a server, tracking user frequency and user activities for a predetermine time, computing and ranking engagement scores with the web site based on the tracked user frequency as a function of user action categories for the predetermined time and business objectives, the user action categories being associated with the user activities, segmenting users based the engagement scores, and directing an advertisement to a user of at least one user segment.
++
Karimi, Peng Lim (2010) Children, engagement and enjoyment in digital narrative
There have been many experiments being carried out in recent years by educators and designers in the digital narrative areas which engaged children progressively in the story environment in order to ensure that the experience is fun and enjoyable while maintaining the educational values (Robertson &Good, 2005). However, with the research and experiments in place, the demand is more focused onmore engagement and enjoyment in children learning and digital playing environment. The intention of this study is to look into children’s engagement and enjoyment in a 3D digital narrative environment and to find out their likes and dislikes based on their experience. This study focuses specifically on how children interact with the Quest Atlantis and seeksto identify participants’ enjoyment level by applying the combination of Intrinsic Motivation Inventory (IMI) instrument (Ryan, 2006) and smileyometer (Read, MacFarlane, & Casey, 2002). Engagement was measured by time related factors as well as observation of each participant’s facial expression. Data sources used in this study include questionnaire, interview content, observational notes, and time records while a mixed method of qualitative and quantitative approaches was employed for analyzing the data. Findings of this study showed that in general, there was a high level of enjoyment which demonstrates a fun environment in terms of interest, enjoyment,perceived competency and perceived choice. Perceived choice was reported positive with low degree of pressure and tension. The related time duration data also showed that there was a high-level of engagement among the participants of this study. Facial expression observed from the children supported the results from the time related analysis while interview responses provided some interesting points about their enjoyment features. In this paper, the implication for digital narrative design, engagement features, and IMI scale findings will be discussed.
++
Chen, Kolko, Cuddihy, Medina (2011) Modeling but NOT measuring engagement in computer games
https://dl.acm.org/citation.cfm?id=2206383
This paper describes a study that was performed to define a model of engagement in digital games and the problems encountered with our testing methods. Drawing upon multiple disciplines, our working model of engagement was meant to help describe, predict, and analyze the conditions that create a high level of engagement in players. To refine the model and develop a methodology for studying engagement, an exploratory prototype study was performed in which participants were observed playing a pair of games (The Curse of Monkey Island and The Oregon Trail 5th Edition). This study used common usability testing methods along with a pre-and post-test modeled after Witmer and Singer’s Presence Questionnaire and a flow test at timed intervals. Unfortunately, it became clear that our testing methods needed refinement, though we believe the engagement model may still be useful as a common artifact informed from multiple disciplines.
+
Schoenau-Fog (2011) The Player Engagement Process– An Exploration of Continuation Desire in Digital Games
http://www.digra.org/wp-content/uploads/digital-library/11307.06025.pdf
Engagement is an essential element of the player experience, and the concept is described in various ways in the literature. To gain a more detailed comprehension of this multifaceted concept, and in order to better understand what aspects can be used to evaluate engaging game play and to design engaging user experiences, this study investigates one dimension of player engagement by empirically identifying the components associated with the desire to continue playing. Based on a description of the characteristics of player engagement, a series of surveys were developed to discover the components, categories and triggers involved in this process. By applying grounded theory to the analysis of the responses, a process-oriented player engagement framework was developed and four main components consisting of objectives, activities, accomplishments and affects as well as the corresponding categories of engagement, disengagement and their triggers – were identified and rank-ordered.
++
O’Brien, Toms (2013) Examining the generalizability of the User Engagement Scale (UES) in exploratory search
https://www.sciencedirect.com/science/article/abs/pii/S0306457312001124
The user experience is an integral component of interactive information retrieval (IIR). However, there is a twofold problem in its measurement. Firstly, while many IIR studies have relied on a single dimension of user feedback, that of satisfaction, experience is a much more complex concept. IIR in general, and exploratory search more specifically, are dynamic, multifaceted experiences that evoke pragmatic and hedonic needs, expectations, and outcomes that are not adequately captured by user satisfaction. Secondly, questionnaires, which are typically the means in which user’s attitudes and perceptions are measured, are not typically subjected to rigorous reliability and validity testing. To address these issues, we administered the multidimensional User Engagement Scale (UES) in an exploratory search environment to assess users’ perceptions of the Perceived Usability (PUs), Aesthetics (AE), Novelty (NO), Felt Involvement (FI), Focused Attention (FA), and Endurability (EN) aspects of the experience. In a typical laboratory-style study, 381 participants performed three relatively complex search tasks using a novel search interface, and responded to the UES immediately upon completion. We used Principal Axis Factor Analysis and Multiple Regression to examine the factor structure of UES items and the relationships amongst factors. Results showed that three of the six sub-scales (PUs, AE, FA) were stable, while NO, FI and EN merged to form a single factor. We discuss recommendations for revising and validating the UES in light of these findings.
+++
Bouvier, Lavoué, Seheba (2014) Defining Engagement and Characterizing Engaged-Behaviors in Digital Gaming
https://doi.org/10.1177/1046878114553571
Background. Although the analysis of engagement is crucial for digital entertainment or learning games, the concept of players’ or learners’ engagement is still confusing. Indeed, in digital games research, several concepts referring to the idea of engagement such as immersion, involvement, presence, and flow are used. Also, while the characterization of engaged-behaviors may be useful for designers or teachers in assessing players’ or learners’ engagement, the nature and the scope of these behaviors are still unclear.
Aim. In this article, based on a multidisciplinary state of the art on the concept of engagement, we define and delineate the concepts related to digital gaming engagement.
Results. We characterize engaged-behaviors by identifying four types of engagement: environmental, social, self, and action. We thus refine, disambiguate, and characterize the concepts of engagement and engaged-behaviors. This work therefore constitutes an effective support for analyzing, designing, assessing, and personalizing engaging activities in digital games.
+++
Wiebe, Lamb, Hardy, Sharek (2014) Measuring engagement in video game-based environments: Investigation of the User Engagement Scale
https://doi.org/10.1016/j.chb.2013.12.001
This research investigated the use of the User Engagement Scale (UES) as a psychometric tool to measure engagement during video game-play. Exploratory factor analysis revealed four factors (Focused Attention, Perceived Usability, Aesthetics, and Satisfaction) as compared to the six found in the original development of the UES. In the context of video game-play, a revised UES (UESz) demonstrated better psychometric properties than the original UES defined by six subscales, including enhanced reliability. Further validity analysis included comparisons with the Flow State Scale (FSS), showing the complementary nature of the two scales and what constructs both scales might be measuring in a video game context. Criterion validity analysis demonstrated that UESz was more predictive of game performance than the FSS. Findings related to both the UESz and FSS were discussed relative to an overarching framework of hedonic and utilitarian qualities of game-play.
++
Lalmas, O’Brien, Yom-Tov (2014) Measuring User Engagement : Synthesis Lectures on Information Concepts, Retrieval, and Services
November 2014, 132 pages, (https://doi.org/10.2200/S00605ED1V01Y201410ICR038)
User engagement refers to the quality of the user experience that emphasizes the positive aspects of interacting with an online application and, in particular, the desire to use that application longer and repeatedly. User engagement is a key concept in the design of online applications (whether for desktop, tablet or mobile), motivated by the observation that successful applications are not just used, but are engaged with. Users invest time, attention, and emotion in their use of technology, and seek to satisfy pragmatic and hedonic needs. Measurement is critical for evaluating whether online applications are able to successfully engage users, and may inform the design of and use of applications. User engagement is a multifaceted, complex phenomenon; this gives rise to a number of potential measurement approaches.
Common ways to evaluate user engagement include using self-report measures, e.g., questionnaires; observational methods, e.g. facial expression analysis, speech analysis; neuro-physiological signal processing methods, e.g., respiratory and cardiovascular accelerations and decelerations, muscle spasms; and web analytics, e.g., number of site visits, click depth. These methods represent various trade-offs in terms of the setting (laboratory versus “in the wild”), object of measurement (user behaviour, affect or cognition) and scale of data collected. For instance, small-scale user studies are deep and rich, but limited in terms of generalizability, whereas large-scale web analytic studies are powerful but negate users’ motivation and context. The focus of this book is how user engagement is currently being measured and various considerations for its measurement. Our goal is to leave readers with an appreciation of the various ways in which to measure user engagement, and their associated strengths and weaknesses. We emphasize the multifaceted nature of user engagement and the unique contextual constraints that come to bear upon attempts to measure engagement in different settings, and across different user groups and web domains. At the same time, this book advocates for the development of “good” measures and good measurement practices that will advance the study of user engagement and improve our understanding of this construct, which has become so vital in our wired world.
+++
Nisa Bakkalbasi, Damon Jaggars , Barbara Rockenbach (2015) Re-skilling for the digital humanities: measuring skills, engagement, and learning
https://www.emerald.com/insight/content/doi/10.1108/LM-09-2014-0109/full/html
The purpose of this paper is to describe an assessment design for the Developing Librarian training program. The Developing Librarian training program created by and for librarians and professional staff in the Humanities and History division is a two-year training program to acquire new skills and methodologies to support the digital humanities. The program is based on the assumption that learning must happen in context; therefore the training is project based with all participants engaged in building a digital humanities research site as a team. This approach enables participants to learn about new tools in a sustained manner that parallels the way humanities researchers are likely to use them.
++
O’Brien, Freund, Kopak (2016) Investigating the Role of User Engagement in Digital Reading Environments
https://dl.acm.org/citation.cfm?id=2854973
User engagement is recognized as an important component of the user experience, but relatively little is known about the effect of engagement on the learning outcomes of such interactions. This experimental user study examines the relationship between user engagement (UE) and comprehension in varied academic reading environments. Forty-one university students interacted with one of two sets of texts presented in 4 conditions in the context of preparing for a class assignment. Employing the User Engagement Scale (UES), we found evidence of a relationship between students’ comprehension of the texts and their degree of engagement with them. However, this association was confined to one of the UES subscales and was not consistent across levels of engagement. An examination of additional variables found little evidence that system and content characteristics influenced engagement; however, we noted that all students’ reported increased knowledge, but topical interest for non-engaged students declined. Results contribute to existing literature by adding further evidence that the relationship between engagement and comprehension is complex and mediated.
+
O’Brien, Cairn (2016) Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital Media
Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital MediaHeather O’Brien, Paul Cairns Springer, May 30, 2016 – Computers – 222 pages 0 Reviews User Engagement (UE) is a complex concept to investigate. The purpose of this book is not to constrain UE to one perspective, but to offer a well-rounded appreciation for UE across various domains and disciplines. The text begins with two foundational chapters that describe theoretical and methodological approaches to user engagement; the remaining contributions examine UE from different disciplinary perspectives and across a range of computer-mediated environments, including social and communications media, online search, eLearning, games, and eHealth. The book concludes by bringing together the cross-disciplinary perspectives presented in each chapter and proposing an agenda for future research in this area. The book will appeal to established and emerging academic and industry researchers looking to pursue research and its challenges. This includes scholars at all levels with an interest in user engagement with digital media, from students to experienced researchers, and professionals in the fields of computer science, web technology, information science, museum studies, learning and health sciences, human-computer interaction, information architecture and design, and creative arts. Preview this book »
+
O’Brien, Cairns, Hall (2018) A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form
https://doi.org/10.1016/j.ijhcs.2018.01.004
User engagement (UE) and its measurement have been of increasing interest in human-computer interaction (HCI). The User Engagement Scale (UES) is one tool developed to measure UE, and has been used in a variety of digital domains. The original UES consisted of 31-items and purported to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. A recent synthesis of the literature questioned the original six-factors. Further, the ways in which the UES has been implemented in studies suggests there may be a need for a briefer version of the questionnaire and more effective documentation to guide its use and analysis. This research investigated and verified a four-factor structure of the UES and proposed a Short Form (SF). We employed contemporary statistical tools that were unavailable during the UES’ development to re-analyze the original data, consisting of 427 and 779 valid responses across two studies, and examined new data (N=344) gathered as part of a three-year digital library project. In this paper we detail our analyses, present a revised long and short form (SF) version of the UES, and offer guidance for researchers interested in adopting the UES and UES-SF in their own studies.
+++ √
OpenEdition vous propose de citer ce billet de la manière suivante :
Ximena Miranda B. (12 septembre 2019). Veille bibliographique sur la mesure de l’engagement. Base de donnée et interface de lecture. Consulté le 4 décembre 2024 à l’adresse https://doi.org/10.58079/ngyp
Ma méthode de travail est très modulaire et schématique. Elle me permet d’ordonner le VRAC de mes pensées. Je rédige en trois étapes «idées principales», «réécriture» et «correction». Pour me maintenir active, je publie aussi mes billets par étapes.